I am trying to use Elmo Embedding using TensorFlow 2 and Getting this error for model.fit command: 'NoneType' object has no attribute 'outer_context'

Question

Importing Elmo Embedding Layer from TF-hub Using TF 2

# Imported Elmo Layer
elmo_model_path = "https://tfhub.dev/google/elmo/3"
elmo_layer = hub.KerasLayer(elmo_model_path, input_shape=[], dtype=tf.string, trainable=False)

# Creating Model

model = tf.keras.Sequential([
    elmo_layer,
    tf.keras.layers.Dense(8, activation='sigmoid'),
    tf.keras.layers.Dense(1, activation='sigmoid')
])

Training

num_epochs = 5
history = model.fit(training_data.shuffle(10000).batch(2), epochs=num_epochs, verbose=2)

The Data I am using:

data = ['our deeds reason earthquake may allah forgive us', 'forest fire near la ronge sask canada', 'all residents asked shelter place notified officers no evacuation shelter place orders expected', ' people receive wildfires evacuation orders california', 'just got sent photo ruby alaska smoke wildfires pours school', 'rockyfire update  california hwy  closed directions due lake county fire  cafire wildfires', 'flood disaster heavy rain causes flash flooding streets manitou colorado springs areas', 'im top hill i can see fire woods', 'theres emergency evacuation happening now building across street', 'im afraid tornado coming area', 'three people died heat wave far']
['our deeds reason earthquake may allah forgive us', 'forest fire near la ronge sask canada', 'all residents asked shelter place notified officers no evacuation shelter place orders expected', ' people receive wildfires evacuation orders california', 'just got sent photo ruby alaska smoke wildfires pours school']
label = ['1', '1', '1', '1', '1']

#converting the labels to int value
label = list(map(np.int64, label))

#Creating Training Dataset
training_data = tf.data.Dataset.from_tensor_slices((data,label)).prefetch(1)

print(type(training_data))
print(training_data)

The Error I am getting: From the error it seems There is data-structure error or a shape miss-match, But I am not sure

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_24964/3287467954.py in <module>
      1 num_epochs = 5
----> 2 history = model.fit(training_data.shuffle(10000).batch(2), epochs=num_epochs, verbose=2)

~\anaconda3\lib\site-packages\keras\utils\traceback_utils.py in error_handler(*args, **kwargs)
     65     except Exception as e:  # pylint: disable=broad-except
     66       filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67       raise e.with_traceback(filtered_tb) from None
     68     finally:
     69       del filtered_tb

~\anaconda3\lib\site-packages\tensorflow\python\framework\func_graph.py in autograph_handler(*args, **kwargs)
   1145           except Exception as e:  # pylint:disable=broad-except
   1146             if hasattr(e, "ag_error_metadata"):
-> 1147               raise e.ag_error_metadata.to_exception(e)
   1148             else:
   1149               raise

AttributeError: in user code:

    File "C:\Users\saika\anaconda3\lib\site-packages\keras\engine\training.py", line 1021, in train_function  *
        return step_function(self, iterator)
    File "C:\Users\saika\anaconda3\lib\site-packages\keras\engine\training.py", line 1010, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "C:\Users\saika\anaconda3\lib\site-packages\keras\engine\training.py", line 1000, in run_step  **
        outputs = model.train_step(data)
    File "C:\Users\saika\anaconda3\lib\site-packages\keras\engine\training.py", line 863, in train_step
        self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
    File "C:\Users\saika\anaconda3\lib\site-packages\keras\optimizer_v2\optimizer_v2.py", line 530, in minimize
        grads_and_vars = self._compute_gradients(
    File "C:\Users\saika\anaconda3\lib\site-packages\keras\optimizer_v2\optimizer_v2.py", line 583, in _compute_gradients
        grads_and_vars = self._get_gradients(tape, loss, var_list, grad_loss)
    File "C:\Users\saika\anaconda3\lib\site-packages\keras\optimizer_v2\optimizer_v2.py", line 464, in _get_gradients
        grads = tape.gradient(loss, var_list, grad_loss)

    AttributeError: 'NoneType' object has no attribute 'outer_context'

Any help would be appreciated.

Answer 1

I was able to replicate this error and found the fixes as below:

1. You need to run this code by selecting Tensorflow 1.x.
2. The dimensions of input and output should be the same to train the model (otherwise it will show this error "ValueError: Dimensions 11 and 5 are not compatible").
3. Also, You must compile your model before training/testing.

Please check this below fixed code:

%tensorflow_version 1.x

!pip install tensorflow_hub

import tensorflow as tf
import tensorflow_hub as hub
import numpy as np
print(tf.__version__)


elmo_model_path = "https://tfhub.dev/google/elmo/3"
elmo_layer = hub.KerasLayer(elmo_model_path, input_shape=[], dtype=tf.string, trainable=False)
    
data = ['rockyfire update  california hwy  closed directions due lake county fire  cafire wildfires', 'flood disaster heavy rain causes flash flooding streets manitou colorado springs areas', 'theres emergency evacuation happening now building across street', 'im afraid tornado coming area', 'three people died heat wave far']
['our deeds reason earthquake may allah forgive us', 'forest fire near la ronge sask canada', 'all residents asked shelter place notified officers no evacuation shelter place orders expected', 'people receive wildfires evacuation orders california', 'just got sent photo ruby alaska smoke wildfires pours school']
label = ['1', '1', '1', '1', '1']

#converting the labels to int value
label = list(map(np.int64, label))

#Creating Training Dataset
training_data = tf.data.Dataset.from_tensor_slices((data,label))

print(type(training_data))

model = tf.keras.Sequential([
    elmo_layer,
    tf.keras.layers.Dense(8, activation='sigmoid'),
    tf.keras.layers.Dense(1, activation='sigmoid')
])

model.compile(optimizer='adam', loss=tf.compat.v1.keras.losses.BinaryCrossentropy(from_logits=True),
              metrics=[tf.compat.v1.keras.metrics.BinaryAccuracy(threshold=0.0, name='accuracy')])

num_epochs = 5
history = model.fit(training_data.shuffle(10000).batch(2), epochs=num_epochs, verbose=2)

Output:

Train on 3 steps
Epoch 1/5
3/3 - 1s - loss: 0.4621 - accuracy: 1.0000
Epoch 2/5
3/3 - 0s - loss: 0.4337 - accuracy: 1.0000
Epoch 3/5
3/3 - 0s - loss: 0.4143 - accuracy: 1.0000
Epoch 4/5
3/3 - 0s - loss: 0.3986 - accuracy: 1.0000
Epoch 5/5
3/3 - 0s - loss: 0.3882 - accuracy: 1.0000