Predicting on GCP Ai Platform

Question

I deployed a tensorflow model on GCP AI Platform. This model predicts whether a text is sarcasm (1) or not (0).

A text is represented (with a given function "tokenize_text") as two tensors. That could look like this:

text = tokenize_text('This is a text')
print(text)


>>> <tf.Tensor: shape=(1, 512), dtype=int32, numpy=
array([[  101, 71284, 92947, 11962, 10168, 12830,   102,     0,...]],
array([[1, 1, 1, 1, 1, 1, 1, 0, 0, ...]]>

Furthermore

model.predict(text)    #result: not sarcasm (4%)


>>> array([[0.04065517]], dtype=float32)

Now I want to do the same thing on the same model but on GCP AI Platform. Therefore, the input ("text") will be wrapped up in a JSON as the model only works with JSON files. But I get the following error:

TypeError: Object of type EagerTensor is not JSON serializable

I knew that Tensors are not directly convertable to JSON. However, I only used tensors for prediction before deploying on GCP.

Do you have any ideas/approaches?

Answer 1

I found a solution to this issue. If you are using tensors for your model to predict, simply name your input of your model before training.

For example:

 # First tensor 
 tf.keras.layers.Input(shape (max_length,),name='input_ids',dtype='int32')
 
 # Second Tensor
 tf.keras.layers.Input(shape (max_length,),name='attention_mask',dtype='int32')

Now my two tensors need to be named "input_ids" and "attention_mask". Naming tensors makes sense if you use multiple tensor. For instance, BERT models use atleast two tensors for representing a text. Save and deploy your model in GCP Ai Platform.

Now you can predict. First change your tensors into lists and name them for the json file like this:

instances = [{
      'input_ids': [int(v.numpy()) for v in list(text[0][0])],
      'attention_mask': [int(v.numpy()) for v in list(text[1][0])]}]

print({'instances': instances})

Your json file should look something like this (sorry for the format):

{
    'instances': [{'input_ids':[101,10216,10339,10299,58606,102,0,0,0,0],
                   'attention_mask': [1, 1, 1, 1, 1, 1, 0, 0, 0, 0]}]
}

You can directly test your model based on that output or use example function to automate your request which already is provided by Google Ai platform. For this go to your model and the corresponding version and go to test and application. Now click on Use provided model.

Hopefully this helps if someone encounters the same problem as I did.