ML-Engine Local Predict - Failed to Run Model

Question

gcloud ml-engine local predict fails

First I identified the required input.json structure with saved_model_cli show --all --dir saved_model/ Response:

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
    dtype: DT_UINT8
    shape: (-1, -1, -1, 3)
    name: image_tensor:0
....

From this I formatted my input.json for gcloud ml-engine local predict as:

{"inputs": {"b64": "ENCODED"}}
...

Finally, I ran gcloud ml-engine local predict --model-dir saved_model/ --json-instances=PATH-TO-INPUTS.json

Response:

ERROR: (gcloud.ml-engine.local.predict) /usr/local/lib/python2.7/dist-packages/requests/__init__.py:83:
2018-10-03 09:20:06.598090: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
ERROR:root:Exception during running the graph: invalid literal for long() with base 10: '\xff\xd8\xff\xdb'
Traceback (most recent call last):
  File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/command_lib/ml_engine/local_predict.py", line 172, in <module>
    main()
  File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/command_lib/ml_engine/local_predict.py", line 167, in main
    signature_name=args.signature_name)
  File "/usr/lib/google-cloud-sdk/lib/third_party/ml_sdk/cloud/ml/prediction/prediction_lib.py", line 106, in local_predict
    predictions = model.predict(instances, signature_name=signature_name)
  File "/usr/lib/google-cloud-sdk/lib/third_party/ml_sdk/cloud/ml/prediction/prediction_utils.py", line 233, in predict
    preprocessed, stats=stats, **kwargs)
  File "/usr/lib/google-cloud-sdk/lib/third_party/ml_sdk/cloud/ml/prediction/frameworks/tf_prediction_lib.py", line 350, in predict
    "Exception during running the graph: " + str(e))
cloud.ml.prediction.prediction_utils.PredictionError: Failed to run the provided model: Exception during running the graph: invalid literal for long() with base 10: '\xff\xd8\xff\xdb' (Error code: 2)

Any help overcoming this obstacle would be great. I have not been able to identify a solution from research online thus far. Thank you!

Answer 1

The data in input.json does not match the shape of inputs['inputs']. You have not provided enough information about what the various dimensions represent, but I suspect this is NHWC (batch size x height x width x channels) encoding of an image.

I also suspect these are supposed to be raw pixel values. In which case, you should not be base64 encoding the values, i.e., you should send data like this:

{"inputs": [[[0, 0, 0], ... [0, 0, 0]]]}

That being said, you should consider sending the image as a byte string and decoding the image in the graph. More information about various approaches can be found here:

https://stackoverflow.com/a/46222990/1399222

It seems you are using "Raw Tensor Encoded as JSON" and I recommend "Compressed Imaged Data", or the slightly simpler "Tensors Packed as Byte Strings"