I have a trained net in tensorflow that i wish to use in gcloud ml-engine serving for prediction.
Predict gcloud ml serving should accept numpy array float32 type images with size of 320x240x3 and return 2 tiny matrices as an output.
Does anyone knows how should i create the input layers that would accept this kind of input type?
I have tried multiple ways, for example using base64 encoded json files, but casting the string into float type produces an error in which it's not supported:
"error": "Prediction failed: Exception during model execution: LocalError(code=StatusCode.UNIMPLEMENTED, details=\"Cast string to float is not supported\n\t [[Node: ToFloat = Cast[DstT=DT_FLOAT, SrcT=DT_STRING, _output_shapes=[[-1,320,240,3]], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](ParseExample/ParseExample)]]\")"
This is an example of creating the json file (after saving the numpy array above as jpeg):
python -c 'import base64, sys, json; img = base64.b64encode(open(sys.argv[1], "rb").read()); print json.dumps({"images": {"b64": img}})' example_img.jpg &> request.json
And the tensorflow commands attempting to handle the input:
raw_str_input = tf.placeholder(tf.string, name='source')
feature_configs = {
'image': tf.FixedLenFeature(
shape=[], dtype=tf.string),
}
tf_example = tf.parse_example(raw_str_input, feature_configs)
input = tf.identity(tf.to_float(tf_example['image/encoded']), name='input')
the above is an example of one of the tests done, also tried multiple attempts of different tensorflow commands to handle the input but none of them worked...
I would recommended not using
parse_example
to start with. There are several options for sending image data, each with tradeoffs in complexity and payload size:In each case, it is important to note that the input placeholders must have 'None' as the outer-dimension of their shape. This is the "batch_size" dimension (required, even if you intend to send images one-by-one to the service).
Raw Tensor Encoded as JSON
The JSON you send to the service will look like as documented (see "Instances JSON string"). For example, (I recommend removing as much white space as possible; pretty printed here for readability):
Please note that
gcloud
builds that request body from an input file format where each input is on a separate line (and most be packed on a single line), i.e.:Tensors Packed as Byte Strings
If you're doing resizing, etc. on the client, my recommendation is to send a byte string. JSON can be a fairly inefficient way to send floats over the wire; even sending integer data causes bloat. Instead, you can encode the bytes on the client and decode them in TensorFlow. My recommendation is to use
uint8
data.This is the TensorFlow Model code to decode bytes strings:
One special note here: as pointed out by Jeremy Lewi, the name of this input alias must end in
_bytes
(image_bytes
). This is because JSON doesn't have a way of distinguish text form binary data.Note that the same trick can be applied to float data, not just uint8 data.
Your client would be responsible for creating a bytes string of uint8s. Here's how you would do that in Python using
numpy
.Compressed Image Data
It is often most convenient to send the original images and do the resizing and decoding in TensorFlow. This is exemplified in this sample, which I won't repeat here. The client simply needs to send the raw JPEG bytes. Same note about
_bytes
suffix applies here.