Placeholder tensors require a value in ml-engine predict but not local predict

Question

I've been developing a model for use with the cloud ML engine's online prediction service. My model contains a placeholder_with_default tensor that I use to hold a threshold for prediction significance.

threshold = tf.placeholder_with_default(0.01, shape=(), name="threshold")

I've noticed that when using local predict:

gcloud ml-engine local predict --json-instances=data.json --model-dir=/my/model/dir

I don't need to supply values for this tensor. e.g. this is a valid input:

{"features": ["a", "b"], "values": [10, 5]}

However when using online predict:

gcloud ml-engine predict --model my_model --version v1 --json-instances data.json

If I use the above JSON I get an error:

{
    "error": "Prediction failed: Exception during model execution: AbortionError(code=StatusCode.INVALID_ARGUMENT, details=\"input size does not match signature\")"
}

However if I include threshold, then I don't. e.g:

{"features": ["a", "b"], "values": [10, 5], "threshold": 0.01}

Is there a way to have "threshold" be an optional input?

Thanks

Matthew

Answer 1

Looks like currently it's not possible in CloudML. If you're getting predictions from a JSON file, you need to add the default values explicitly (like you did with "threshold": 0.01).

In Python I'm just dynamically adding the required attributes before doing the API request:

def add_empty_fields(instance):
    placeholder_defaults = {"str_placeholder": "", "float_placeholder": -1.0}
    for ph, default_val in placeholder_defaults.items():
        if ph not in instance:
            instance[ph] = default_val

which would mutate the instance dict that maps placeholder names to placeholder values. For a model with many optional placeholders, this is a bit nicer than manually setting missing placeholder values for each instance.