Using Sagemaker Python SDK, I am deploying a pre trained model on tensorflow.serving endpoint with a custom inference.py script via the entry point parameter
model = Model(role='xxx', framework_version='2.1.1', entry_point='inference.py',
model_data='xxx')
predictor = model.deploy(instance_type='xxx', initial_instance_count=1, endpoint_name='xxx')
inference.py constains of an input_handler and an output_handler functions, but when i call predict with:
text=“how are you”
input = {
'instances': [text]
}
predictions = predictor.predict(input)
I'm getting the following error:
botocore.errorfactory.ModelError: An error occurred (hModelError) when calling the InvokeEndpoint operation: Received client error (400) from model with message "{"error": "Failed to process element: 0 of 'instances' list. Error: Invalid argument: JSON Value: "xxx" Type: String is not of expected type: float" }"
It seems the function is never calling the input_handler function in inference.py script. Do you know why this might be happening? How can I make my predict to accept text string? I am using the inout_handler to tokenise the input.