Sagemaker inference.py input_handler not working

29 Views Asked by At

Using Sagemaker Python SDK, I am deploying a pre trained model on tensorflow.serving endpoint with a custom inference.py script via the entry point parameter

model = Model(role='xxx',                 framework_version='2.1.1',            entry_point='inference.py',
          model_data='xxx')

predictor = model.deploy(instance_type='xxx',                        initial_instance_count=1,                      endpoint_name='xxx')

inference.py constains of an input_handler and an output_handler functions, but when i call predict with:

text=“how are you”

input = {
'instances': [text]
}

predictions = predictor.predict(input)

I'm getting the following error:

botocore.errorfactory.ModelError: An error occurred (hModelError) when calling the InvokeEndpoint operation: Received client error (400) from model with message "{"error": "Failed to process element: 0 of 'instances' list. Error: Invalid argument: JSON Value: "xxx" Type: String is not of expected type: float" }"

It seems the function is never calling the input_handler function in inference.py script. Do you know why this might be happening? How can I make my predict to accept text string? I am using the inout_handler to tokenise the input.

0

There are 0 best solutions below