Understanding Sagemaker Neo

Question

I have few questions for Sagemaker Neo:

1) Can I take advantage of Sagemaker Neo if I have an externally trained tensorflow/mxnet model?

2) Sagemaker provides container image for 'image-classification' and it has released a new image with name 'image-classification-neo' for the neo compilation job. What is the difference between both of them? Do I require a new Neo compatible image for each pre built sagemaker template(container) similarly?

Any help would be appreciated

Thanks!!

Answer 1

It is long time that this question has been asked. But in case someone turns up here after searching for the same question:

Mainly, Amazon NEO is an optimizer for making the program compatible for multiple underlying hardware and platform. Based on documentation: "Neo is a new capability of Amazon SageMaker that enables machine learning models to train once and run anywhere in the cloud and at the edge. "

And yes, those 2 docker images are different. As one of them has the optimiser code, the other doesn't. The difference is not in the input, so 'image-classification-neo' can work with images that 'image-classification' can work. But the output is different. The output of 'image-classification-neo' can be used on multiple platforms. you can check out the supported hardware platforms in the link below: https://docs.aws.amazon.com/sagemaker/latest/dg/neo.html

Answer 2

1) Yes. Upload your model to an S3 bucket as a model.tar.gz file (similar to what SageMaker would save after training) and you can compile it.

2) The Neo versions use the Neo runtime to load and predict, so yes, the containers are different. Right now, Neo supports the XGBoost and Image Classification built-in algos. Of course, you could build your own custom container and use Neo inside that. For more info: https://docs.aws.amazon.com/sagemaker/latest/dg/neo.html

Julien