where to host small GPU machine learning API cheaply

Question

I am trying to make some new web APIs that would strongly benefit from GPU processing for completed/trained machine learning models. I would like to make this a publicly available endpoint. However I don't know where is a realistic place to host a machine learning hobby project API? If there is a better way (i.e. only use the GPU while processing an API request which would be infrequent) I'm also open to that.

AWS Pricing https://aws.amazon.com/ec2/pricing/on-demand/

The cheapest I can see is $0.50/hr which is around $350 monthly

Google Cloud Pricing https://cloud.google.com/compute/gpus-pricing

Cheapest I can see is $180 monthly

Vast AI pricing https://vast.ai/console/create/

Cheapest i've found is $0.077/hr which is $56 a month

And I found this quora post https://www.quora.com/Which-cloud-hosting-provides-GPU-servers-at-the-lowest-cost which pointed me to https://www.paperspace.com/pricing which is an $8/mo solution but I'm not sure if this is actually server hosting.

Answer 1

There are different possibility:

• Google Colab: It's free, includes Jupyter notebook system with a nice user interface. It's integrated with Google Drive and GitHub. And you can collaborate on it. Both GPU and TPU is available.
• GoogleCloud: More powerful and customizable than Google Colab is. GPU and TPU are available.
• Preemptible instances on GoogleCloud: Preemptible VMs offer the same machine types and options as regular compute instances and last for up to 24 hours. They are fine for most learning tasks. It can reduce your Compute Engine costs by up to 80% against normal GoogleCloud!
• AWS EC2: Even if the configuration is not the easiest one, the spot instances pricing offers a way to spare up to 90% against on-demand prices.

Update:

• Vast.ai: GPU Sharing Economy with a market place
• Paperspace has a free offer for hobbyist too with 5GB persistent storage and 6 hours running time. Other plans are available (8$/month for 200GB permanent storage). Hourly price for computing is available there (GPU: 0.5-2.9$/hour for most of the GPU)

Answer 2

Check VPS Smart. I think it's one of the best choice for small projects. It's starting from $45.00.

Answer 3

If you are planning to host your ML project API then you I'd recommend going with a platform that has K8s serving capability. So that it can auto manage scaling up/down for you on-demand as and when there is load on your API.

Otherwise you'd end up scrambling for resources online. I think ovhcloud and Alibabacloud provide managed Kubernetes offering and could potentially be less costly for your requirement.

And if your sole purpose is to train ML models then you can explore services such as:

1. Q Blocks - Decentralized GPU computing for ML: 80% less costly
2. Paperspace - GPU optimized platform for ML
3. Google colab - Free but there are a lot of limitations

Answer 4

You can check Jarvislabs.ai, we have GPU starting at 0.49$ for RTX 5000. Since you may have occasional loads you could also automate the process through a simple API and reduce your cost further.

Disclaimer: I am the founder of the startup.