Possible to outsource computations to AWS and utilize results locally?

Question

I'm working on a robot that uses a CNN that needs much more memory than my embedded computer (Jetson TX1) can handle. I was wondering if it would be possible (with an extremely low latency connection) to outsource the heavy computations to EC2 and send the results back to the be used in a Python script. If this is possible, how would I go about it and what would the latency look like (not computations, just sending to and from).

Answer 1

I think it's certainly possible. You would need some scripts or a web server to transfer data to and from. Here is how I think you might achieve it:

1. Send all your training data to an EC2 instance
2. Train your CNN
3. Save the weights and/or any other generated parameters you may need
4. Construct the CNN on your embedded system and input the weights from the EC2 instance. Since you won't be needing to do any training here and won't need to load in the training set, the memory usage will be minimal.
5. Use your embedded device to predict whatever you may need

It's hard to give you an exact answer on latency because you haven't given enough information. The exact latency is highly dependent on your hardware, internet connection, amount of data you'd be transferring, software, etc. If you're only training once on an initial training set, you only need to transfer your weights once and thus latency will be negligible. If you're constantly sending data and training, or doing predictions on the remote server, latency will be higher.

Answer 2

Possible: of course it is.

You can use any kind of RPC to implement this. HTTPS requests, xml-rpc, raw UDP packets, and many more. If you're more interested in latency and small amounts of data, then something UDP based could be better than TCP, but you'd need to build extra logic for ordering the messages and retrying the lost ones. Alternatively something like Zeromq could help.

As for the latency: only you can answer that, because it depends on where you're connecting from. Start up an instance in the region closest to you and run ping, or mtr against it to find out what's the roundtrip time. That's the absolute minimum you can achieve. Your processing time goes on top of that.

Answer 3

I am a former employee of CENAPAD-UFC (National Centre of HPC, Federal University of Ceará), so I have something to say about outsourcing computer power.

CENAPAD has a big cluster, and it provides computational power for academic research. There, professors and students send their computation and their data, defined the output and go drink a coffee or two, while the cluster go on with the hard work. After lots of flops, the operation ended and they retrieve it via ssh and go back to their laptops.

For big chunks of computation, you wish to minimize any work that is not a useful computation. One such thing is commumication over detached computers. If you need to know when the computation has ended, let the HPC machine tell you that.

To compute stuff effectively, you may want to go deeper in the machine and performe some kind of distribution. I use OpenMP to distribute computation inside the same machine/thread distribution. To distribute between physically separated computers, but next (latency speaking), I use MPI. I have installed also another cluster in UFC for another department. There, the researchers used only MPI.

Maybe some read about distributed/grid/clusterized computing helps you:

• https://en.m.wikipedia.org/wiki/SETI@home ; the first example of massive distributed computing that I ever heard
• https://en.m.wikipedia.org/wiki/Distributed_computing
• https://en.m.wikipedia.org/wiki/Grid_computing
• https://en.m.wikipedia.org/wiki/Supercomputer ; this is CENAPAD like stuff

In my opinion, you wish to use a grid-like computation, with your personal PC working as a master node that may call the EC2 slaves; in this scenario, just use communication from master to slave to send program (if really needed) and data, in such a way that the master will have another thing to do not related with the sent data; also, let the slave tells your master when the computation reached it's end.