Pass map, slice over channel and over network?

Question

What is the best way to pass slice and map structure over channel that is distributed over network? I need to distribute the application running over several EC2 instances and wonder how I can achieve this by communicating each application by Go channel.

Here's the workflow that I would like to run:

1. Process data in one application
2. Distribute the data into 10 replica applications
3. Each 10 application does its job in a separate EC2 instance
4. Once they are all done, they send the result back to the original program
5. This is sent over the channel

Please let me know. Thanks!

Answer 1

If depends on the format you will chose for the serialization.

One well-suited for over-the-network communication is MessagePack (an efficient binary serialization format. It lets you exchange data among multiple languages like JSON. But it's faster and smaller)

A Go library like philhofer/msgp can serializaze any struct (like one with a map), including composite types like maps and arrays.

However, it uses Go1.4 go generate command. (go 1.4rc1 is already out)

From there, a library like docker/libchan can help: Libchan is an ultra-lightweight networking library which lets network services communicate in the same way that goroutines communicate using channels.