Node.JS scalable (using Node.JS-cluster) socket-server with Engine.IO, pairing/syncing (also called second screen) two devices, on same worker…

Question

I have a question… I want to run a NodeJS socket-server (my decision is to go with EngineIO, from reading up on performance / cross-browser / etc…) and be able to scale up (adding cores). Therefore I want to user cluster-module (even though in experimental stage, it seems like everybody says its reliable). I looked into using a load-balancer in nginx instead, but it seems a bit too complicated for this project (everywhere this is mentioned to be the advanced option).

So my big problem that I need to solve is this: I need to have two different devices on the same thread, so that they can “talk” (second screen solution > for example this: https://www.google.com/intl/en/chrome/browser/promo/supersync/). Anyway, EngineIO doesnt seem to support shared storage (eg shared RedisStore), and I think its a bit too advanced to implement myself…

So I’m thinking about a setup where I put each process worker on a different port, and then, when one of the devices has connected to the EngineIO instance on that specific worker (load balanced by nodejs cluster), it can show a unique code (that should be input on the second device), where the last digit will let the second device know what port to connect to. I figured out that I need sticky-session for the connection of each device to stay on the same thread (or at least the first device, since I dont know the port of that worker when I connect the EngineIO.

I managed to start multiple workers on different ports... But something feels wrong. I haven't found any examples on developers implementing this kind of structure. Maybe that doesnt make it wrong, but I feel like I should get someones opinion on this. So my question is: is it a good idea to have the different worker processes on different ports, to be able to know which port to connect to with the second device? Something like this:

var cluster = require('cluster'),
http = require('http');

var numCPUs = require('os').cpus().length; // or just hardcode to 2, when not more than one cpu is available...

if (cluster.isMaster) {
  // Fork workers.
  for (var i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  cluster.on('listening', function(worker, address) {
    console.log("worker.id: " +  worker.id);
  });
} else {
  console.log("cluster.worker.id: " +  cluster.worker.id);

  http.createServer(function(req, res) {
    res.writeHead(200);
    res.end("hello world " + cluster.worker.id + "\n");
  }).listen(8000 + parseInt(cluster.worker.id));
}

And, of course, the EngineIO is an important part of the solution, that is not incorporated in the code above, but thats next step.

Answer 1

You could try SocketCluster (http://socketcluster.io/) - It's designed specifically for running on multiple CPU cores and it has a shared data store for sharing data between workers. It's built on top of Engine.io - I'm the main contibutor. You should check the GitHub page for documentationto see if it is suitable for you: https://github.com/TopCloud/socketcluster