How to architect a messaging bus with Node.js and an API distributed amongst several listeners?

Question

Here's what I want to happen:

My user will hit an endpoint, say /api/findFile/app.js. express is listening for this call (standard REST type stuff) and will contact several workers (can be any number) asking them to perform work, specifically find the file. When the first one does, it should respond back to the function in express so that the results can be sent back to the user.

I imagine some sort of messaging bus / AMQP setup will be needed. I figure that the express function can publish a request to the workers, each of which are subscribed do that event:

bus.publish('findFile', {fileName: 'app.js'}};

Something along those lines. The workers have:

bus.subscribe('findFile', function(event) {....

Each of the workers will check to see if it has the file on it's hard disk. The file can be on MULTIPLE workers disks. So when the FIRST one finds it, I want to somehow abort the others from returning to express.

To return to express, I'd imagine we'd do a pub/sub in reverse, this time with express listening for the response?

Any ideas on how to properly architect this? Especially with the race condition.

Answer 1

You can use socket.io. Its an event-driven client-server pub-sub framework that works well with express.