CRON + Nodejs + multiple cores => behaviour?

Question

I'm building in a CRON like module into my service (using node-schedule) that will get required into each instance of my multi-core setup and I'm wondering since they are all running their own threads and they are all scheduled to run at the same time, will they get called for every single thread or just once because they're all loading the same module.

If they do get called multiple times, then what is the best way to make sure the desired actions only get called once?

Answer 1

The best way to handle this in a generic way is to have a shared database that you write a "lock" entry to. As in, let's say all tasks wrote a DB entry such as {instanceId: "a", taskId: "myTask", timestamp: "2021-12-22:10:35"}.

All tasks would submit the same thing except with their own instanceId. You then have an unique index on 'timestamp' so that only 1 gets accepted.

Then they all do a query and see if their node was the one that was accepted to do the cron.

You could do the same thing but also add a "random" field that generates a random number and the task with the lowest number wins.

Answer 2

node-schedule runs inside a given node process and it schedules things that that particular node process asked it to schedule.

If you are running multiple node processes and each is using node-schedule, then all the node-schedule instances within those separate node processes are independent (no cooperation or coordination between them). If each node process asks it's own node-schedule instance to run a particular task at 3pm on the first wednesday of the month, then all the node processes will start running that task at that time.

If you only want the action carried out once, then you have to coordinate among your node-instances so that the action is only scheduled in one node process, not in all of them or only schedule these types of operations in one of your node instances, not all of them.

Answer 3

if you are using pm2 with cluster mode, then can use process.env.NODE_APP_INSTANCE to detect which instance is running. You can use the following code so your cron jobs will be called only once.

// run cron jobs only for first instance
if(process.env.NODE_APP_INSTANCE === '0'){
    // cron jobs
}