mongodb: building a job queue

1.1k Views Asked by At

I am building a job queue (Nodejs+Mongodb)which has this kind of structure :

{
    "_id" : ObjectId("53e09abd23bbaeea776598e1"),
    "createDate" : ISODate("2014-08-05T08:50:05.643Z"),
    "status" : "new",
    ......
}

I have defined 4 statuses in my system as : new, waiting, working, failed, done

Requirement: I need to pick the most recent job in the job queue with status as "new".

Currently, I am doing this by using

jobQueue.findAndModify(
        {
          status: 'new',
        },
        {date: 1},
        {$set: { ... } },
        {remove: false},
        function (err, job) {......

which works fine when the job documents are less. But there have been situations when document nos. rises to 50k. Then the locking period (using mongostat) goes beyond 100%(at times reaches 200% ). Hence my server is not able to deliver the new jobs in such scenario.

Can anybody suggest me a more efficient way to this query??

Thanks in advance :)

0

There are 0 best solutions below