Heavy API endpoint Heroku time out

51 Views Asked by At

I am building an application with a node/express backend. The endpoint has a really long response time due to heavy computation (take 1 - 2 mins). Heroku times me out at 30 seconds.

I'm using Kue and redis in order to fulfill the computation in redis.

A little bit of pseudo 1. When a user hits the endpoint 2. Initiate the job 3. Send response after invoking job to not time out of heroku 4. Once Job is complete, respond/send completed job/data => which will go into redux.

Here is my code so far:

const redis = require('redis');
const client = redis
  .createClient();

const kue = require('kue');
const queue = kue.createQueue();

router.post('/endpoint', (req, res) => {
  const param = req.body.param;

  var job = queue
    .create('job', param)
    .priority('high')
    .save(err => {
      if (err) {
        console.log('failed');
        process.exit(0);
        return;
      }
      job.on('complete', result => {
        res.send(result);
        console.log('completed');
      });
      job.on('failed', errorMessage => {
        console.log(errorMessage);
        process.exit(0);
      });
    });

  queue.process('job', async (job, done) => {
    // heavy heavy computation
    // heavy heavy heavy heavy
    // heavy heavy heavy
    //output is results => passed into promise
    done(null, await Promise.all(results));
  });

  res.json({ message: 'job in progress' });
});

The problem with this code is that I can't send a header response once it has been set. I am very new to redis and very new kue and background jobs. I've read webhooks can help, but I don't have much experience with webhooks either.

Thank you in advance!

0

There are 0 best solutions below