set a queue to run jobs one at a time

Question

in my application i have a route that get a big array of objects and chunk it to smaller parts (for example 250 rows in each part) and push every part to the queue. but this data must process row by row and each job must run only after the previous one is done. because every row depend on some of rows before itself

my queue driver is redis and horizon and it execute jobs very fast. I tried to use later method and set a dynamic delay. but it doesnt solve my problem. because the client might cant handle big data and splits its data (paginate it) and send them to my api. and so the data in each request must add to the end of the queue.

for now i add each part three time to the queue like below. but im looking for a better way

foreach ($dataParts as $i => $dataPart) {
    Queue::later((($i * 5)), new ImportData($dataPart));
    Queue::later((($i * 60) ), new ImportData($dataPart));
    Queue::later((($i * 120)), new ImportData($dataPart));
}

Answer 1

I tried job batching but still didn't solve my problem. then tried withoutOverlaping method and that didn't work too.

finally i added a new worker and queue in horizon.php file and set maxProcess => 1 for it. and push my jobs to that. and worked as i wanted

horizon.php:

...
...
...

'defaults' => [
        // the default queue
        'supervisor-1' => [
            'connection' => 'redis',
            'queue' => ['default'],
            'balance' => 'auto',
            'maxProcesses' => 10,
            'memory' => 128,
            'tries' => 1,
            'nice' => 0,
        ],

         // new queue process only one job at a time
        'supervisor-single' => [
            'connection' => 'redis',
            'queue' => ['single-queue'],
            'balance' => 'auto',
            'maxProcesses' => 1,
            'memory' => 128,
            'tries' => 1,
            'nice' => 0,
        ],
    ],

in controller:

foreach ($dataParts as $dataPart) {
    Queue::push(new ImportData($dataPart), null, 'single-queue');
}