Python RQ csv file processing

27 Views Asked by At

What is more prefferable way to process csv files using RQ? File possible can have few or few hundred thousands rows

def process_row():
    pass

# Is it better to push function per csv row as a job.

for row in csv:
    queue.enqueue(process_row, row)

# or process the whole csv file as one job ( loops over csv file in job process )

def some_func():
    for row in csv:
        process_row(row)


queue.enqueue(some_func, csv_path)

I was not able to find answer to that question in RQ docs.

0

There are 0 best solutions below