How to maintain a pool of processes

78 Views Asked by At

I have a project in which I run multiple processes to do multiple jobs, the jobs are distributed via a queue.

queue = multiprocessing.Queue()
process = multiprocessing.spawn(
    run_alg,
    args=(queue,),
    nprocs=process_num,
    join=False,
)
process.join()

However, it is possible for the processes to stop working or to crash mid-operation. I want to create a pool of x processes where every time a process is shut down, a new process will be created to take its place until the queue is empty.

Is it possible, how?

0

There are 0 best solutions below