is PYTHON Gearman Worker accept multi-tasks

622 Views Asked by At

For example: I have a task named "URLDownload", the task's function is download a large file from internet. Now I have a Worker Process running, but have about 1000 files to download. It is easy for a Client Process to create 1000 task, and send them to Gearman Server.

My Question is the Worker Process will do the task one by one, or it will accept multi-tasks at one time,

If the Worker Process can accept multi-tasks, So How can I limit the task-pool-size in Worker Process.

1

There are 1 best solutions below

0
On BEST ANSWER

Workers process one request at a time. You have a few options:

1) You can run multiple workers (this is the most common method). Workers sit in poll() when they aren't processing so this model works pretty well.

2) Write a fork() implementation around the worker. This way you can fire up a set number of worker processes, but don't have to monitor multiple processes.