Sleep after spawn in Python

Question

I would like to launch 4 threads to do some work every 1 second in Python. To make sure the work is done every 1 sec I inserted a sleep after the spawn as shown in the following code snippet. From printing it seems that the number of times do_work was executed was 10 while I was expecting 10*4 --> the number of loop iterations x the number of threads. Also, it seems that the first thread starts to do the work after 4 seconds from the time printed in the main loop.(I printed the time also inside the thread).

    run_pool = GreenPool(4)
    counter = 0
    while counter < 10:
        run_pool.spawn(self.do_work)
        time.sleep(1)
        counter += 1
        print time.time()

Answer 1

The parameter to the constructor for GreenPool is the number of worker threads that will be used. By passing the number 4, you are telling it to call self.do_work by a maximum of four times simultaneously. Since you call spawn ten times, you've queued up 10 "jobs", so that's how many times do_work will get called. The amount of parallelism (4 in this case) doesn't affect the number of times your tasks get execute, but instead limits the amount of threads that may run simultaneously.

Answer 2

First of all, you should read the documentation to get a better idea of how eventlet works and how to use it. I myself have never used eventlet, everything that follows I figured out by skimming the docs for a few minutes.

it seems that the number of times do_work was executed was 10 while I was expecting 10*4

GreenPool.spawn() only spawns one thread per call. The size of the pool limits how many threads can be running at once. If all threads in the pool are in use then spawn will block until a thread becomes available.

it seems that the first thread starts to do the work after 4 second from the time printed in the main loop

You need to monkey-patch the time module to make it voluntarily yield control while sleeping. Since you haven't done this, the spawned threads can't do work while the main thread is sleeping. As a result, the threads don't run until the 4th iteration of the loop. At that point, the main thread can't spawn any more threads so it "blocks" and yields control to the worker threads.

Calling eventlet.monkey_patch() before your loop will fix the problem. Alternatively, use eventlet.sleep() instead of time.sleep().