Python Multi-threading - How to kill all child threads when one of them raise an exception or failed

Question

Depending on the needs of the program, I used multi-threads within a sub-thread for parallel tasking. In this "deep" multi-threading, potential problems such as timeouts and failures may be encountered.

I would like to ask, how to kill all child threads (known as "deep" multi-threading above) when one of them raise an exception or failed.

Answer 1

You shouldn't really be trying to kill threads.

If a thread needs to terminate because of some unrelated condition then it needs to [somehow] be notified. You just need a "flag" that's accessible to all threads.

Here's a pattern that you could adapt. It requires that each thread checks the "flag" from time to time.

The "flag" in this case is a list that contains a single arbitrary value. The run condition is true if the list is not empty.

If a problem arises in one thread it should just clear the run list. Thus all other threads will terminate.

from threading import Thread
from time import sleep
from random import randint

def process(run):
    while run:
        sleep(1) # do work
        if randint(1, 50) == 25: # simulate failure
            print("Failing")
            run.clear()
    print("Ending")

if __name__ == "__main__":
    run = [None]
    threads = [Thread(target=process, args=(run,)) for _ in range(10)]
    for thread in threads:
        thread.start()
    for thread in threads:
        thread.join()

You could make this slightly simpler by using a global variable if you're that way inclined