Asynchronous file distribution with python

Question

I'm new to coding and to python. I tried to create a code that will distribute files over the network, problem is, it iterates through a list I made, and distributes one after another. Only when the first location gets the files, just then it will start with the second distribution.

This is the code I wrote:

from shutil import copytree

source = r"O:\versions"

Eilat = r"\\eilat-zrv\c$\version"
BeerSheba = r"\\beer-sheba-zrv\c$\version"
Ashdod = r"\\ashdod-zrv\c$\version"
Rehovot = r"\\rehovot-zrv\c$\version"
Rishon = r"\\rishon-zrv\c$\version"
TelAviv = r"\\dizingof-zrv\c$\version"
Netanya = r"\\netanya-zrv\c$\version"
RamatIshay = r"\\ramaty-zrv\c$\version"
Haifa = r"\\haifa-zrv\c$\version"



DestList = [Eilat, BeerSheba, Ashdod, Rehovot, Rishon, TelAviv, Netanya, RamatIshay, Haifa]

def Copy_funct(i):
    for i in DestList:
         copytree(source, i)

Copy_funct(DestList)

How can I make it distribute simultaneously??

Answer 1

Using threads it would look sth like this:

import threading

def Copy_funct(DestList):
  threads = [ threading.Thread(target=lambda dest=dest: copytree(source, dest))
              for dest in DestList ]
  for thread in threads:
    thread.start()
  for thread in threads:
    thread.join()

EDIT:

A bit of explanation:

Threading is a way of splitting your program into several threads of action which all are pursued in parallel by the computer; if one thread has to wait for something (typically I/O), another thread can continue its own task. Doing stuff in a parallel fashion opens up a large Box of Pandora of possible bugs you probably never have heard of if you haven't done concurrency before, and I can't go into detail here. But I think your usecase is simple enough to not hit such problems.

The first statement in my function builds a list of threads (a "thread" is represented by an object in Python). These threads are told upon creation what their task will be (using the parameter target). The argument given is a lambda closure; this way I can easily pass information (dest) into the closure. Basically, what I pass to parameter target is a function that, when called, will do that copytree() call with the correct parameters for one of your tasks.

The second statement is a loop over all threads and just starts each. Before that they are just unstarted threads, like post-it notes with ToDo reminders but without an action taking place. With start() I start each thread; that starting takes virtually no time, the call to start them will return immediately, so I can start all threads practically at the same moment. I have no direct control over when they actually start doing their thing; the first may have already begun its work when the last is being started.

The third statement also is a loop over all threads and uses join() to wait for their finishing. It doesn't matter which threads finishes first or last, I will first wait for the first thread, then for the second, etc. If the second actually finishes earlier than the first, this is no problem, it will just be a finished thread which hasn't been joined yet. It will be joined (and then properly cleaned up) by the join in the second iteration of the second loop.

I hope this explains it enough to be useful for you. Feel free to suggest further improvements on this answer if still vital things are unclear.