Download Blobs from Azure with Multiprocessing in Python

Question

Is there anyone that knows how to use multiprocessing in python to download Blobs from Azure Storage?

The code below gives me this error: " TypeError: 'Blob' object is not iterable "

How to fix that?

Code:

def downloadBlobs(generator):
    for Blob in generator:
        path = 'temp/' + Blob.name.split('/')[-1]
        block_service.get_blob_to_path(CONTAINER_NAME,Blob.name,path)


if __name__  == '__main__':

    start = timeit.default_timer()

    generator = block_service.list_blobs(CONTAINER_NAME, prefix='trt2', num_results=1000)
    p = Pool()
    p.map(downloadBlobs, generator)

    final = timeit.default_timer() - start

    print(final)

Answer 1

Is it a case problem ?? Can you try with lower case :

for blob in generator:
    path = 'temp/' + blob.name.split('/')[-1]
    block_service.get_blob_to_path(CONTAINER_NAME,blob.name,path)

I guess Blob is a type so you can`t use it.