Upload multiple files with pyCurl

60 Views Asked by At

I am using pyCurl to upload a single file to a WebDAV server. The content is stored in a base64 string.

I am using this codesnipped:

def fileupload(url, filename, filedata, upload_user, upload_pw):
    filedata = base64.b64decode(filedata).decode('ascii', 'ignore')

    # Initialize pycurl
    c = pycurl.Curl()
    url = url + filename
    c.setopt(pycurl.URL, url)
    c.setopt(pycurl.UPLOAD, 1)
    c.setopt(pycurl.READFUNCTION, StringIO(filedata).read)
    c.setopt(pycurl.CONNECTTIMEOUT, 5)
    c.setopt(pycurl.TIMEOUT, 20)
    c.setopt(pycurl.SSL_VERIFYPEER, 1)
    c.setopt(pycurl.SSL_VERIFYHOST, 2)
    c.setopt(pycurl.CAINFO, "ca.pem")
    c.setopt(pycurl.USERPWD, upload_user + ":" + upload_pw)

    # Set size of file to be uploaded.
    filesize = len(filedata)
    c.setopt(pycurl.INFILESIZE, filesize)

    # Start transfer
    logPes.debug('Uploading file %s to url %s' % (filename, url))
    try:
        c.perform()

        # Check result of transfer
        if c.getinfo(pycurl.HTTP_CODE) == 201:
            c.close()
            logPes.info('Fileupload successful')
            return True
        else:
            logPes.error('Fileupload NOT successful. Error code {0}.'.format(c.getinfo(pycurl.HTTP_CODE)))
            c.close()
            return False
    
    except pycurl.error as error:
        logPes.error('Fileupload: {0}'.format(error.args[1]))
        return False

For uploading multiple files can I call this function multiple times. But then a new connection for each upload is opened and closed. Is it possible to upload multiple files with pyCurl with the same connection?

Perhaps the pycurl.CurlMulti() function is the right one for this. Unfortunatly I can not find information if the "multi" function is carried out in the same connection. Is it running serial or asyncron? I am interessted in an upload one file after the another with the same open connection to the http server to reduce the load (the server is a little bit wonky).

Thanks in advance!

1

There are 1 best solutions below

0
nico On

Based on this comment Sending multiple files to webdav server in one request I came to the conclusion, that multiple upload in one request is not possible with WebDAV. I went now for a solution by changing to httpv2 for speed up the process. At the moment it is working fine for us. If I found an alternative to reduce the amount of connections to the WebDAV server I will add this here.