cStringIO.StringO fails to save uploaded file stream when file is smaller than 1kByte

Question

I borrowed this code for saving a file stream to disk, and it works except for when the file is less than 1kb in size. I get this error:

in stuff_uploaded:
copy(theFile.file.name, './tmp/'+theFile.filename) #saves temporary file to /cp2/tmp/

AttributeError: 'cStringIO.StringO' object has no attribute 'name'

@cherrypy.expose
@cherrypy.tools.noBodyProcess()
def stuff_uploaded(self, theFile=None):
    import cgi
    import tempfile
    # convert the header keys to lower case
    lcHDRS = {key.lower():val for key, val in cherrypy.request.headers.iteritems()}
    class myFieldStorage(cgi.FieldStorage):
        """uses a named temporary file instead of the default non-named file; keeping it visibile (named), allows us to create a
        2nd link after the upload is done, thus avoiding the overhead of making a copy to the destination filename."""
        def make_file(self, binary=None):
            return tempfile.NamedTemporaryFile()
    formFields = myFieldStorage(fp=cherrypy.request.rfile,
                                headers=lcHDRS,
                                environ={'REQUEST_METHOD':'POST'},
                                keep_blank_values=True)
    theFile = formFields['theFile']
    # we now create a 2nd link to the file, using the submitted filename.
    from shutil import copy
    copy(theFile.file.name, './tmp/'+theFile.filename) #saves temporary file 
    msgs = csv_to_survey.match_fieldnames('./tmp/'+theFile.filename)
    return './tmp/'+theFile.filename

So what can I do to ensure that cStringIO.StringO handles small uploaded files?

Answer 1

Just open and write the file directly:

with open('./tmp/'+theFile.filename, "w") as f:
    f.write(theFile.file.getvalue())

Or to deal with it regardless of whether the file is on disk or StringIO, use it as a file-like object:

import shutil

with open('./tmp/'+theFile.filename, "w") as f:
    # If the file pointer might not be at the beginning of theFile.file, add:
    # theFile.file.seek(0)
    shutil.copyfileobj(theFile.file, f)
    # While:
    #     f.write(theFile.file.read())
    # would work most of the time, it involves holding the whole contents of the
    # file in memory at once (which you want to avoid; that's why CherryPy
    # uses temp files for larger data). shutil.copyfileobj does block by
    # block copies, which have fixed peak memory usage while still running
    # (almost) as fast

Note: This (and your original solution) is insecure, since uploading the same file twice will overwrite the previous file, and (depending on server filters on names) file names could traverse the file system to overwrite unintended files outside the tmp directory.