Django/Python: CSV for-in loop overriding first row each time through

Question

class CSVDownload(View):
    """ Prepares CSV file version to download """       

        #more code here

        f = StringIO.StringIO()
        writer = csv.writer(f, dialect='excel')
        for v in visit_list:
            writer.writerow([v.idfa.idfa, v.name, v.duration, v.firstSeen, v.lastSeen, v.identifier, v.closestProximity])
        f.seek(0)
        response = HttpResponse(f, content_type='text/csv')
        response['Content-Disposition'] = 'attachment; filename=AdvertiserData.csv'

        return response

For some reason, when opening the file in excel, the file will only output the last item in the list, see

This leads me to believe that each row is overriding the first row. Although this shouldn't be the case. Look at these tests I have preformed in terminal.

>>> f.getvalue()
'FFAC6F6C-1B2E-47C2-8110-5E619B239FB1,PPTest1,00:00:24,2014-11-11 23:20:24.730000,2014-11-11 23:20:48.750000,nkfe-cnb7s,NEAR\r\nFFAC6F6C-1B2E-47C2-8110-5E619B239FB1,moo2,00:00:24,2014-11-11 23:20:24.730000,2014-11-11 23:20:48.750000,nkfe-cnb7s,NEAR\r\n'

This returns more than one value, with \r\n between the values.

Also, I tried

>>> print response
Content-Type: text/csv
Content-Disposition: attachment; filename=boo.csv

FFAC6F6C-1B2E-47C2-8110-5E619B239FB1,PPTest1,00:00:24,2014-11-11 23:20:24.730000,2014-11-11 23:20:48.750000,nkfe-cnb7s,NEAR
FFAC6F6C-1B2E-47C2-8110-5E619B239FB1,moo2,00:00:24,2014-11-11 23:20:24.730000,2014-11-11 23:20:48.750000,nkfe-cnb7s,NEAR

And that also showed more than one value.

I saw recently a SO thread here that discusses adding to a file here:How do you append to a file? But since I am creating this item in memory only (using stringIO), how can I get the same effect?

Thanks!

EDIT For extra information:

The visit_list is compromised of visit models:

class Visit(models.Model):
    idfa = models.ForeignKey(Report) 
    name = models.CharField(max_length=255)
    lastSeen = models.CharField(max_length=255)
    duration = models.CharField(max_length=255)
    firstSeen = models.CharField(max_length=255)
    identifier = models.CharField(max_length=255)
    closestProximity = models.CharField(max_length=255)

    def __unicode__(self):
        return self.name

    class Meta:
        verbose_name = "Visit"
        verbose_name_plural = "Visits"

In this particular case, visit_list returns:

>>> visit_list
[<Visit: PPTest1>, <Visit: moo2>]

Answer 1

Bottom Line: I suspect you need to pass f.getvalue() to HttpResponse rather than f.

The Explanation:

HttpResponse prefers strings, although it can also be treated like a file object or accept iterators. Nevertheless, f is a StringIO.StringIO instance and not a string per se.

Consider the following setup:

import StringIO
import csv

>>> f = StringIO.StringIO()
>>> writer = csv.writer(f, dialect='excel')
>>> row = range(4)                           # first row is [0, 1, 2, 3]
>>> for i in range(5):
        writer.writerow([row])               # write more rows
        row = [x + 1 for x in row]

>>> f.seek(0)

Now compare f with the result of f.getvalue():

# an instance
>>> f
<StringIO.StringIO instance at 0x03656990>

# a string
>>> f.getvalue()
'"[0, 1, 2, 3]"\r\n"[1, 2, 3, 4]"\r\n"[2, 3, 4, 5]"\r\n"[3, 4, 5, 6]"\r\n"[4, 5, 6, 7]"\r\n'

Now note the difference in your response object when you pass the instance versus the string to HttpResponse:

# The instance
>>> response = HttpResponse(f, content_type='text/csv')  # f is an instance
>>> response['Content-Disposition'] = 'attachment; filename=AdvertiserData.csv'
>>> response._container
['']

# The string
>>> response = HttpResponse(f.getvalue(), content_type='text/csv')  # f.getvalue() returns a string
>>> response['Content-Disposition'] = 'attachment; filename=AdvertiserData.csv'
>>> response._container
['"[0, 1, 2, 3]"\r\n"[1, 2, 3, 4]"\r\n"[2, 3, 4, 5]"\r\n"[3, 4, 5, 6]"\r\n"[4, 5 , 6, 7]"\r\n']

Note that the response's _container attribute is empty when you pass the instance to HttpResponse but not empty when you pass the f.getvalue() string to HttpResponse.

I would try this instead, then:

>>> response = HttpResponse(f.getvalue(), content_type='text/csv')
>>> # etc.