Issue with Outputting data from CSV File

Question

I'm accessing an API and am then outputting the data to a CSV File as follows:

from urllib2 import Request, urlopen, URLError
import csv
request = Request('https://www.quandl.com/api/v1/datasets/YC/CHE10Y.csv?   auth_token=AUTH_TOKEN') # Data available in CSV Format

try:
    response = urlopen(request)
    datafromfile = response.read()
    with open('QuandlData.csv', 'w') as out:
        writer = csv.writer(out)
    for data in datafromfile:
        writer.writerow(data);

except URLError, e:
    print 'No data. Got an error code:', e

This is outputting data. However, the data is coming out like this:

D

a

t

e

,

R

a

t

e

"
"

2

0

1

5

-

0

6

-

0

5

,

0

I understand this is to do with the way I'm outputting the data into the CSV File. However, I don't know how to rectify this.

Answer 1

You are iterating over the characters returned from response.read() so only writing a char at a time. If you want to write lines:

writer.writerows(response.read().splitlines())

Or simply write response.read() with file.write.

You are basically doing:

In [12]: for c in s:
             print(c)
   ....:     
f
o
o
b
a
r

Answer 2

Much easier by,

import urllib
import csv

url = 'https://www.quandl.com/api/v1/datasets/YC/CHE10Y.csv?auth_token=mv1c1_V3kqVeL4yyjPZA'
urllib.urlretrieve(url, '/path-to-file/30718958.csv')

The reason of your output is the result set is formed is single. Hence the iteration was performed as single digits. Please avoid the escape in the Url.

Answer 3

Don't read the file all together, you don't know how large it is. If you want to write CSV rows you need to pass an iterable type like list to writerow. Basically, you need to split the data into it's columns.

Here is my suggestion, you need to change the block of try:

response = urlopen(request)
with open('QuandlData.csv', 'w') as out:
    writer = csv.writer(out)
    for data in response:
        writer.writerow(data.strip().split(','))

Answer 4

Since the response is a csv, another option is to read the response using csv.reader:

from urllib2 import Request, urlopen, URLError
import csv
request = Request('https://www.quandl.com/api/v1/datasets/YC/CHE10Y.csv?auth_token=mv1c1_V3kqVeL4yyjPZA') #% Data available in CSV Format
try:
    response = urlopen(request)
    # read with csv.reader
    datafromfile = csv.reader(response)
    with open('QuandlData.csv', 'w') as out:
        writer = csv.writer(out)
        for data in datafromfile:
            writer.writerows([data])

except URLError, e:
    print 'No data. Got an error code:', e

and I get a file that looks like:

Date,Rate
2015-06-08,0.13
2015-06-05,0.12
2015-06-04,0.13
2015-06-03,0.01
2015-06-02,-0.05
...