UnicodeDecodeError after dealing with UnicodeEncodeError

205 Views Asked by At

Taking any of the fixes described here gets me past the problem with attempting to run the following:

row = ['=HYPERLINK("%s")' % cell if 'http' in str(cell) else cell for cell in row].

against a row like:

(17, u'http://british-business-bank.co.uk/what-the-british-business-bank-does/job-vacancies/', u'EMERY MCLAVEN ORR LIMITED', u'10 GREAT PULTENEY STREET, LONDON', u'Senior Manager \u2013 Financial Planning - EMERY MCLAVEN ORR LIMITED', u'http://british-business-bank.co.uk/what-the-british-business-bank-does/job-vacancies/senior-manager-financial-planning-2/', 4, u'2016-02-20')

Now though, when I attempt to make the CSV (of all rows processed as as described in the solution) donwloadable in Flask via Flask-Excel's make_response_from_array method:

# The database query page is only accessible to authenticated users
@app.route('/query', methods=['GET', 'POST'])
@login_required                                 # Use of @login_required decorator
def query():
    if request.method == "POST":
        print request.form.get('title')
        print request.form['title']
        conn = sqlite3.connect('jobsaggregator.db')
        c = conn.cursor()
        c.execute("select * from scrapedjobs")
        results = c.fetchall()
        hyperlinked_results = []
        for row in results:
            print row
            row = [u'=HYPERLINK("{}")'.format(cell) 
   if isinstance(cell, unicode) and cell.startswith(u'http') else cell 
   for cell in row]

            row = [cell.encode('utf-8') if isinstance(cell, unicode) else str(cell) 
   for cell in row]
            hyperlinked_results.append(row)
        save_csv_of_query('output.csv', hyperlinked_results) # no longer needed
        # Ref:http://flask-excel.readthedocs.org/en/latest/#flask_excel.make_response_from_array
        return excel.make_response_from_array(hyperlinked_results, "csv", file_name='Query Results %s' % datetime.datetime.today())

I get the following:

Traceback (most recent call last):
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__
    return self.wsgi_app(environ, start_response)
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/flask_user/decorators.py", line 27, in decorated_view
    return func(*args, **kwargs)
  File "/Users/Pyderman/repos/example/basic_app.py", line 146, in query
    return excel.make_response_from_array(hyperlinked_results, "csv", file_name='Query Results %s' % datetime.datetime.today())
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/pyexcel_webio/__init__.py", line 260, in make_response_from_array
    io = pe.save_as(array=array, dest_file_type=file_type, **keywords)
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/pyexcel/sources/__init__.py", line 318, in save_as
    sheet.save_to(dest_source)
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/pyexcel/sheets/sheet.py", line 20, in save_to
    source.write_data(self)
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/pyexcel/sources/file.py", line 76, in write_data
    **self.keywords)
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/pyexcel_io/__init__.py", line 296, in save_data
    **keywords)
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/pyexcel_io/__init__.py", line 260, in store_data
    writer.write(data)
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/pyexcel_io/base.py", line 290, in write
    sheet.write_array(sheet_dicts[name])
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/pyexcel_io/base.py", line 253, in write_array
    self.write_row(r)
  File "/Users/Pyderman/repos/example/lib/python2.7/site-packages/pyexcel_io/csvbook.py", line 237, in write_row
    for s in array])
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 15: ordinal not in range(128) 

So we're past the original UnicodeEncodeError, and now encounter a UnicodeDecodeError, and things are getting a little circular. I don't know the pyexcel code at all, but I'm getting the feeling that I shouldn't need to, and that this is avoidable; what I am failing to grasp?

0

There are 0 best solutions below