Error in import FloatField, using django-import-export

Question

I am using django-import-export for import csv file. I have a FloatField in my model :

models.py

purchase_price = models.FloatField(null=True, blank=True)

When I import csv file with blank value, it throws an error :

ValueError at /admin/csv_imp/book/process_import/

could not convert string to float:

Request Method:     POST
Request URL:    http://localhost:8000/admin/csv_imp/book/process_import/
Django Version:     1.7.1
Exception Type:     ValueError
Exception Value:    

could not convert string to float: 

Exception Location:     /home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py in get_prep_value, line 1550
Python Executable:  /home/bgdev/Desktop/virtualenvs/django17/bin/python
Python Version:     2.7.6
Python Path:    

['/home/bgdev/Desktop/virtualenvs/django17/test_pro',
 '/home/bgdev/Desktop/virtualenvs/django17/src/admin-bootstrap',
 '/home/bgdev/Desktop/virtualenvs/django17/lib/python2.7',
 '/home/bgdev/Desktop/virtualenvs/django17/lib/python2.7/plat-i386-linux-gnu',
 '/home/bgdev/Desktop/virtualenvs/django17/lib/python2.7/lib-tk',
 '/home/bgdev/Desktop/virtualenvs/django17/lib/python2.7/lib-old',
 '/home/bgdev/Desktop/virtualenvs/django17/lib/python2.7/lib-dynload',
 '/usr/lib/python2.7',
 '/usr/lib/python2.7/plat-i386-linux-gnu',
 '/usr/lib/python2.7/lib-tk',
 '/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages',
 '/home/bgdev/Desktop/virtualenvs/django17/lib/python2.7/site-packages']

Server time:    Tue, 9 Dec 2014 05:33:36 +0000



Environment:


    Request Method: POST
    Request URL: http://localhost:8000/admin/csv_imp/book/process_import/

    Django Version: 1.7.1
    Python Version: 2.7.6
    Installed Applications:
    ('bootstrap_admin',
     'import_export',
     'django.contrib.admin',
     'django.contrib.auth',
     'django.contrib.contenttypes',
     'django.contrib.sessions',
     'django.contrib.messages',
     'django.contrib.staticfiles',
     'csv_imp')
    Installed Middleware:
    ('django.contrib.sessions.middleware.SessionMiddleware',
     'django.middleware.common.CommonMiddleware',
     'django.middleware.csrf.CsrfViewMiddleware',
     'django.contrib.auth.middleware.AuthenticationMiddleware',
     'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
     'django.contrib.messages.middleware.MessageMiddleware',
     'django.middleware.clickjacking.XFrameOptionsMiddleware')


    Traceback:
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
      111.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/django/utils/decorators.py" in _wrapped_view
      105.                     response = view_func(request, *args, **kwargs)
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/django/views/decorators/cache.py" in _wrapped_view_func
      52.         response = view_func(request, *args, **kwargs)
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/django/contrib/admin/sites.py" in inner
      204.             return view(request, *args, **kwargs)
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/import_export/admin.py" in process_import
      130.                                  raise_errors=True)
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/import_export/resources.py" in import_data
      359.                     six.reraise(*sys.exc_info())
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/import_export/resources.py" in import_data
      345.                         self.save_instance(instance, real_dry_run)
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/import_export/resources.py" in save_instance
      163.             instance.save()
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/django/db/models/base.py" in save
      591.                        force_update=force_update, update_fields=update_fields)
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/django/db/models/base.py" in save_base
      619.             updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/django/db/models/base.py" in _save_table
      681.                                       forced_update)
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/django/db/models/base.py" in _do_update
      725.         return filtered._update(values) > 0
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/django/db/models/query.py" in _update
      600.         return query.get_compiler(self.db).execute_sql(CURSOR)
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py" in execute_sql
      1004.         cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py" in execute_sql
      775.             sql, params = self.as_sql()
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py" in as_sql
      969.                 val = field.get_db_prep_save(val, connection=self.connection)
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py" in get_db_prep_save
      627.                                       prepared=False)
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py" in get_db_prep_value
      619.             value = self.get_prep_value(value)
    File "/home/bgdev/Desktop/virtualenvs/django17/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py" in get_prep_value
      1550.         return float(value)

    Exception Type: ValueError at /admin/csv_imp/book/process_import/
    Exception Value: could not convert string to float: 

Answer 1

I found similar issue with and IntegerField. Being the last column in the csv file, it turns out that I was introducing an extra End of Line character in the dataset, even if the field had an empty value. Removing the EOL character from every line before adding it to the dataset fixed the issue:

for line in file.readlines():
    line = line.rstrip('\r\n')
    dataset.append(tuple(item for item in line.split(',')))

Answer 2

This link will show you what null and blank do: differentiate null=True, blank=True in django

If you can arrange your csv file to have all of the null values at the end, you can just remove the commas. That might work. Or perhaps create a temporary model that assigns those fields to a Charfield. Then copy appropriate fields to your final model

This is untested:

update <table> set <field> = 0 where id in (select id from <table> where <field> = "");

Fill in your table name and the field name as appropriate. You can run this from your sql command line program