Google App Engine : Bulkuploader : int64 too big error

Question

i am getting this error while uploading data to datastore using bulkuploader. Data used to be uploaded fine with the previous csv file. the new csv file has an extrafield that contains a list of strings. (ex. A,B,E,G,E,F). Following is the error that i get.

Traceback (most recent call last):
  File "/opt/google_appengine_1.6.4/google/appengine/tools/adaptive_thread_pool.py", line 176, in WorkOnItems
    status, instruction = item.PerformWork(self.__thread_pool)
  File "/opt/google_appengine_1.6.4/google/appengine/tools/bulkloader.py", line 764, in PerformWork
    transfer_time = self._TransferItem(thread_pool)
  File "/opt/google_appengine_1.6.4/google/appengine/tools/bulkloader.py", line 935, in _TransferItem
    self.request_manager.PostEntities(self.content)
  File "/opt/google_appengine_1.6.4/google/appengine/tools/bulkloader.py", line 1420, in PostEntities
    datastore.Put(entities)
  File "/opt/google_appengine_1.6.4/google/appengine/api/datastore.py", line 576, in Put
    return PutAsync(entities, **kwargs).get_result()
  File "/opt/google_appengine_1.6.4/google/appengine/datastore/datastore_rpc.py", line 786, in get_result
    results = self.__rpcs[0].get_result()
  File "/opt/google_appengine_1.6.4/google/appengine/api/apiproxy_stub_map.py", line 592, in get_result
    return self.__get_result_hook(self)
  File "/opt/google_appengine_1.6.4/google/appengine/datastore/datastore_rpc.py", line 1556, in __put_hook
    self.check_rpc_success(rpc)
  File "/opt/google_appengine_1.6.4/google/appengine/datastore/datastore_rpc.py", line 1191, in check_rpc_success
    rpc.check_success()
  File "/opt/google_appengine_1.6.4/google/appengine/api/apiproxy_stub_map.py", line 558, in check_success
    self.__rpc.CheckSuccess()
  File "/opt/google_appengine_1.6.4/google/appengine/api/apiproxy_rpc.py", line 156, in _WaitImpl
    self.request, self.response)
  File "/opt/google_appengine_1.6.4/google/appengine/ext/remote_api/remote_api_stub.py", line 248, in MakeSyncCall
    handler(request, response)
  File "/opt/google_appengine_1.6.4/google/appengine/ext/remote_api/remote_api_stub.py", line 397, in _Dynamic_Put
    'datastore_v3', 'Put', put_request, put_response)
  File "/opt/google_appengine_1.6.4/google/appengine/ext/remote_api/remote_api_stub.py", line 177, in MakeSyncCall
    self._MakeRealSyncCall(service, call, request, response)
  File "/opt/google_appengine_1.6.4/google/appengine/ext/remote_api/remote_api_stub.py", line 185, in _MakeRealSyncCall
    request_pb.set_request(request.Encode())
  File "/opt/google_appengine_1.6.4/google/net/proto/ProtocolBuffer.py", line 56, in Encode
    self.Output(e)
  File "/opt/google_appengine_1.6.4/google/net/proto/ProtocolBuffer.py", line 205, in Output
    self.OutputUnchecked(e)
  File "/opt/google_appengine_1.6.4/google/appengine/datastore/datastore_pb.py", line 4400, in OutputUnchecked
    self.entity_[i].OutputUnchecked(out)
  File "/opt/google_appengine_1.6.4/google/appengine/datastore/entity_pb.py", line 2380, in OutputUnchecked
    self.property_[i].OutputUnchecked(out)
  File "/opt/google_appengine_1.6.4/google/appengine/datastore/entity_pb.py", line 1307, in OutputUnchecked
    self.value_.OutputUnchecked(out)
  File "/opt/google_appengine_1.6.4/google/appengine/datastore/entity_pb.py", line 945, in OutputUnchecked
    self.referencevalue_.OutputUnchecked(out)
  File "/opt/google_appengine_1.6.4/google/appengine/datastore/entity_pb.py", line 675, in OutputUnchecked
    self.pathelement_[i].OutputUnchecked(out)
  File "/opt/google_appengine_1.6.4/google/appengine/datastore/entity_pb.py", line 135, in OutputUnchecked
    out.putVarInt64(self.id_)
  File "/opt/google_appengine_1.6.4/google/net/proto/ProtocolBuffer.py", line 402, in putVarInt64
    raise ProtocolBufferEncodeError, "int64 too big"

Answer 1

Changing the data type of problematic entries from IntegerProperty to StringProperty might help.

I was having the same problem, as I was storing user_id for Users entity as Integer, but when confronted with a bigger number, it simply can't hold it. So I am storing it as String now.