Best way to save a dictionary which contains non-ASCII characters in Datastore with Python ndb

237 Views Asked by At

I have this dict: {'ª': 'a'} and want to save it to ndb object.

The superscript ª is a non-ASCII character.

Initially I was saving the mapping as a json dump string. I had this class:

class MyClass:
  mapping = ndb.TextProperty()

And saving was doing something like:

obj = MyClass()
obj.mapping = json.dumps({'ª': 'a'})

This was raising an error, because dumps (in Python 2.7) doesn't support non-ASCII chars.

So, I tried to define different model:

class MyClass:
  mapping = ndb.JsonProperty()

and save like this:

obj = MyClass()
obj.mapping = {'ª': 'a'}
obj.put()

This fails during obj.put() with the following error:

utf8 codec can't decode byte 0xaa in position 1: invalid start byte

I'm interested in some general advise around best practices how to save non-ASCII dicts in Datastore using Python 2 and ndb.

I am thinking about trying pickle/cPickle, but am a bit reluctant because here we store user provided data, so using pickle might not be very secure.

0

There are 0 best solutions below