UCS2 coding and decoding using Python

Question

s = "خالد".encode("utf-16be")
uni = s.decode("utf-16be")
print (uni)

UnicodeEncodeError: 'ascii' codec can't encode characters in position 3-7: ordinal not in range(128).

Any suggestion?

Answer 1

Ok, you have an unicode encode error with the ascii charset. This error should not have been raised on any of your two first lines, because none is trying to encode an unicode string as ascii.

So I assume that it is caused by the print in the third line. Depending on your system, and your exact Python version, the print will try to encode with a default encoding which happens to be ascii here.

You must find what encoding is supported by your terminal, or if you can use 'UTF-8'.

Then you can print with

print(uni.encode("utf-8", errors="replace")) # or the encoding supported by your terminal

Answer 2

In Python 3 what you have would work already, because string literals are unicode by default.

In Python 2, you can make a unicode string literal with the u prefix.

s = u"خالد".encode("utf-16be")
uni = s.decode("utf-16be")
print (uni)

Result:

خالد