How can i convert my hex values to binary in Python? binascii.a2b_hex error

Question

Hello when I use binascii.a2b_hex, error is occured. There is no error message but the return value is wrong. Here is the data and the resule.

Input : FEB10AA86764FFFF Output : b'\xfe\xb1\n\xa8gd\xff\xff'

import binascii

ta['data'] = ta['data'].apply(binascii.a2b_hex)

n\xa8gd <- what it happend? how can i fix it?

I am trying to use various hex values with my code. Most of values such as fe, b1 would work well.

Answer 1

you can use lambda to convert each hex int

ta['data'] = ta['data'].apply(lambda x: binascii.a2b_hex(x))

Answer 2

It's actually working fine. What's confusing you is the output as a bytestring. If a byte is a valid printable character it will output the character, otherwise it will put out an escape sequence.

FE  \xfe
B1  \xb1
0A  \n
A8  \xa8
67  g
64  d
FF  \xff
FF  \xff

Answer 3

What you are seeing is Python being "helpful" and taking any bytes in the ASCII range and displaying the ASCII character rather than the byte value when displaying to the screen.

There are various ways you can change how the value is displayed for human consumption

$ python3
Python 3.9.2 (default, Mar 12 2021, 04:06:34) 
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> data = bytes.fromhex('FEB10AA86764FFFF')
>>> data
b'\xfe\xb1\n\xa8gd\xff\xff'
>>> print(data)
b'\xfe\xb1\n\xa8gd\xff\xff'
>>> print(list(data))
[254, 177, 10, 168, 103, 100, 255, 255]
>>> for idx, x in enumerate(data):
...     ascii_char = data[idx:idx+1].decode('ascii', 'ignore')
...     print(f"data[{idx}] is denary={x:3d} or hex={x:#04x} or ascii={ascii_char}")
... 
data[0] is denary=254 or hex=0xfe or ascii=
data[1] is denary=177 or hex=0xb1 or ascii=
data[2] is denary= 10 or hex=0x0a or ascii=

data[3] is denary=168 or hex=0xa8 or ascii=
data[4] is denary=103 or hex=0x67 or ascii=g
data[5] is denary=100 or hex=0x64 or ascii=d
data[6] is denary=255 or hex=0xff or ascii=
data[7] is denary=255 or hex=0xff or ascii=
>>> data.hex()
'feb10aa86764ffff'