I'm trying to parse a proprietary binary-format (Wintec NAL) with python. There's existing and working C-code that does the same (Author: Dennis Heynlein), which i'm trying to port to Python.
I'm struggling to understand parts of the C-code. Here's the definition of the binary format in C:
/*
* File extension:. NAL
* File format: binary, 32 byte fixed block length
*/
/*
* For now we will read raw structs direct from the data file, ignoring byte
* order issues (since the data is in little-endian form compatible with i386)
*
* XXX TODO: write marshalling functions to read records in the proper
* byte-order agnostic way.
*/
#pragma pack (1)
typedef struct nal_data32 {
unsigned char point_type; /* 0 - normal, 1 - start, 2 - marked */
unsigned char padding_1;
unsigned int second: 6, minute: 6, hour: 5;
unsigned int day: 5, month: 4, year: 6; /* add 2000 to year */
signed int latitude; /* divide by 1E7 for degrees */
signed int longitude; /* divide by 1E7 for degrees */
unsigned short height; /* meters */
signed char temperature; /* °C */
unsigned short pressure; /* mbar */
unsigned char cadence; /* RPM */
unsigned char pulse; /* BPM */
signed char slope; /* degrees */
signed short compass; /* °Z axis */
signed short roll; /* °X axis */
signed short yaw; /* °Y axis */
unsigned char speed; /* km/h */
unsigned char bike; /* ID# 0-3 */
unsigned char padding_2;
unsigned char padding_3;
} nal_t;
I'm using python-bitstring to replicate this functionality in Python, but i have difficulties in understanding the time-format given above and adopting it to Python.
from bitstring import ConstBitStream
nal_format=('''
uint:8,
uint:8,
bin:32,
intle:32,
intle:32,
uint:16,
uint:8,
uint:16,
uint:8,
uint:8,
uint:8,
uint:16,
uint:16,
uint:16,
uint:8,
uint:8,
uint:8,
uint:8
''')
f = ConstBitStream('0x01009f5a06379ae1cb13f7a6b62bca010dc703000000c300fefff9ff00000000')
f.pos=0
#type,padding1,second,minute,hour,day,month,year,lat,lon,height,temp,press,cad,pulse,slope,compass,roll,yaw,speed,bike,padding2,padding3=f.peeklist(nal_format)
type,padding1,time,lat,lon,height,temp,press,cad,pulse,slope,compass,roll,yaw,speed,bike,padding2,padding3=f.readlist(nal_format)
print type
print padding1
#print second
#print minute
#print hour
#print day
#print month
#print year
print time
print lat
print lon
While i've figured out that latitude and longitude has to be defined as little-endian, i have no idea how to adapt the 32bit wide timestamp so it fits the format given in the C-definition (And i also couldn't figure out a matching mask for "height" - correspondingly i didn't try the fields after it).
These are the values for the hex-string above:
- date: 2013/12/03-T05:42:31
- position: 73.3390583° E, 33.2128666° N
- compass: 195°, roll -2°, yaw -7°
- alt: 458 meters
- temp: 13 °C
- pres: 967 mb
I'm not familiar with
bitstring
, so I'll convert your input into packed binary data and then usestruct
to handle it. Skip to the break if you're uninterested in that part.I can go over this part in more detail if you want. It's just turning
'0100...'
intob'\x01\x00...'
.Now, the only "gotcha" in unpacking this is figuring out that you only want to unpack ONE unsigned int, since that bit field fits into 32 bits (the width of a single unsigned int):
That converts the output into an output we can use. You can unpack that into your long list of variables, like you were doing before.
Now, your question seemed to be centered around how to mask
time
(above:923163295
) to get the proper values out of the bit field. That's just a little bit of math:In function form, the whole thing is a bit more natural:
Which (now that I think about it) rearranges into:
And if you want to get fancy,
Format all those numbers correctly and finally out comes the expected date/time:
(There is likely a way to do all of this bitwise math elegantly with that
bitstring
module, but I just find it satisfying to solve things from first principles.)