How to read pcapng file stream from kismet

Question

I am using kismet running on a raspberry pi to capture network data that I need to analyze in real time on another device. Kismet's api has an endpoint to receive a stream of binary data in the pcapng format. I have successfully been able to read the stream but I cannot find any documentation on getting useful data out of the stream besides the actual pcapng standard. This would be fine but the data coming in is not consistent with the format and I have found no libraries that have been able to successfully parse this stream. I think that the issue is that when reading the stream extra data is inserted between each captured packet so determining the start of the packet is the main issue. I am currently using python but am open to other languages if they can easily solve this problem or have libraries already written.

This is the code I am using to get the data into python where print(line.hex()) would be where I accessed the relevant data.

import requests
r = requests.get(url, stream=True)
for line in r.iter_lines():
    print(line.hex())

Answer 1

It looks like the error was that the python requests module was not capturing the entire stream and certain bytes were missing. By using

curlSubProcess = subprocess.Popen(['curl', 'url/pcap/all_packets.pcapng'], stdout=subprocess.PIPE, bufsize=1)
curlSubProcess.stdout.read(lengthnext)

in a while loop where lengthnext is the length of the next packet I was able to read it correctly. I then parsed this binary block data with a class I wrote and the packet data with scapy and it is working as expected.