I'm looking for a fast way to find the time that a pcap log file stopped logging. I assume that I need a fast way to find the timestamp in the last message of a pcap file.
I have a test system that creates a pcap log file for every test. While a test is running there is a different system that connects to mine and exchanges information over Ethernet (DoIP). The second systems sometimes executes multiple tests of its own during one of my tests. The second systems creates a pcap file for each of its tests.
I end up with an unknow amount of 1 or more pcap log files that are created by the second system during the time that my system created 1 pcap log file. When I do my test result analysis I need to identify which pcap log files on the second system were created during the time that my system created a specific pcap file. The name of my pcap log file on my system includes the start time. So far I have not been able to find a quick way to get the timestamp of the last message in the pcap file.
During some tests the pcap files on my system can grow to more than 1 GB. We usually move the files from the test system to other storage locations. This often changes the modified date of the file, so I cannot use that to determine when the file stopped recording.
I found useful information on this forum to get the timestamps with scapy and pyshark. The pyshark script works but takes around 3 minutes for one of my smaller 120MB files. It will take much too long for our big logfiles. I also hate that I ended up using a global variable to get the timestamp of the last packet, but I do not understand scapy well enough to find another way that works.
from time import strftime, localtime
from scapy.all import *
import pandas as pd
import time
# ############## SCAPY TEST #################
def check_pcap(file_packet):
global stop_t
new_packet_time = int(file_packet[0].time)
if new_packet_time > stop_t:
stop_t = new_packet_time
for root,dirs,files in os.walk("./Rig_pcap/"):
file_start_stop = []
for pcap_file in files:
start_time = pcap_file[pcap_file.find('-')-4:pcap_file.rfind('-')+3] # From file name
first_packet = True
stop_t = 5
print('####### ',pcap_file,' ########')
sniff(offline="./Rig_pcap/" + pcap_file, prn=check_pcap, store=0)
print(pcap_file,' ',stop_t,' ',strftime('%Y-%m-%d_%H:%M:%S',localtime(stop_t)))
print('-------------')
file_start_stop.append([pcap_file,start_time,strftime('%Y-%m-%d_%H-%M-%S', localtime(stop_t))])
pcap_df = pd.DataFrame(file_start_stop,columns = ['pcap_File_Name','Start_Time','End_Time'])
Below is the latest pyshark code that I used. Here I tried to speed up getting the timestamp by just reading the summaries "pyshark.FileCapture("./Rig_pcap/" + pcap_file,only_summaries=True)" but I'm not getting a timestamp that is close to the end of the file. I commented out my previous pyshark code that read the whole file as I kept getting warning messages that some pyshark background routines were already running.
import pyshark
from time import strftime, localtime
import pandas as pd
import time
########## PYSHARK TEST #####################
for root,dirs,files in os.walk("./Rig_pcap/"):
file_start_stop = []
for pcap_file in files:
print('####### ',pcap_file,' ########')
start_time = time.time() # To measure execution time
pcap = pyshark.FileCapture("./Rig_pcap/" + pcap_file,only_summaries=True) ### for reading PCAP file
#packet_start_time = pcap[0].sniff_time
#packet_end_time = pcap[len(pcap)].sniff_time
#start_time = str(packet_start_time.year)+'-'+str(packet_start_time.month)+'-'+str(packet_start_time.day)+'_'+str(packet_start_time.hour)+':'+str(packet_start_time.minute)+':'+str(packet_start_time.second)
#stop_time = str(packet_end_time.year)+'-'+str(packet_end_time.month)+'-'+str(packet_end_time.day)+'_'+str(packet_end_time.hour)+':'+str(packet_end_time.minute)+':'+str(packet_end_time.second)
#packet_count = len([packet for packet in pcap])
#print('packet_count',packet_count)
for i, packet in enumerate(pcap):
# print(packet.layers)
# print(packet.frame_info)
# print(packet.summary_line)
print(packet.timestamp)
if i == 0:
file_start_time = packet.timestamp
file_stop_time = file_start_time # Setting stop = to start as some summaries only have 1 time.
# print(i,' Start Time:',file_start_time)
else:
file_stop_time = packet.timestamp
#print(i,'Stop time:',sniff_timestamp_stop_time)
print(i,' Start:',file_start_time,' Stop:',file_stop_time)
print("--Execution duration - %s seconds ---" % (time.time() - start_time))
print('------')
file_start_stop.append([pcap_file,file_start_time,file_stop_time])
pcap_df = pd.DataFrame(file_start_stop,columns = ['pcap_File_Name','Start_Time','End_Time'])