How to measure latency of low latency c++ application

Question

I need to measure message decoding latency (3 to 5 us ) of a low latency application.

I used following method,

1. Get time T1
2. Decode Data
3. Get time T2
4. L1 = T2 -T1
5. Store L1 in a array (size = 100000)
6. Repeat same steps for 100000 times. 
7. Print array.
8. Get the 99% and 95% presentile for the data set.

But i got fluctuation between each test. Can some one explain the reason for this ? Could you suggest any alternative method for this.

Note: Application is tight loop (acquire 100% cpu) and Bind to CPU via taskset commad

Answer 1

There are a number of different ways that performance metrics can be gathered either using code profilers or by using existing system calls.

NC State University has a good resource on the different types of timers and profilers that are available as well as the appropriate case for using each and some examples on their HPC website here.

Fluctuations will inevitably occur on most modern systems, certain BIOS setting related to hyper threading and frequency scaling can have a significant impact on the performance of certain applications, as can power-consumption and cooling/environmental settings.

Looking at the distribution of results as a histogram and/or fitting them to a Gaussian will also help determine how normal the distribution is and if the fluctuations are normal statistical noise or serious outliers. Running additional tests would also be beneficial.