How to plot miss rate vs false positive rate?

Question

i have a problem to plot a curve for miss rate vs false positive rate to analyze the performance of my proposed system (as sampled on picture below). I have two samples dataset for positive and negative sample. I want to plot the performance of my system whether it can classify people or non people with this curve.

As far as I know, I need to get True Positive and False Positive values after the classification, but I am not sure yet how to plot the curve. Any one can help please??

Answer 1

There are two type of bounding boxes in the object detection, the boxes that data-set labeled theme as object and second the boxes that your algorithm detects.

If your bbox have huge intersection with data-set bbox, it is okey.

If your bbox have NOT intersection with data-set bbox it is False Posative.

And we call All data-set bbox without intersection with your bbox in an image MISS Rate. and after calculating these numbers, plotting these values is straight forward.

Answer 2

Since MATLAB R2017a, you can use evaluateDetectionMissRate function.

[logAverageMissRate,fppi,missRate] = evaluateDetectionMissRate(detectionResults,groundTruthData) 

This function returns data points for plotting the log MR-FPPI curve. (MR: Miss-Rate, FPPI: False Positive Per Image).

For an example of its usage, type doc evaluateDetectionMissRate command in MATLAB or go to here.

Answer 3

You can use the following GitHub repo for plotting MR vs FPPI. It may seem like the code only calculates the mAP but it does much more than that. It also calculates the miss-rate, false positives per image and the log-average miss rate. All these are computed in the main.py file present in the repo(line 81) but are not plotted. All you have to do is just plot the MR vs FPPI using matplotlib (or any other module). Just follow the ReadMe file to get started. Hope this helps!