I wish to detect and track the mouse cursor in a screen recording video. My goal is to track the mouse cursor in the video and know about the events like closing of a window, exiting an application etc.
Use of normal CV algorithms to track objects does not work as the cursor and background are both dynamic and also there is not a very clear contrast between the two.
Is there a way to tackle this problem?
On
You could probably detect the pointer using a HAAR Classifier: It is a machine learning based approach where a cascade function is trained from a lot of positive and negative images. It is then used to detect objects in other images. OpenCV already has trained HAAR Classifiers built in for faces, eyes etc. but it should be possible to train your own for the cursor.
You can find a tutorial on how to train your own classifier here (how to do this is too extensive to include in this answer).
On
I use frame differencing for this problem combined with connected component analysis.
If the images are uncompressed, all pixel values of the cursor are known exactly and do not change as the cursor is moving (unless there is antialising or transparency on the edges ?).
Then an exact pattern search should do. (Slide the cursor to every possible position and check the matching of the pixels, until all pixels do match.)
If you arrange the search in such a way that the first cursor pixel tried has a different color than the background (the most frequent color), you will maximize the probability of immediate rejection. Even better if the first two pixels tried have different colors. By doing so, the number of pixels tried will get close to twice the image size, which is hard to beat.
Added by Mark Setchell below this line:
It seems cursors do change so you will need to take account of this: