I have some screencasts that explain how to perform a task in a specific software. My objective is to detect where the user clicks on a button or menu item or anything else that leads to a change. One solution to this work is to detect the mouse cursor's location. There are several challenges:
- The cursor's icon changes and is not always the same in all videos (e.g. Mac vs Win, arrow shaped and hand shaped cursors).
- I tried template matching, but I did not get good results because the display settings of each person who is capturing the video may be different and therefore the size of the cursor will be different.
- Calculating the difference between two consequent images can give the mouse cursors only in the output but I need to have only the second or the last image's cursor's location in the output not both of them.
- I also tried to find an object tracking sample solution, but they are either for live videos or for multiple objects ( I only need to spot the mouse cursor or the locations where the mouse was clicked on)
I would appreciate if if anyone could suggest a solution, ready to use code (in Java/Matlab/Python), software or API for this work.