For a private project I'm trying to build a keypoint-detection-system for live fish in a watertank. Keypoints are the eyes, tailtip and so on, only in side-view.
So now it comes to image annotation I wonder how to deal with the not-so-clear instances. Should I annotate all instances, even if they are barely visible? What about those that are only partly in the image? What about the missing or bare visible keypoint-areas?
The attached fotos might give a little bit of an idea on what my problem is.
So far I've built a fairly okay model using pytorch's Keypoint-RCNN as a backbone. My focus is not to detect all instances and keypoints in an image, but to only detect the very clear ones (with high confidence on keypoint location). This is why in my first attempt I only annotated "the good ones", so those fish that are:
- fully in the image
- not covered by another fish
- not too far away, so the brightness and size is reasonable
- don't have an angle of more than 45° to the image plane
I also did some augmentation, like brightness variation and flipping. The results are okay but not too good, a lot of the instances which are very well visible are missed by the model.