I am working on an implementation of the Faster R-CNN object detection algorithm in Python from scratch. I am using TensorFlow to create the Region Proposal Network (RPN). When I train the network, the loss slowly gets close to 0, then decreases into negative numbers. I am not sure why. I think that this negative loss is indicative of an underlying problem, which is probably why my RPN produces inaccurate outputs. I tried to visualize the bounding boxes with the highest confidence generated by my RPN. The boxes were completely incorrect. I am trying to fix the negative loss, which may be the reason for inaccuracy I am facing.
I am absolutely not sure why this is happening. I am using BCE for the region classification loss (to determine if the region contains an object or not). For this, I am using a sigmoid activation function, which should only produce outputs between 0 and 1. For the bounding box transformations, I am using a linear activation function and a SmoothL1 loss function.
In my loss function, I confirmed that I do not mix up the transformation predictions and the region proposal classification predictions, so this is not the source of the problem. Inside my loss function, I used tf.print to print out the transform and classification prediction tensors. The classification tensors only contained numbers between 0 and 1, while the transformation predictions contained both positive and negative numbers (which are correct). This is how I know that the transformation predictions and the region proposal classification predictions were not mixed up.
I suspect that the BCE loss is the one that is negative, because the SmoothL1 loss has an absolute value function that would prevent it from being negative.
My code is accessible here: https://colab.research.google.com/drive/1EI9S8d78xjPb2j227HcJ2iz76LrBSR5I#scrollTo=l68DAoAQTqmD&uniqifier=6. It's not too long.
I am new to TensorFlow and to machine learning in general. Any help would be very much appreciated. I have invested a lot of time into researching and trying to diagnose the problem without any avail. I would like to thank you for your help and time in advance, as I am a bit loss-t (no pun intended)!