I have been working on a Door detector in an uncontrolled environment. I used mask rcnn with to get the mask and bounding boxes. Though the class accuracy is very good, the mask is not very accurate. Also the expected result is a bounding quadrilateral (get the box connecting the exact 4 corners of the door ) but the bounding box returned is rectangular. I tried taking the mask, finding the convex hull and then simplify the hull using cv2.approxPolyDP to get the quadrilateral. In some cases it's giving great output where the mask is accurate but doesn't work for all specifically when there is an occlusion or the mask obtained is not very uniform around the edges.
Using Oriented Bounding Box detector also gives a rotated rectangle not quadrilateral. I would like to have some suggestion regarding how can I achieve the required result.
TIA