I am really new to object detection, so sorry if this question seems too obvious.
I have a trained FasterRCNN model on detectron2 that detects objects, and I am trying to extract the features of each detected object for the output of my model's prediction. The tutorials available seems to look for ROI and make new predictions, along with their boxes and features. I have the boxes from inference, I just need to extract the features of each box. I added the code I've been working with below. Thank you
# Preprocessing
images = predictor.model.preprocess_image(inputs) # don't forget to preprocess
# Run Backbone Res1-Res4
features = predictor.model.backbone(images.tensor) # set of cnn features
#get proposed boxes + rois + features + predictions
# Run RoI head for each proposal (RoI Pooling + Res5)
proposal_boxes = [x.proposal_boxes for x in proposals]
features = [features[f] for f in predictor.model.roi_heads.in_features]
proposal_rois = predictor.model.roi_heads.box_pooler(features, proposal_boxes)
box_features = predictor.model.roi_heads.box_head(proposal_rois)
predictions = predictor.model.roi_heads.box_predictor(box_features)#found here: https://detectron2.readthedocs.io/_modules/detectron2/modeling/roi_heads/roi_heads.html
pred_instances, pred_inds = predictor.model.roi_heads.box_predictor.inference(predictions, proposals)
pred_instances = predictor.model.roi_heads.forward_with_given_boxes(features, pred_instances)
# output boxes, masks, scores, etc
pred_instances = predictor.model._postprocess(pred_instances, inputs, images.image_sizes) # scale box to orig size
# features of the proposed boxes
feats = box_features[pred_inds]
proposal_boxes = proposals[0].proposal_boxes[pred_inds]
This question has already been well discussed in this issue: https://github.com/facebookresearch/detectron2/issues/5.
The documentation also explains how to achieve this.