JavaCV DNN RetinaFace Model how to get the face and landmark information

110 Views Asked by At

I am using the JavaCV DNN module to build a face detector using the RetinaFace model. Here is the model: https://github.com/serengil/retinaface I am referring to the OpenCV C++ version to develop the Java version. However, I am not very familiar with Java and Python, so when I use JavaCV, I am having trouble understanding the network output.

Here is the code I have used:

String model="model/retinaface/mnet.caffemodel";
String proto="model/retinaface/mnet.prototxt";
Mat image = opencv_imgcodecs.imread(testPhoto);
net = opencv_dnn.readNetFromCaffe(proto, model);
Mat newImage = new Mat();
resize(image, newImage, new Size(640, 640));
Mat blob = blobFromImage(newImage, 1.0, new Size(640, 640), new Scalar(104.0, 117.0, 123.0, 0), false, false, CV_32F);       
net.setInput(blob);
Mat output = net.forward();

StringVector outLayerNames = net.getUnconnectedOutLayersNames();
for (long i=0; i< outLayerNames.size(); i++) {
    LOGGER.info("Index {} out layer name: {}", i, outLayerNames.get(i).getString());
}

LOGGER.info("Output mat: {}", output);
LOGGER.info("Output mat dimensionality: {}", output.dims());
LOGGER.info("Output mat size width: {}", output.size().width());
LOGGER.info("Output mat size height: {}", output.size().height());
LOGGER.info("Output mat size 0: {}", output.size(0));
LOGGER.info("Output mat size 1: {}", output.size(1));
LOGGER.info("Output mat size 2: {}", output.size(2));
LOGGER.info("Output mat size 3: {}", output.size(3));
LOGGER.info("Output mat rows: {}", output.rows());
LOGGER.info("Output mat cols: {}", output.cols());

here are the debug output information:

Index 0 out layer name: face_rpn_bbox_pred_stride32
Index 1 out layer name: face_rpn_landmark_pred_stride32
Index 2 out layer name: face_rpn_cls_prob_reshape_stride32
Index 3 out layer name: face_rpn_bbox_pred_stride16
Index 4 out layer name: face_rpn_landmark_pred_stride16
Index 5 out layer name: face_rpn_cls_prob_reshape_stride16
Index 6 out layer name: face_rpn_bbox_pred_stride8
Index 7 out layer name: face_rpn_landmark_pred_stride8
Index 8 out layer name: face_rpn_cls_prob_reshape_stride8
 Output mat: org.bytedeco.opencv.opencv_core.Mat[width=-1,height=-1,depth=32,channels=1]
Output mat dimensionality: 4
Output mat size width: 4
Output mat size height: 1
Output mat size 0: 1
Output mat size 1: 4
Output mat size 2: 80
Output mat size 3: 80
Output mat rows: -1
Output mat cols: -1

The code does not have any errors, but I do not know how to get the face and landmark information. Can someone help me? I have googled for a long time but have not found any valuable information.

0

There are 0 best solutions below