OpenCV : OpenCVForUnity MediaPipe: MediaPipeUnityPlugin
I have a rotation matrix from face landmarks (generated by MediaPipe) using solvePnP called rvec (below). I have tested that the resultant rotation is accurate in estimating head pose. However, I want to use this matrix to rotate the MatOfPoint3f object containing the face landmarks back to a "forward facing" direction. I have tried using OpenCVForUnity.CoreModule.Core.transform(MatLandmarks, MatLandmarks, rmat.inv()) but the transformed values still are affected by the movement of the head. Any ideas?
Calib3d.solvePnP(object_points, image_points, camMatrix, distCoeffs, rvec, tvec);
Calib3d.Rodrigues(rvec, rmat);