I am trying to create a disparity map of the images created by the stereo camera mounted on the Valve Index VR headset. I am using OpenVR and OpenCV. OpenVR allows access to the cameras using the IVRTrackedCamera interface. In order to perform OpenCV StereoBM the left and right images need to be rectified.
My first question:
OpenVR allows for acquiring frames using GetVideoStreamFrameBuffer(). This method allows for passing an EVRTrackedCameraFrameType, either Distorted, Undistorted or MaximumUndistorted. What do the different FrameTypes mean? Can I assume the frames are already rectified onto a common plane when using the Undistorted or MaxUndistorted frametypes?
Second question:
If the frames are not yet rectified unto a common plane, how to do so? With OpenVR I can get camera intrinsics for each individual camera using GetCameraIntrinsics(), again supplying an EVRTrackedCameraFrameType. I can also acquire the Distortion Parameters for each individual camera using GetArrayTrackedDeviceProperty(Prop_CameraDistortionCoefficients_Float_Array). Now, the two parameters I am missing for OpenCV's stereoRectify() are:
R – Rotation matrix between the coordinate systems of the first and the second cameras.
T – Translation vector between coordinate systems of the cameras.
Is it possible to acquire these parameters from OpenVR?