Roll, Pitch and Yaw Estimation with Visual Odometry using Monocular Camera

Question

I am currently using OpenCV functions to read a video with the .mp4 extension. This video is from a forward-facing camera on an aerial vehicle. In this video's consecutive frames, I'm detecting keypoints and descriptors. For this purpose, I'm utilizing the ORB detector. I'm detecting keypoints between two frames and finding the transformation matrix. I'm using the knnmatch function for this task. Then, I'm identifying the best matches among these matched keypoints using a threshold. Subsequently, I'm passing these matched keypoints to the estimateAffinePartial2D function to obtain a transformation matrix. From this matrix, I extract the cosine and sine values, which I then input into the atan2 function to calculate the rotation value (the roll angle of the aerial vehicle). When I perform these steps and compare the obtained results with the actual values, I achieve satisfactory outcomes. Now, what I'm looking for from you is guidance on how to use a similar approach to calculate the pitch and yaw angles of the aerial vehicle. I would like you to explain the procedure and provide the relevant code.

#include <opencv2/opencv.hpp>

int main() {
    cv::VideoCapture cap("video.mp4");

    if (!cap.isOpened()) {
        std::cout << "Video file not found." << std::endl;
        return -1;
    }

    cv::Mat prev_frame, curr_frame;
    cap >> prev_frame;

    cv::Ptr<cv::ORB> orb = cv::ORB::create();
    std::vector<cv::KeyPoint> keypoints_prev, keypoints_curr;
    cv::Mat descriptors_prev, descriptors_curr;

    while (true) {
        cap >> curr_frame;
        if (curr_frame.empty()) {
            break;
        }

        orb->detectAndCompute(prev_frame, cv::noArray(), keypoints_prev, descriptors_prev);
        orb->detectAndCompute(curr_frame, cv::noArray(), keypoints_curr, descriptors_curr);

        cv::BFMatcher matcher(cv::NORM_HAMMING, true);
        std::vector<cv::DMatch> matches;
        matcher.match(descriptors_prev, descriptors_curr, matches);

        // 
        std::vector<cv::Point2f> src_pts, dst_pts;
        for (size_t i = 0; i < matches.size(); i++) {
            src_pts.push_back(keypoints_prev[matches[i].queryIdx].pt);
            dst_pts.push_back(keypoints_curr[matches[i].trainIdx].pt);
        }

**        cv::Mat M = cv::estimateAffinePartial2D(src_pts, dst_pts);**

        // 
**        double roll = std::atan2(M.at<double>(1, 0), M.at<double>(0, 0));
        double pitch = std::atan2(-M.at<double>(1, 2), std::sqrt(M.at<double>(0, 2) * M.at<double>(0, 2) + M.at<double>(2, 2) * M.at<double>(2, 2)));
        double yaw = std::atan2(M.at<double>(0, 2), M.at<double>(2, 2));**

        roll = roll * 180.0 / CV_PI;
        pitch = pitch * 180.0 / CV_PI;
        yaw = yaw * 180.0 / CV_PI;

        std::cout << "Roll: " << roll << " degrees" << std::endl;
        std::cout << "Pitch: " << pitch << " degrees" << std::endl;
        std::cout << "Yaw: " << yaw << " degrees" << std::endl;

        prev_frame = curr_frame;

        cv::imshow("Current Frame", curr_frame);
        if (cv::waitKey(30) == 27) {
            break;
        }
    }

    cap.release();
    cv::destroyAllWindows();

    return 0;
}

The example code provides accurate results for the roll value, but the yaw and pitch values are irrelevant. What approach should I take to address this issue?

As I mentioned earlier, I've tried the steps. While the obtained result accurately estimates the roll value, the yaw and pitch values are incorrectly estimated. It's possible that the mathematical calculations I used for deriving the yaw and pitch values from the transformation matrix could be incorrect.