OpenCV PerspectiveTransform does not work

81 Views Asked by At

I'm trying to use the opencv function called PerspectiveTransform, but I cannot get the function to work. I have have tried to changed data types and sizes but nothing seems to work. This is my code:

// get extrinsic transformation information
    cv::Mat extrinsicCalibration = getTranformations(std::string("..\\ExtrinsicCalibration\\" + sensor_3.serialNo + "\\transformation_Mat.yml")).inv();
    
    // convert bbox points into cv::mat
    cv::Mat points_to_transform = convertToHomogeneousCoords(minBound, maxBound);

    // apply extrinsic transformation to 3D bbox points to represent from camera perspective
    cv::Mat transformed_points(4, 2, CV_32F);
    std::cout << points_to_transform.type() << "  --  "
              << points_to_transform.size() << "\n"
              << extrinsicCalibration.type() << "  --  "
              << extrinsicCalibration.size() << "\n"
              << transformed_points.type() << "  --  "
              << transformed_points.size() << "\n";
    try {
        cv::perspectiveTransform(points_to_transform, transformed_points, extrinsicCalibration);
    } catch (cv::Exception& e) {
        std::cout << "Exception caught: " << e.what() << std::endl;
    }

And here is the function convertToHomogeneousCoords:

cv::Mat convertToHomogeneousCoords(const glm::vec3& point1, const glm::vec3& point2) {
    cv::Mat points_to_transform(4, 2, CV_32F);
    points_to_transform.at<float>(0, 0) = point1.x;
    points_to_transform.at<float>(1, 0) = point1.y;
    points_to_transform.at<float>(2, 0) = point1.z;
    points_to_transform.at<float>(3, 0) = 1.0f; // set w component to 1
    points_to_transform.at<float>(0, 1) = point2.x;
    points_to_transform.at<float>(1, 1) = point2.y;
    points_to_transform.at<float>(2, 1) = point2.z;
    points_to_transform.at<float>(3, 1) = 1.0f; // set w component to 1
    return points_to_transform;
}

When I print out the datatypes and size I get the following output:

5  --  [2 x 4]  
5  --  [4 x 4]  
5  --  [2 x 4] 

Which I think suggest that the datatypes are the same and also that the sizes seems to be correct. The exception that is thrown is:

Unhandled exception at 0x00007FFF680640AC in RAU_Control_Interface_Software.exe: Microsoft C++ exception: cv::Exception at memory location 0x00000032570FD7F0. 

And:

Exception caught: OpenCV(4.6.0) C:\Users\Mikke\source\repos\opencv-4.6.0\modules\core\src\matmul.dispatch.cpp:550: error: (-215:Assertion failed) scn + 1 == m.cols in function 'cv::perspectiveTransform'

Hopefully someone knows what is going on and how to fix this problem :)


EDIT

@Miki made me aware that the functions takes care of converting to homogeneous coords. So I have edited the code to be as follows:

// get extrinsic transformation information
cv::Mat extrinsicCalibration = getTranformations(std::string("..\\ExtrinsicCalibration\\" + sensor_3.serialNo + "\\transformation_Mat.yml")).inv();

// convert bbox points into cv::mat
cv::Mat points_to_transform(2, 3, CV_32F);
points_to_transform.at<float>(0, 0) = minBound.x;
points_to_transform.at<float>(0, 1) = minBound.y;
points_to_transform.at<float>(0, 2) = minBound.z;
points_to_transform.at<float>(1, 0) = maxBound.x;
points_to_transform.at<float>(1, 1) = maxBound.y;
points_to_transform.at<float>(1, 2) = maxBound.z;
// apply extrinsic transformation to 3D bbox points to represent from camera perspective
cv::Mat transformed_points(2, 3, CV_32F);
std::cout << points_to_transform.type() << "  --  "
          << points_to_transform.size() << "\n"
          << extrinsicCalibration.type() << "  --  "
          << extrinsicCalibration.size() << "\n"
          << transformed_points.type() << "  --  "
          << transformed_points.size() << "\n";
try {
    cv::perspectiveTransform(points_to_transform, transformed_points, extrinsicCalibration);
} catch (cv::Exception& e) {
    std::cout << "Exception caught: " << e.what() << std::endl;
}

This results in the type and sizes being:

5  --  [3 x 2]
5  --  [4 x 4]
5  --  [3 x 2]

I still get the error message:

OpenCV(4.6.0) Error: Assertion failed (scn + 1 == m.cols) in cv::perspectiveTransform, file C:\Users\Mikke\source\repos\opencv-4.6.0\modules\core\src\matmul.dispatch.cpp, line 550
Exception caught: OpenCV(4.6.0) C:\Users\Mikke\source\repos\opencv-4.6.0\modules\core\src\matmul.dispatch.cpp:550: error: (-215:Assertion failed) scn + 1 == m.cols in function 'cv::perspectiveTransform'

OpenCV(4.6.0) Error: Assertion failed ((unsigned)(i1 * DataType<_Tp>::channels) < (unsigned)(size.p[1] * channels())) in cv::Mat::at, file C:\Users\Mikke\source\repos\opencv-4.6.0\modules\core\include\opencv2/core/mat.inl.hpp, line 899
0

There are 0 best solutions below