Find corners from contour and warp perspective

Question

I have an image like this:

Next i use some techniques to get the contours of the sudoku grid. for demonstration, here's a picture: (green=boundingbox, red=contour)

Now my question; How can i warp this to a perfect square? I have tried to use cvWarpPerspective but i can't seem to figure out how to get the corners from the contours(red line).

Any help would be greatly appreciated!

Answer 1

OK. You have the contour of the puzzle trapezoid. You have the bounding box. You want a nice looking square. This is a subjective part of your system. You can, for example, just snap the target height to the width:

Rect target(0,0,boundbox.width,boundbox.width);

How to actually transform the trapezoid into a square?

First, crop/set the roi to the target:

Mat cropped = source_image(target);

Then, you find the homography matrix, using the source and target quadrangles:

Point SourceTrapezoid[4]; // the trapezoid points, with boundbox.x and y subtracted
Point TargetSquare[4]; // 0,0, roi.width, roi.height
Mat homography_mat = findHomography(InputArray srcPoints, InputArray dstPoints)

Note that the points in TargetSquare must be listed in the same order as in SourceTrapezoid (for example, all in clockwise direction).

Next, just apply the transformation:

Mat transformed;
perspectiveTransform(cropped, transformed, homography_mat);

And copy the transformed box into its place:

transformed.copyTo(source_image(target));

I gave you an example with the c++ opencv api, but the c api is equivalent. Check the opencv documentation for the methods' equivalents.

And as user LSA added in the comments, there are a ton of examples on the web. But the steps for "transform quadrilateral into rectangle" are very simple:

1. Decide on the target rectangle's aspect ratio and placement(if aplicable)
2. Order the points of the target rectangle correctly as the source quadrangle
3. Calculate the homography
4. Apply the perspective transform
5. Put the resultant image where you want it(if applicable)