How to find the right point correspondance between template and a rotated image?

Question

I have a .dxf file containing a drawing (template) which is just a piece with holes, from said drawing I successfully extract the coordinates of the holes and their diameters given in a list [[x1,y1,d1],[x2,y2,d2]...[xn,yn,dn]].

After this, I take a picture of the piece (same as template) and after some image processing, I obtain the coordinates of my detected holes and the contours. However, this piece in the picture can be rotated with respect to the template.

How do I do the right hole correspondance (between coordinates of holes in template and the rotated coordinates of holes in image) so I can know the which diameter corresponds to each hole in the image?

Is there any method of point sorting it can give me this correspondence? I'm working with Python and OpenCV.

All answers will be highly appreciated. Thanks!!! Image of Template: https://ibb.co/VVpWmKx In the template image, contours are drawn to the same size as given in the .dxf file, which differs to the size (in pixels) of the contours of the piece taken from camera.

Processed image taken from the camera, contours of the piece are shown: https://ibb.co/3rjCg5F

I've tried OpenCV functions of feature matching (ORB algorithm) so I can get the rotation angle the piece in picture was rotates with respect to the template? but I still cannot get this rotation angle? how can I get the rotation angle with image descriptors?

is this the best approach for this problem? are there any better methods to address this problem?

Answer 1

Considering the image of the extracted contours, you might not need something as heavy as the feature matching algorithm of the OCV library. One approach would be to take the most outter contour of the piece and get the cv::minAreaRect of it. Resulting rotated rectangle will give you the angle. Now you just have to decide if the symmetry matches, because it might be flipped. That can be done as well in many ways. One of the most simple one (excluding the fact, the scale might be off) is that you take the most outter contour again, fill it and count the percentage of the points that overlay with the template. The one with right symmetric orientation should match in almost all points. Given that the scale of the matched piece and the template are the same.

Answer 2

emm you should use huMoments which gives translation， scale and rotation invariance descriptor for matching.

The hu moment can be found here https://en.wikipedia.org/wiki/Image_moment. and it is implemented in opencv

you can dig up the theory of Moment invariants on the wiki site pretty easily

to use it you can simply call

// Calculate Moments
Moments moments = moments(im, false);

// Calculate Hu Moments
double huMoments[7];
HuMoments(moments, huMoments); 

The sample moment will be

h[0] = 0.00162663
h[1] = 3.11619e-07
h[2] = 3.61005e-10
h[3] = 1.44485e-10
h[4] = -2.55279e-20
h[5] = -7.57625e-14
h[6] = 2.09098e-20

Usually, here is a large range of the moment. There usually coupled with a log transform to lower the dynamic range for matching

H=log(H)

H[0] = 2.78871
H[1] = 6.50638
H[2] = 9.44249
H[3] = 9.84018
H[4] = -19.593
H[5] = -13.1205
H[6] = 19.6797

BTW， you might need to pad the template to extract the edge contour