image aling with cv2 instead of HOG

Question

Hi I am working on facial recognition.

To increase performance I want to use facial alignment.

When I use the HOG face identifier, described e.g., by Adrian I get an aligned image out.

from imutils.face_utils import rect_to_bb
from dlib import get_frontal_face_detector

detector = dlib.get_frontal_face_detector()
shape_predictor = dlib.shape_predictor('/home/base/Documents/facial_landmarks/shape_predictor_5_face_landmarks.dat')
fa = face_utils.facealigner.FaceAligner(shape_predictor, desiredFaceWidth=112, desiredLeftEye=(0.3, 0.3))

img=cv2.imread(pathtoimage)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) 
        rects = detector(gray, 2)  

for rect in rects:                
            (x, y, w, h) = rect_to_bb(rect)
            faceAligned = fa.align(img, gray, rect)

However, I have to work on an embedded hardware and the HOG facial recognition is not fast enough. The best working is the cv2 lbpcascader.

With cv2 I also get the box of the found face, but using that works not.

faces_detected = face_cascade.detectMultiScale(img, scaleFactor=1.1, minNeighbors=4) 

In other examples using the HOG, the coordinates are extracted from the HOG-rect with:

(x, y, w, h) = rect_to_bb(rect)

and then used with

aligned_face = fa.align(img, gray, dlib.rectangle(left = x, top=y, right=w, bottom=h))

The idea would be to exchange the x,y,w,h with the cv2 values. Unfortunately, that does not work as the two lines above result in a complete false alignment. In the first code example, the rect_to_bb function is included but not used.

I checked the values and they are somehow off:

• 224x224 the image
• 156 70 219 219 the cv2 values (slightly different of course)
• 165 101 193 193 the rect values with rect_to_bb
• [(165, 101) (358, 294)] the rect values

I checked the rect_to_bb function, but this seems straight forward:

def rect_to_bb(rect):
    # take a bounding predicted by dlib and convert it
    # to the format (x, y, w, h) as we would normally do
    # with OpenCV
    x = rect.left()
    y = rect.top()
    w = rect.right() - x
    h = rect.bottom() - y

    # return a tuple of (x, y, w, h)
    return (x, y, w, h)

Answer 1

While typing I got the answer... classic

the alignment function needs the bounding box marks slightly different. It can be seen in the rect_to_bb() function.

def rect_to_bb(rect):
    # take a bounding predicted by dlib and convert it
    # to the format (x, y, w, h) as we would normally do
    # with OpenCV
    x = rect.left()
    y = rect.top()
    w = rect.right() - x
    h = rect.bottom() - y

    # return a tuple of (x, y, w, h)
    return (x, y, w, h)

There the rect.right (w in cv2) and the rect.bottom (h in cv2) are subtracted with x and y. So in the alignment function you have to add the values, otherwise the image fed to the alignment function is much to small and out of shape. And this can also be the values from the cv2 detection.

aligned_face = fa.align(img, gray, dlib.rectangle(left = x, top=y, right=w+x, bottom=h+y))

Keep healthy