Opencv - Haar cascade - Face tracking is very slow

Question

I have developed a project to tracking face through camera using OpenCV library. I used haar cascade with haarcascade_frontalface_alt.xml to detect face.

My problem is if image capture from webcame doesn't contain any faces, process to detect faces is very slow so images from camera, which are showed continuosly to user, are delayed.

My source code:

void camera() 
{
    String face_cascade_name = "haarcascade_frontalface_alt.xml";
    String eye_cascade_name = "haarcascade_eye_tree_eyeglasses.xml";
    CascadeClassifier face_cascade;
    CascadeClassifier eyes_cascade;
    String window_name = "Capture - Face detection";
    VideoCapture cap(0);

    if (!face_cascade.load(face_cascade_name))
        printf("--(!)Error loading\n");

    if (!eyes_cascade.load(eye_cascade_name))
        printf("--(!)Error loading\n");

    if (!cap.isOpened()) 
    {
        cerr << "Capture Device ID " << 0 << "cannot be opened." << endl;
    } 
    else 
    {
        Mat frame;
        vector<Rect> faces;
        vector<Rect> eyes;
        Mat original;
        Mat frame_gray;
        Mat face;
        Mat processedFace;

        for (;;) 
        {
            cap.read(frame);
            original = frame.clone();    
            cvtColor(original, frame_gray, CV_BGR2GRAY);
            equalizeHist(frame_gray, frame_gray);
            face_cascade.detectMultiScale(frame_gray, faces, 2, 0,
                    0 | CASCADE_SCALE_IMAGE, Size(200, 200));

            if (faces.size() > 0)
                rectangle(original, faces[0], Scalar(0, 0, 255), 2, 8, 0);

            namedWindow(window_name, CV_WINDOW_AUTOSIZE);
            imshow(window_name, original);
        }

        if (waitKey(30) == 27)
            break;
    }
}

Answer 1

I use Haar cascade classifiers regularly, and easily get 15 frames/second for face detection on 640x480 images, on an Intel PC/Mac (Windows/Ubuntu/OS X) with 4GB Ram and 2GHz CPU. What is your configuration?

Here are a few things that you can try.

1. You don't have to create the window (namedWindow(window_name, CV_WINDOW_AUTOSIZE);) within each frame. Just create it first and update the image.

2. You can try how fast it runs without histogram equalization. Not always required with a webcam.

3. As suggested by Micka above, you should check whether your program runs in Debug mode or release mode.

4. Use a profiler to see whether the bottleneck is.

5. In case you haven't done it yet, have you measured the frame rate you get if you comment out face detection and drawing rectangles?

Answer 2

Haar classifier is relatively slow by nature. Furthermore, there is not much of optimization you can do to the algorithm itself because detectMultiScale is parallelized in OpenCV.

The only note about your code: do you really get some faces ever detected with minSize which equals to Size(200, 200)? Though surely, the bigger the minSize - the better the performance is.

Try scaling the image before detecting anything:

const int scale = 3;
cv::Mat resized_frame_gray( cvRound( frame_gray.rows / scale ), cvRound( frame_gray.cols / scale ), CV_8UC1 );
cv::resize( frame_gray, resized_frame_gray, resized_frame_gray.size() );
face_cascade.detectMultiScale(resized_frame_gray, faces, 1.1, 3, 0 | CASCADE_SCALE_IMAGE, Size(20, 20));

(don't forget to change minSize to more reasonable value and to convert detected face locations to real scale)

Image size reducing for 2, 3, 5 times is a great performance relief for any image processing algorithm, especially when it comes to some costly stuff like detection.

As it was mentioned before, if resizing won't do the trick, try fetching some other bottlenecks using a profiler.

And you can also switch to LBP classifier which is comparably faster though less accurate.

Hope it will help.

Answer 3

May be it will useful for you:

There is a Simd Library, which has an implementation of HAAR and LBP cascade classifiers. It can use standard HAAR and LBP casscades from OpenCV. This implementation has SIMD optimizations with using of SSE4.1, AVX2 and NEON(ARM), so it works in 2-3 times faster then original OpenCV implementation.

Answer 4

You can use LBP Cascade to detect faces. It is much more lightweight. You can find lbpcascade_frontalface.xml in OpenCV source directory.