Very high CPU usage when using opencv2 with multithreading in python

Question

I am trying to create automatic attendance system with opencv2 in which i need to get rtsp stream from IP camera, find faces from it and recognize face.

I created different threads from frame catching and drawing because face recognition function needs some time to recognize face.

But just creating 2 threads, one for frame reading and other for drawing uses around 70% CPU. and creating pytorch_facenet model increase usage 80-90% CPU.

does anyone know how to reduce CPU usage ?

my program:

import cv2
import threading
from facenet_pytorch import InceptionResnetV1

cap = cv2.VideoCapture("rtsp://test:[email protected]")
resnet = InceptionResnetV1(pretrained='vggface2').eval()

ret, frame = cap.read()
exit = False

def th1():
    global ret, frame, exit
    while True:
        ret, frame = cap.read()
        if exit:
            break

def th2():
    global ret, frame, exit
    while True:
        cv2.imshow('frame', frame)
        cv2.waitKey(1)
        if cv2.getWindowProperty('frame',cv2.WND_PROP_VISIBLE) < 1:
            exit = True
            break

t1 = threading.Thread(target=th1)
t1.start()
t2 = threading.Thread(target=th2)
t2.start()

Update:

I used time.sleep(0.2) in my all threads except frame reading. and it worked, my cpu usage is 30% now.

Answer 1

Two issues.

0. th2 runs in an almost-tight-loop. It won't consume a whole core of CPU because waitKey(1) sleeps for some time.

1. No synchronization at all between threads, but you need it. You need a threading.Event to notify the consumer thread of a fresh frame. The consumer thread must wait until a fresh frame is available, because it's pointless to display the same old frame again and again. You can be lazy and use waitKey(30) instead. For the displaying thread, that's good enough.

2. VideoCapture. You don't do any error checking at all! You must check:

cap = cv2.VideoCapture("rtsp://test:[email protected]")
assert cap.isOpened()
...

and

    while True:
        ret, frame = cap.read()
        if not ret:
            break
        ...

Answer 2

This code works.

1. This first loop (thread) will be trying to read frames as fast as it can. The frame can be updated 100 times per second or more, but it's too fast. Try to add time.sleep(0.03).

2. And in the second loop, you can change the waitKey() param to 30.

   import time
   def th1():
       global ret, frame, exit
           while True:
           ret, frame = cap.read()
           time.sleep(0.03)
           if exit:
               break
   
   def th2():
       global ret, frame, exit
       while True:
           cv2.imshow('frame', frame)
           cv2.waitKey(30)
           if cv2.getWindowProperty('frame',cv2.WND_PROP_VISIBLE) < 1:
               exit = True
               break