Optimizing frame iteration in OpenCV capture: Seeking a faster alternative to the 'set' function

Question

I'm currently working on a project where I need to stream multiple videos from multiple OpenCV captures in a Flask backend to a React frontend. I'm facing an issue with the performance when seeking to a specific frame using the 'set' function in OpenCV.

The problem arises when I try to seek a particular frame using the 'set' function. The current implementation is quite complex and ends up being slow, consuming a significant amount of CPU resources. This negatively impacts the overall streaming experience.

I'm seeking advice and suggestions on how to optimize the frame iteration process in OpenCV to achieve faster seeking to specific frames. I would like to explore alternative approaches or techniques that can improve performance and reduce CPU usage during frame seeking.

Additionally, I'm using Flask as the backend framework and React as the frontend framework. The videos are being streamed from the Flask backend to the React frontend using appropriate APIs and communication channels.

Any insights, code examples, or suggestions on how to efficiently iterate through frames in OpenCV captures for streaming purposes would be greatly appreciated. Thank you in advance for your help!

Note that, the entire project is on the local host and all videos are on the disk and they are being read through Opencv captures using multiple threads.

When I was streaming a single video, I noticed that repeatedly seeking 5 seconds forward or backward using the 'set' function in OpenCV resulted in high CPU usage.

but it wasn't a problem until I needed to stream multiple synchronized videos.

now this is an example of what a single stream thread looks like. the queue is being appended to from another thread

capture = cv2.VideoCapture(video)
    prev_frame_no = -5
    while end_stream == False:
        try:
            frame_info = sec_stream_queues[queue_idx].get(timeout=1.0)
            idx = frame_info['frame']
        except queue.Empty:
            continue
        if idx != prev_frame_no + 1:
            capture.set(cv2.CAP_PROP_POS_FRAMES, idx)
        _, frame = capture.read()
        prev_frame_no = idx
            
        yield (b'--frame\r\n'
               b'Content-Type: image/jpeg\r\n\r\n' + encode_frame(frame, 25) + b'\r\n')
        sec_stream_queues[queue_idx].task_done()
    capture.release()

only on seeking the set function gets called for every stream thread.