What is the best way to store a ring buffer of images

Question

I have two cameras with 4MP at 60 fps. In case of an external trigger, I have to make two videos of the last 2 minutes for each camera. The cameras are basler cameras and I'm using pypyplon to get the images and using OpenCV to process them.

For now, I am storing the last 10 seconds of each camera into my RAM on ring buffers. As the external trigger occurs, I create the videos from the ring buffers. This uses all my RAM and I was wondering how I can store the 2 minutes instead of just buying more RAM. I tried to convert them into jpgs to reduce the RAM usage but it was too slow for 60fps : cv2.imencode('.jpg', image, encode_param)

These are the ideas I've had for improvement so far:

1. Keep the ring buffers in the Disk. This would reduce the RAM usage. But I am concerned about the r/w speed and the lifetime of the SSD as the program will be running for a very long time.
2. Compress the images on a GPU and keep them in the buffer.
3. Compress the images using gstreamer.

Is there a standard way of solving problems like this?

Thanks in advance.

Answer 1

So you're building something like a dashcam that records video in a loop and saves the loop when a button is pressed.

I wouldn't even try to keep the data in RAM. Or to save individual images.

You should constantly write video to disk/storage. Write as MPEG transport stream (.MTS/.m2ts), not MP4. Incomplete MP4 files aren't readable anymore, but MTS files are.

Write segments of several seconds. Delete those segment files you don't want to keep.

Later, if anything wants a single video of those pieces, run ffmpeg with -c copy on the segments... or, if it's .MTS, you can just concatenate the files.

MJPEG is an actual video codec, not comparable to individual imencode calls. It might be faster than what you did so far.

OpenCV can now use hardware acceleration through ffmpeg, if you ask for it: https://docs.opencv.org/4.x/db/dc4/group__videoio__hwaccel.html