Record multiple RTSP streams into a single file

Question

I need to record 4 RTSP streams into a single file.

Streams must be placed into the video in this way:

 ---------- ---------- 
|          |          |
| STREAM 1 | STREAM 2 |
|          |          |
|----------|----------|
|          |          |
| STREAM 3 | STREAM 4 |
|          |          |
 ---------- ----------

I need to synchronize these live streams with about ~1 second accuracy. This is challenging because streams have variable framerate (FPS).

I have tried ffmpeg but streams are not synchronized. Here is the code:

ffmpeg \
  -i "rtsp://IP-ADDRESS/cam/realmonitor?channel=1&subtype=00" \
  -i "rtsp://IP-ADDRESS/live?real_stream" \
  -i "rtsp://IP-ADDRESS/live?real_stream" \
  -i "rtsp://IP-ADDRESS/live?real_stream" \
  -filter_complex " \
    nullsrc=size=1920x1080 [base]; \
    [0:v] scale=960x540 [video0]; \
    [1:v] scale=960x540 [video1]; \
    [2:v] scale=960x540 [video2]; \
    [3:v] scale=960x540 [video3]; \
    [base][video0] overlay=shortest=1:x=0:y=0 [tmp1]; \
    [tmp1][video1] overlay=shortest=0:x=960:y=0 [tmp2]; \
    [tmp2][video2] overlay=shortest=0:x=0:y=540 [tmp3]; \
    [tmp3][video3] overlay=shortest=0:x=960:y=540 [v]; \
    [0:a]amix=inputs=1[a]" \
  -map "[v]" -map "[a]" -c:v h264 videos/test-combine-cams.mp4

Is there a way to combine and synchronize streams in ffmpeg or using other utilities like: vlc, openRTSP, OpenCV?

Answer 1

Have you tried gstreamer, it works with my rtsp streams.

gst-launch-1.0 -e rtspsrc location=rtsp_url1 ! rtph264depay ! h264parse ! decodebin ! videoconvert ! m.sink_0 \
               rtspsrc location=rtsp_url2 ! rtph264depay ! h264parse ! decodebin ! videoconvert ! m.sink_1 \
               rtspsrc location=rtsp_url3 ! rtph264depay ! h264parse ! decodebin ! videoconvert ! m.sink_2 \
               rtspsrc location=rtsp_url4 ! rtph264depay ! h264parse ! decodebin ! videoconvert ! m.sink_3 \
               videomixer name=m sink_1::xpos=1280 sink_2::ypos=720 sink_3::xpos=1280 sink_3::ypos=720 ! x264enc ! mp4mux ! filesink location=./out.mp4 sync=true

Of course you will need to add in your rtsp urls and adjust the videomixer xpos/ypos properties based on your video size (mine was 720p).

Before mixing you may want to run just one at a time to make sure you have all the dependencies installed correctly

gst-launch-1.0 rtspsrc location=rtsp_url1 ! rtph264depay ! h264parse ! decodebin ! x264enc ! mp4mux ! filesink location=./out.mp4 sync=true

I have not yet added the audio.