ffmpeg switch inputs / mapping on-the-fly / while recording

Question

I have a rtsp video-source stream1 and an audio source I currently merge and send to a rtmp-server using:

stream1="rtsp://streamurl1"

/usr/bin/ffmpeg                    \
    [...]
    -i "$stream1"                  \
    [...]
    -itsoffset $AUDIOVIDEOOFFSET   \
    -f pulse                       \
    -i default                     \
    [...]
    -vcodec copy                   \
    -map 0:v -map 1:a              \
    [...]
    -f flv "rtmp://streamingserver"

I would now like to add a second video source stream2 and switch between stream1 and stream2 back and forth without interrupting the audio. Both streams are identical / come from identical cameras.

Is there any sane way to do this with ffmpeg? Or how would you recommend doing it?

Just stopping the process and restarting it using stream2 instead of stream1 works but results in several seconds outage on the stream and is the current worst case scenario I would like to improve.

Answer 1

I couldn't get it to work with pure ffmpeg in a reasonable amount of time but the nginx-rtmp module worked out of the box.

It's basically apt install libnginx-mod-rtmp nginx, add ..

rtmp {
        server {
                listen 1935;
                chunk_size 4096;

                application live {
                        live on;
                        record off;
                        # Only localhost is allowed to publish
                        allow publish 127.0.0.1;
                        deny publish all;
                }
        }
}

.. to nginx.conf, systemctl restart nginx and nginx acts as a local loop-back device on rtmp://localhost/live.

If you now have n streams to multiplex, you can create n systemd services camera_i.service

[Unit]
Description=Send stream i to local nginx loopback device
# List all other
Conflicts=camera_1.service, camera_2.service, [...]

[Service]
ExecStart=/usr/bin/ffmpeg -i rtsp://mystreamurl_i -c copy -f flv rtmp://localhost/live
Type=simple
Restart=on-failure
KillSignal=SIGINT
SuccessExitStatus=255

[Install]
WantedBy=xsession.target

and switch to stream i by starting camera_i.service:

systemctl --user start camera_i.service

All other services listed on the Conflicts= option are automatically being terminated by systemd, effectively multiplexing the camera streams.

The resulting stream can then be recorder used using something like the following:

/usr/bin/ffmpeg                    \
    [...]
    -use_wallclock_as_timestamps 1 \
    -fflags +genpts                \
    -i "rtmp://localhost/live"     \
    [...]
    -itsoffset $AUDIOVIDEOOFFSET   \
    -f pulse                       \
    -i default                     \
    [...]
    -vcodec copy                   \
    -map 0:v -map 1:a              \
    [...]
    OUTPUT

The result is a rather smooth flip with continuous audio.

-use_wallclock_as_timestamps 1 -fflags +genpts might be unnecessary if you don't use -vsync 0. But I haven't tested that yet.

---

Appendix

As has been proposed on the ffmpeg mailing-list, there is the zmq filter that supposedly can change filters settings on-the-fly. The idea is to overlay two streams and toggle the opacity of the top stream, effectively switching stream.

I couldn't get it to work but for anyone to try:

# compiled with --enable-libzmq
ffmpeg -i INPUT -filter_complex 'null[main];movie=INPUT2,zmq,lumakey@toggle=tolerance=1,[main]overlay,realtime' OUTPUT

echo lumakey@toggle tolerance 0 | zmqsend

where zmqsend results from:

git clone https://github.com/FFmpeg/FFmpeg
cd FFmpeg/
./configure
cd tools/
gcc zmqsend.c -o zmqsend -I.. `pkg-config --libs --cflags libzmq libavutil`