Use "Clock Time" instead of Running Time for GStreamer Pipeline

Question

I have a two GStreamer pipelines, one is like a "source" pipeline streaming a live camera feed into an external channel, and the second pipeline is like a "sink" pipeline that reads from the other end of that channel and outputs the live video to some form of sink.

[videotestsrc] -> [appsink] -----  Serial Channel ------>  [appsrc] -> [autovideosink]
     First Pipeline                                             Second Pipeline

The first pipeline starts from a videotestsrc, encodes the video and wraps it in gdppay payload, and then sinks the pipeline into a serial channel (but for the sake of the question, any sink that can be read from to start another pipeline like a filesink writing to serial port or udpsink), where it is read by the source of the next pipeline and shown via a autovideosrc:

"Source" Pipeline

gst-launch-1.0 -v videotestsrc ! videoconvert ! video/x-raw,format=I420 ! x265enc ! gdppay ! udpsink host=127.0.0.1 port=5004

"Sink" pipeline

gst-launch-1.0 -v udpsrc uri=udp://127.0.0.1:5004 ! gdpdepay ! h265parse ! avdec_h265 ! autovideosink

Note: Given the latency induced using a udpsink/udpsrc, that pipeline complains about timestamp issues. If you replace the udpsrc/udpsink with a filesrc/filesink to a serial port you can see the problem that I am about to describe.

Problem:

Now that I have described the pipelines, here is the problem: If I start both pipelines, everything works as expected. However, if after 30s, I stop the "source" pipeline, and restart the pipeline, the Running Time gets reset back to zero, causing the timestamps of all buffers to be sent to be considered old buffers by the sink pipeline because it has already received buffers for timestamps 0 through 30s, so the playback on the other end won't resume until after 30s:

Source Pipeline: [28][29][30][0 ][1 ][2 ][3 ]...[29][30][31]
Sink Pipeline:   [28][29][30][30][30][30][30]...[30][30][31]
     ________________________^ 
     Source pipeline restarted
                             ^^^^^^^^^^^^^^^^...^^^^^^^^
                             Sink pipeline will continue 
                             to only show the "frame"
                             received at 30s until a 
                             "newer" frame is sent, when
                             in reality each sent frame
                             is newer and should be shown
                             immediately.

Solution

I have found that adding sync=false to the autovideosink does solve the problem, however I was hoping to find a solution where the source would send its timestamps (DTS and PTS) based on the Clock time as seen in the image on that page.

I have seen this post and experimented with is-live and do-timestamp on my video source, but they do not seem to do what I want. I also tried to manually set the timestamps (DTS, PTS) in the buffers based on system time, however to no avail.

Any suggestions?

Answer 1

I think you should just restart the receiver pipeline as well. You could add the -e switch to the sender pipeline and when you stop the pipeline it should correctly propagate EOS via the GDP element to the receiver pipeline. Else I guess you can send a new segment or discontinuity to the receiver. Some event has to be signaled though to make the pipeline aware of that change, else it is somewhat bogus data. I'd say restarting the receiver is the simplest way.

Answer 2

GstBaseSrc has do-timestamp property which does exactly what you want. When you set do-timestamp=true, most live sources will then use the clock time to timestamp the buffers, if they honor it.

Here's an example from Aravis: https://github.com/AravisProject/aravis/commit/d1cbf643e5251d26199d799d98b8d05d1ad255a7

Here's the property I'm talking about: https://gstreamer.freedesktop.org/documentation/base/gstbasesrc.html?gi-language=c#GstBaseSrc:do-timestamp