I'm looking to host a live gaming stream with a group of friends and broadcast to a popular streaming service.
I want minimal delay and maximum sync between each of the 4 participants. I have setup an rtmp media server on my end (using node-media-server) which the participants can send their captured video via OBS, and I have provided them with instructions on achieving the lowest latency on their end.
I have also setup a chat server on discord.
So now I have 4 rtmp video streams being delivered to my media server and a combined audio feed via discord.
I plan to write a simple web application which will consist of 4 video elements in a grid, as well as a control panel for my purposes (volume for each video, notifications, sound effects, etc. for viewers), which I then capture in OBS, and send to a a popular streaming service called Twitch (you might have heard of it)
I initially tried using a library called flv.js but it had several issues. On first loading a video stream, there was a few seconds of stuttering as it appeared to stabilize, and occasionally it would buffer for a moment, slowly allowing the stream to go out of sync with the others. The video in the browser was about 3 seconds behind what I was receiving (I was sending said video stream to myself via the rtmp media server). Not a huge issue, but preferably I would want to display the incoming video as quickly as possible to align with the discord feed.
I then tried altering my media server to output HLS and used a library called (appropriately) hls.js for the browser. Although this was very stable with no stuttering, the delay when it was played back in the browser was enormous, around 20-30 seconds.
Further research led me to believe that rtmp or hls protocols would always have a significant delay during playback.
Disregarding the remote video sources - let's just consider that I have 4 local media sources on my rtmp media server... this is all happening locally on my PC, so there is no network delay to consider.
How can I get these local video sources to play all at once in a browser with the lowest possible latency?
HLS is segmented... audio/video are recorded for a few seconds, then uploaded, then downloaded, buffered, and played back. This has high latency by its nature.
RTMP can have much lower latency, but as you've seen, it isn't designed for low latency. It won't try to catch-up to live on its own.
WebRTC is the technology you'll want to use. The easiest way to do this is use web-based capture on the encoders of the others. (There are some cheap $11 HDMI capture USB dongles out there right now, which make this even easier.) Then, it's just web pages everywhere, and the whole WebRTC stack takes care of keeping the latencies low for you.