WebRTC 'playoutDelayHint' automaticle synchronizes all tracks

Question

I wrote a simple application that streams from one master to several clients. Since the Master may use something like an IP-Webcam (Has ~1sec Latency) but the internal microphone (No Latency) i wanted to add a delay to the audiotrack. Unfortunately it seems like the delay does not work on Firefox and on chrome it automaticle synchronizes all tracks to the highest set playoutDelayHint. So everything becomes delayed one second. I checked both consumer RTPreceivers values for both tracks, only audio has set playoutDelayHint to one second which doesn't change over time, but after a few secons streaming the video becomes delayed for one second too.

const stream = new MediaStream;
    [...]
let el = document.querySelector('#remote_video');
    [...]
function addVideoAudio(consumer) {
  if (consumer.kind === 'video') {
      el.setAttribute('playsinline', true);
      consumer._rtpReceiver.playoutDelayHint = 0;
  } else {
      el.setAttribute('playsinline', true);
      el.setAttribute('autoplay', true);
      consumer._rtpReceiver.playoutDelayHint = 1;
  }
    stream.addTrack(consumer.track.clone());
    el.srcObject = stream;
    el.consumer = consumer;
}

Even when i add another video element and another mediastream, so every stream (consumer) get's it's own html element i still get the same effect:

const stream1 = new MediaStream;
const stream2 = new MediaStream;
    [...]
let el1 = document.querySelector('#remote_video');
let el2 = document.querySelector('#remote_audio');
    [...]
function addVideoAudio(consumer) {
  if (consumer.kind === 'video') {
      el1.setAttribute('playsinline', true);
      consumer._rtpReceiver.playoutDelayHint = 0;
      stream1.addTrack(consumer.track);
      el1.srcObject = stream1;
      el1.consumer = consumer;
  } else {
      el2.setAttribute('playsinline', true);
      el2.setAttribute('autoplay', true);
      consumer._rtpReceiver.playoutDelayHint = 1;
      stream2.addTrack(consumer.track);
      el2.srcObject = stream2;
      el2.consumer = consumer;
  }
}

Is it possible to delay only one track and why does the delay only (kinda) work on chrome? Thanks in advance. :)

Answer 1

You can use jitterBufferDelayHint to delay the audio.

Weirdly enough, playoutDelayHint on a video delay the video and audio. But to delay the audio only, it seem jitterBufferDelayHint fixes it.

audioReceiver.playoutDelayHint = 1;
audioReceiver.jitterBufferDelayHint = 1;

This behavior might change over time.