How can I prevent breakup/choppiness/glitches when using an AudioWorklet to stream captured audio?

980 Views Asked by At

We've been working on a JavaScript-based audio chat client that runs in the browser and sends audio samples to a server via a WebSocket. We previously tried using the Web Audio API's ScriptProcessorNode to obtain the sample values. This worked well on our desktops and laptops, but we experienced poor audio quality when transmitting from a handheld platform we must support. We've attributed this to the documented script processor performance issues (https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API). On the handheld, with a script processor buffer size of 2048, audio consistently had breakups. At the next highest size interval (4096), the audio was smooth (no breakups), but there was too much latency (around two seconds).

Our results from ScriptProcessorNode prompted experimentation with Audio Worklet. Unfortunately, with our worklet implementation, audio quality is worse: both breakups and latency, even on our laptops. I'm wondering if there's a way to tweak our worklet implementation to get better performance, or if what we're experiencing is to be expected from (is "par for the course" for) the current state of audio worklets (Chromium issues 796330, 813825, and 836306 seem relevant).

Here's a little more detail on what the code does:

  1. Create a MediaStreamStreamSourceNode with the MediaStream obtained from getUserMedia.
  2. Connect the source node to our worklet node implementation (extends AudioWorkletNode).
  3. Our worklet processor implementation (extends AudioWorkletProcessor) buffers blocks that arrive as the "input" argument to its process method.
  4. When buffer is full, use MessagePort to send the buffer contents to the worklet node.
  5. Worklet node transmits the buffer contents over a WebSocket connection.

The process method is below. The var "samples" is a Float32Array, which gets initialized to the buffer size and reused. I've experimented with buffer size a bit, but it doesn't seem to have an impact. The approach is based on the guidance in section 4.1 of AudioWorklet: The future of web audio to minimize memory allocations.

if (micKeyed == true) {

    if (inputs[0][0].length == framesPerBlock) {
        samples.set(inputs[0][0], currentBlockIndex * framesPerBlock);
        currentBlockIndex++;

        if (currentBlockIndex == lastBlockIndex) {
            // console.log('About to send buffer.');
            this.port.postMessage(samples);
            currentBlockIndex = 0;
        }
    } else {
        console.error("Got a block of unexpected length!!!");
    }
}
return true;

Currently testing with PCs running Chrome 72.0.3626.109 on CentOS 7. Our handhelds are Panasonic FZ-N1 running Chrome 72.0.3626.105 on Android 6.0.1.

Thank you for reading and any suggestions you may be able to provide.

0

There are 0 best solutions below