Converting waveform data to visuals?

137 Views Asked by At

I have a decent understanding of what I am doing but am missing a few details so let me break it down.

First I am converting an audio file to waveform data using audiowaveform then on the client it gets converted to a WaveformData object.

What I understand is that the data contains pairs of min/max waveform data points interleaved

Example

{
  "version": 2,
  "channels": 2,
  "sample_rate": 48000,
  "samples_per_pixel": 512,
  "bits": 8,
  "length": 3,
  "data": [-65,63,-66,64,-40,41,-39,45,-55,43,-55,44]
}

With this understanding if I want to achieve drawing bars like this then I simply take that data and create a bar for every min/max pair, kind of like this.

 63  64  41  45  43  44
-65 -66 -40 -39 -55 -55

Here is how I am achieving this using the waveform-data.js API. I also made an interactive Code Sandbox to make it easy to play with

const Visualizer: FC<IVisualizer> = ({
  currentTime,
  height,
  width,
}) => {
  const canvasRef = useRef<HTMLCanvasElement | null>(null);
  const [waveform, setWaveform] = useState<WaveformData | null>(null);

  // redraw with new time
  const drawCanvas = (waveform: WaveformData) => {
    if (canvasRef.current) {
      const canvas = canvasRef.current;
      const ctx = canvas.getContext("2d");

      if (ctx) {
        ctx.clearRect(0, 0, canvas.width, canvas.height);
        ctx.beginPath();

        const channel = waveform.channel(0);
        const centerY = canvas.height / 2;
        const sDelta = 900; // 900s aka 15 min offset
        const sampleDuration = 0.3; // 300ms of data
        const offsetTime = sDelta + currentTime;
        const startIndex = waveform.at_time(offsetTime);
        const endIndex = waveform.at_time(offsetTime + sampleDuration);
        const length = Math.round(canvas.width); // one bar per px can be improved

        // get min/max given a slice of time offset time + sample duration aka 15 min + 300ms
        const maxData = channel.max_array().slice(startIndex, endIndex);
        const minData = channel.min_array().slice(startIndex, endIndex);

        for (let i = 0; i < length; i++) {
          ctx.moveTo((i + 1) * 15, centerY); // plot axis in center
          ctx.lineTo((i + 1) * 15, centerY - maxData[i]); // draw upwards from center point given max value

          ctx.moveTo((i + 1) * 15, centerY); // plot axis in center
          ctx.lineTo((i + 1) * 15, centerY - minData[i]); // draw downwards from center point given min value
        }

        ctx.lineWidth = 10;
        ctx.lineCap = "round";
        ctx.fillStyle = "#fff";
        ctx.strokeStyle = "#fff";
        ctx.closePath();
        ctx.stroke();
        ctx.fill();
      }
    }
  };

  // currentTime changes at 60fps this effect causes a redraw with new time information
  useEffect(() => {
    if (waveform) {
      drawCanvas(waveform);
    } else {
      console.log("Waveform is missing");
    }
  }, [waveform, currentTime]);

  // get waveform data and convert to WaveformData object on mount
  useEffect(() => {
    fetch(
      "https://s3.us-west-2.amazonaws.com/motionbox.audiowaveform.dev/0163b1e0-7a3c-11ec-8459-8141f656f7bf"
    )
      .then((response) => response.json())
      .then((json) => WaveformData.create(json))
      .then((waveform) => {
        console.log(`Waveform has ${waveform.channels} channels`);
        console.log(`Waveform has length ${waveform.length} points`);
        setWaveform(waveform);
      });
  }, []);

  return (
    <div>
      <canvas ref={canvasRef} width={width} height={height} />
    </div>
  );
};

The Problem

With this code and my lack of understanding here, the x axis aka time moves right to left, I understand it has to do with currentTime and how I am redrawing, but I can't wrap my head around why it's doing this, my mental model isn't clear enough. Here is an example result. Again, this is what I am trying to achieve.

It seems the way the desired visual works is each frame has a fixed amount of bars representing time of some sort, maybe 200ms (not sure how to determine). Then somehow is mapped to the waveform data in a way that doesn't cause movement on the x axis. I know I am close but missing some details.

Basically startIndex and endIndex are constantly moving as currentTime moves. This formula needs to be changed slightly so that it gets every 200ms at a time, rather than shifting indexes 1ms at a time.

Again here is an interactive sandbox -- I enjoyed making it

0

There are 0 best solutions below