How to stream Base64 data into a video

Question

I am trying to get a video to load from a stream. I would like to be able to "play" the video even if the entire video has not "downloaded" yet. The base64 data is coming in as a stream of buffer chunks.

const videoTag = document.getElementById('videoTag');
const mimeCodec = 'video/mp4; codecs="avc1.42E01E, mp4a.40.2"';
const mediaSource = new MediaSource();

videoTag.src = URL.createObjectURL(mediaSource);

await new Promise((resolve, reject) => {
    mediaSource.addEventListener('sourceopen', function (_) {
        resolve();
    });
});

const sourceBuffer = mediaSource.addSourceBuffer(mimeCodec);


for await (const chunk of ipfsNode.cat(CID)) {//Chunk is coming in as an array buffer
    sourceBuffer.appendBuffer(chunk);
}

sourceBuffer.addEventListener('updateend', async function (_) {
    mediaSource.endOfStream();
    $('#videoTag')[0].load();
});

I've tried using new MediaSource() and sourceBuffer.appendBuffer(chunk); to push new downloaded chunks to the video, but not only does the video not ever play, the method I am using requires that the entire video downloads before it can play.

How can I get my chunks of base64 data into a video?

EDIT (in response to comments): Regardless of whether or not I use a String or binary steam, I am a little confused on how I get a stream into a video.

Answer 1

You have two problems: 1. how to add arbitrary data to a video stream? 2. how to stream?

1. Given that you use a mp4 container (const mimeCodec = 'video/mp4; codecs="avc1.42E01E, mp4a.40.2"';) you can add arbitrary number of private streams, hence your encrypted stream (that can be base64 or bytes[] as well).

   From the MP4 registration authority the codecs encm (Encrypted/Protected metadata), encs (Encrypted Systems stream), enct (Encrypted Text) seems to be good candidates for your encrypted private stream.

2. ffmpeg have the ability to mux multiple streams into the final stream; this might be the solution on the backend side. Here there is no need to have the input/private streams finished in order to be able to mux and broadcast into the final stream; the reading should also happen without reaching the end of the stream.

Answer 2

Use a data: URL and directly insert that into the video src property.