I am trying to understand how DRM systems work so my journey begins by trying to play a cenc encrypted mp4 video using the Clear Key DRM system without using any library like dash.js or Shaka Player.
The first problem I encountered is that I do not always receive the "encrypted" event. I receive the "encrypted" only on Safari, but not on Google Chrome nor on Firefox.
Interestingly I do receive the "waitingforkey" on Google Chrome and Safari, but not on Firefox.
This fact confuses me the most, because if Google Chrome knows it needs a key, I assume it has to know that the media is encrypted, so why does it not fire the "encrypted" event?
Below you can find the code I use. I am using some convenience functions. I hope it is clear what they do. If not you see their definitions here. Also my example is online here for you to test and debug right in the browser.
async function playClearkeyVideoFromUrls(videoElement, initUrl, urls) {
// for debugging purposes
videoElement.addEventListener(`waitingforkey`, () => console.log(`Event: waitingforkey`))
videoElement.addEventListener(`encrypted`, () => console.log(`Event: encrypted`))
videoElement.addEventListener(`error`, function () {
console.log(`Event: HTMLMediaElement.onerror`)
console.log(this.error)
})
// we create a MediaSource
const mediaSource = new MediaSource()
// attach the MediaSource to the Video tag, only then it will fire the "sourceopen" event
videoElement.src = URL.createObjectURL(mediaSource)
// add a SourceBuffer to the MediaSource, we need to specify the MIME type of the video we want to play
const sourceBuffer = await mediaSource.asyncAddSourceBuffer(`video/mp4;codecs="avc1.64001f"`)
// for debugging purposes
sourceBuffer.addEventListener(`error`, e => {
console.log(`Event: SourceBuffer.onerror`);
console.log(e)
})
// append the first (init) segment
console.log(`Appending the first (init) segment`)
await sourceBuffer.asyncAppendBuffer(await fetchArrayBuffer(initUrl), videoElement)
// here I expect the "encrypted" AND "waitingforkey" event to fire
// now append the rest of the segments
for (let i = 0; i < urls.length; i++) {
const url = urls[i]
console.log(`Appending a segment ...`)
if (!await sourceBuffer.asyncAppendBuffer(await fetchArrayBuffer(url), videoElement)) {
console.log(`Canceling playback as an error has occurred.`)
console.log(videoElement.error)
break
}
}
}
The cenc encrpyted mp4 files I have are from the dash.js examples page, so I assume that this is not the root of my problem.
To sum up my main question is: Why is the "encrypted" event not fired or is my assumption wrong that it should be fired?
I also thought that my fancy util functions could be the cause of the problem. Sadly this is not the case. You can check out my version without the utils file here. It behaves just like the other version.
let initUrl
let urls
let segmentIndex = 0
Number.prototype.toStringPadded = function(size) {
let thisString = this.toString();
while (thisString.length < size) thisString = "0" + thisString;
return thisString;
}
async function fetchArrayBuffer(url) {
return await (await (await fetch(url)).blob()).arrayBuffer()
}
async function updateend() {
console.log(`Event: updateend`)
this.appendBuffer(await fetchArrayBuffer(urls[segmentIndex]))
segmentIndex++
if (segmentIndex === urls.length) {
this.removeEventListener(`updateend`, updateend)
}
console.log(`Appended segment with id ${segmentIndex}.`)
}
async function sourceopen() {
console.log(`Event: sourceopen`)
// add a SourceBuffer to the MediaSource, we need to specify the MIME type of the video we want to play
const sourceBuffer = this.addSourceBuffer(`video/mp4;codecs="avc1.64001f"`)
// for debugging purposes
sourceBuffer.addEventListener(`error`, e => {
console.log(`Event: SourceBuffer.onerror`);
console.log(e)
})
sourceBuffer.addEventListener(`updateend`, updateend)
sourceBuffer.appendBuffer(await fetchArrayBuffer(initUrl))
}
async function playClearkeyVideoFromUrls(videoElement) {
// for debugging purposes
videoElement.addEventListener(`waitingforkey`, (event) => {
console.log(`Event: waitingforkey`)
console.log(event)
})
videoElement.addEventListener(`encrypted`, (mediaEncryptedEvent) => {
console.log(`Event: encrypted`)
console.log(mediaEncryptedEvent)
})
videoElement.addEventListener(`error`, function () {
console.log(`Event: HTMLMediaElement.onerror`)
console.log(this.error)
})
// we create a MediaSource
const mediaSource = new MediaSource()
mediaSource.addEventListener(`sourceopen`, sourceopen)
// attach the MediaSource to the Video tag, only then it will fire the "sourceopen" event
videoElement.src = URL.createObjectURL(mediaSource)
}
async function testPlayClearkeyVideoFromUrls() {
// video urls are from here https://reference.dashif.org/dash.js/nightly/samples/drm/clearkey.html
// and here https://media.axprod.net/TestVectors/v7-MultiDRM-SingleKey/Manifest_ClearKey.mpd
const streamId = 1
initUrl = `https://media.axprod.net/TestVectors/v7-MultiDRM-SingleKey/${streamId}/init.mp4`
const videoUrlPrefix = `https://media.axprod.net/TestVectors/v7-MultiDRM-SingleKey/${streamId}/`
const videoUrlSuffix = `.m4s`
const numberOfSegments = 4
// first we generate our urls we will download
urls = []
for (let i = 0; i < numberOfSegments; i++) {
const url = `${videoUrlPrefix}${(i + 1).toStringPadded(4)}${videoUrlSuffix}`
urls.push(url)
}
const videoElement = document.querySelector(`video`)
await playClearkeyVideoFromUrls(videoElement)
}
testPlayClearkeyVideoFromUrls()
I think the issue you are seeing is that some of these events are 'Experimental' status and so support may not be consistent - e.g.:
From experience the browser compatibility changes quite frequently so you may have to experiment as you have done.
To add a bit more complexity there are actually multiple places that a stream or a container may indicate that a track or even a section of a track is encrypted.
Different players in the past, in particular, have given different results depending on where they looked.
For example, one common player used to only look at the manifest for encryption information and if it saw nothing there it would assume it was unencrypted even if the media stream itself had 'atoms' in the mp4 indicating that it was encrypted - this caused the playback to fail.
More specifically looking at the events fired when playing back encrypted media in a browser that supports Encrypted Media Extensions. From the standard, the high level flow is as in the following diagram:
It can be seen that reporting the encrypted event to the application is optional. The 'waiting for key' event is not shown unfortunately, but it is included in the detail of the EME spec.
As you saw, and from a quick check I did here also based on DASH.js samples, the encrypted event is not fired on Chrome but is on Safari and the waiting for key event is fired on both. On chrome you can look in much more detail at the events and messaging, if you want, with a Chrome extension to view EME messaging - again this will show the encrypted event does not appear to be sent to the app.
Unfortunately, browser implementaions are different in some of these details and if you look through the open source players encryption handling code you will see this reflected - for example for videojs:
(https://github.com/videojs/videojs-contrib-eme#initializemediakeys)
The reason the player can do this before it sees one of the several indications that may be present in the media stream flagging that the content is encrypted, is because it can also read information in the manifest file, which it reads before loading the media streams, which indicates the encryption scheme(s) that are being used for the media streams.
It's worth noting also the CDMs, the element that does the actual decryption and (optionally) the output to the display are proprietary per browser or DRM usually also.
Going back to your original goal of understanding how EME and encryption works, I think your approach is good. You will probably see other browser and CDM differences also. The solutions do evolve quite quickly sometimes, especially in response to a particular attack or vulnerability, so you do need to be aware of this. Again, the open source player issue and discussion lists are a great resource for understanding the latests changes, as well as the history.