Why is the hls.js library giving deteriorating video quality whilst the native Apple streaming library works fine?

Question

I'm hosting a video website and ran into a problem I can't really explain. I host the videos using cloudflare which provides a hls stream url.

On non-apple devices I use the hls.js library to stream these videos, this is done using the (React) code below:

useEffect(() => {
    let video = document.getElementById(video_id);
    if (video.canPlayType('application/vnd.apple.mpegurl')) {
        video.src = hlsUrl
        return (() => {
          clearWatchTimeout()
          video.src = null
        })
    } else if (Hls.isSupported()) {
      let hls = new Hls({});
      hls.attachMedia(video);
      hls.on(Hls.Events.MEDIA_ATTACHED, function () {
        hls.loadSource(hlsUrl);
        hls.on(Hls.Events.MANIFEST_PARSED, function (event, data) {
          console.log("manifest loaded, found " + data.levels.length + " quality level");
          })
        });
      
      return (() => {
        clearWatchTimeout()
        hls.detachMedia()
        hls.destroy()
      })
    } else {
        console.log("This device is not capable of playing hls streams")
    }
    
  });

The videoplayer works, but somehow the quality drops with the hls.js implementation. This can most easily be noticed on apple devices by playing the video in safari (quality remains ok) compared to playing the video in chrome (quality deteriorates). An example video can which should have crappy quality can be seen following the link below:

https://www.etudor.nl/chapter/1008/1216/1196/Getal-en-Ruimte-vwo-B-deel-4-(11e-editie)-Hoofdstuk-13

I have turned on debugging mode of the hls library but I haven't found anything weird in the logs.

EDIT: Just as an extra tidbit of information: If I manually raise the quality to a higher level (say level 4) the video uses that quality and runs perfectly, so bandwidth really does not seem to be the issue.

EDIT 2: Another bit of information. If I log the bandwidth estimate over time, it seems to decrease every time the quality level has been decreased. If i manually up the quality to level 4, the bandwidth estimate immediately increases.