Best logical formula to determine perceptual / "experienced" quality of a video, given resolution / fps and bitrate?

Question

I am looking for a formula that can provide me with a relatively decent approximation of a Video's playback quality that can be calculated based off of four metrics: width, height, fps, and bitrate (bits/sec). Alternatively, I can also use FFMPEG or similar tools to calculate a Video's playback quality, if any of those tools provide something like what I am looking for here.

An example of what a Video might look like in my problem is as follows:

interface Video {
  /** The width of the Video (in pixels). */
  width: number
  /** The height of the Video (in pixels). */
  height: number
  /** The frame rate of the Video (frames per second). */
  fps: number
  /** The bitrate of the video, in bits per second (e.g. 5_000_000 = 5Mbit/sec) */
  bitrate: number
}

I came up with the following function to compute the average amount of bits available for any given pixel per second:

const computeVideoQualityScalar = (video: Video): number => {
  // The amount of pixels pushed to the display, per frame.
  const pixelsPerFrame = video.width * video.height
  
  // The amount of pixels pushed to the display, per second.
  const pixelsPerSecond = pixelsPerFrame * video.fps
  
  // The average amount of bits used by each pixel, each second,
  // to convey all data relevant to that pixel (e.g. color data, etc)
  const bitsPerPixelPerSecond = video.bitrate / pixelsPerSecond
  
  return bitsPerPixelPerSecond
}

While my formula does do a good job of providing a more-or-less "standardized" assessment of mathematical quality for any given video, it falls short when I try to use it to compare videos of different resolutions to one another. For example, a 1080p60fps video with a bitrate of 10Mbit/sec has a greater visual fidelity (at least, subjectively speaking, to my eyes) than a 720p30fps video with a bitrate of 9Mbit/sec, but my formula would score the 720p30fps video significantly higher than the 1080p60fps video because the 720p video has more bits available per pixel per second than the 1080p video.

I am struggling to come up with ideas as to how to either come up with a different way to calculate the "subjective video quality" for a given video, or extend upon my existing idea here.

Answer 1

Video quality perception is quite a specialised subject area and I think you will find it difficult to come up with as simple a model as it looks like you are targeting.

Many factors including the encoding scheme, the type of content, colour scheme etc can impact the quality and it can vary scene by scene or even frame by frame.

There is a lot of existing information in the domain and emerging AI/ML techniques may provide better, if more complex, models to improve predictive accuracy.

Even when you can analyse the video frame by frame, either comparing to an original known quality reference video or with no reference, the best mathematical models are usually not as good as subjective measures, in particular MOS (https://en.wikipedia.org/wiki/Mean_opinion_score).

I suspect you will find most techniques are moving towards AI/ML at this point. Many encoders are also using AI/ML models to determine how to encode different scenes or section in content also.

I think your best place to start might be to look at some up to date research papers on this subject as it is fast evolving - e.g. https://www.mdpi.com/2079-9292/10/22/2851