Metal Core Image kernels for Image statistics

Question

I am currently computing histogram, waveform, and vectoroscope using Accelerate and Metal shaders. This code is for 8-bit SDR pixel buffers and it works. I want to rewrite them using Core Image so that they can work for both 10-bit HDR & SDR buffers.

1. Accelerate works very well for histogram for 8-bit images. However it does not support 10bit YCbCr pixel buffers. CoreImage supports it but it is unclear how to grab raw histogram data from the CIAreaHistogram output so as to display it using CoreGraphics or MTKView. The builtin histogram CIHistogramDisplayFilter to display histogram has less flexibility. I also want histogram of luminance as well apart from rgb.

2. I use atomic_fetch_add_explicit in Metal shaders to compute statistics such as Waveform and vectoroscope. It seems Metal core image kernels have no support for atomics. Is there any way to do the same in CoreImage?

Answer 1

For the histogram, I think you have two options:

1. You can write your own histogram visualization kernel, similar to what CIHistogramDisplayFilter does. You could pass it an RGB and a luminance histogram and visualize them to your liking.
2. You calculate the histograms (e.g., with CIAreaHistogram) and render the result into a bitmap buffer. You can then read the bin values from that buffer and visualize them custom UI components, e.g., using SwiftUI. In our CoreImageExtensions library on Github we also have some convenience APIs for reading values from a CIImage that might ease this process.

Concerning the atomics: If you already have a working Metal implementation for those statistics, I would recommend you write a CIImageProcessorKernel around it. It's made for this exact purpose: including custom image processors into a Core Image pipeline. Especially working with Metal is very convenient, as you already get a Metal device, textures, and a command buffer to work with.

---

Some details on CIAreaHistogram:

The output of the CIAreaHistogram filter is an image that is 1 pixel tall and as wide as specified by inputCount. inputCount specifies how many bins you want to have in your histogram. Each pixel in the output contains the percentage of pixels that fall into the corresponding bin, separated by channels.

So for instance, if you set inputCount to 2 (so two bins), and the output image looks like this: [(0.3, 0.6, 0.2, 0.0), (0.7, 0.4, 0.8, 1.0)], that means that 30 % of all red values fall into the first bin and 70 % into the second bin; 60 % of all green values in the first bin, 40 % in the second bin; and so on.

Notice that the values per channel across all bins will add up to 1.0 by default. You can change that, however, using the inputScale parameter, to multiply all values with a factor. It can make sense, especially when you have a large inputCount, to increase the inputScale to not lose precision due to potentially very small values per bin.