How do I use vImage_Buffer to manage pixels and create an image?

Question

I want to define an image as RGB(A) values in an array, then I want to use vImage_buffer to produce ultimately produce a CG/UI image.

There's a Pixel_8888 data type (an alias for (UInt8, UInt8, UInt8, UInt8)) that seems like a promising data type to use. Is this right?

So far, I start from an array [Pixel_8888] and use .withUnsafeMutableBytes which I create a vImage_Buffer which gives me a striped image.

But I can't then make an image without the striping (which is the showing each RGBA channel).

I think a step to convert from RGBA to planar is required, but I could be wrong.

Answer 1

Apple have just introduced a Swift friendly wrapper around vImage_Buffer called vImage.PixelBuffer that may make your life easier.

The underlying data in a multiple-channel vImage buffer is generally interleaved. That means, for RGB, pixels are stored red, green, blue, red, green, blue, etc.

The following code shows how to use a vImage.PixelBuffer to create a very simple gradient. The code creates a new buffer and then accesses its data to write red, green, and blue pixel values. Finally, the code creates a CGImage:

let buffer = vImage.PixelBuffer<vImage.Interleaved8x4>(
    size: .init(width: 640, height: 480)
)

buffer.withUnsafeMutableBufferPointer { bufferPtr in
    for x in 0 ..< buffer.width  {
        for y in 0 ..< buffer.height {
         
            let rowBytes = (buffer.rowStride * buffer.byteCountPerPixel)
            let index = y*rowBytes + x*buffer.channelCount
            
            let red = Pixel_8(Float(x) / Float(buffer.width) * 255)
            let blue = Pixel_8(Float(y) / Float(buffer.height) * 255)
            
            bufferPtr[index + 0] = red
            bufferPtr[index + 1] = 0 // green
            bufferPtr[index + 2] = blue
            bufferPtr[index + 3] = 0 // alpha
        }
    }
}

let format = vImage_CGImageFormat(bitsPerComponent: 8,
                               bitsPerPixel: 8 * 4,
                               colorSpace: CGColorSpaceCreateDeviceRGB(),
                               bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.noneSkipLast.rawValue))!

let image = buffer.makeCGImage(cgImageFormat: format)

The resulting image looks like:

Here's the same code using the existing API:

let buffer = try! vImage_Buffer(width: 640,
                                height: 480,
                                bitsPerPixel: 8 * 3)

let bufferPtr = buffer.data.assumingMemoryBound(to: Pixel_8.self)

for x in 0 ..< Int(buffer.width) {
    for y in 0 ..< Int(buffer.height) {
        
        let index = y*buffer.rowBytes + x*3
        
        let red = Pixel_8(Float(x) / Float(buffer.width) * 255)
        let blue = Pixel_8(Float(y) / Float(buffer.height) * 255)
        
        bufferPtr[index + 0] = red
        bufferPtr[index + 1] = 0 // green
        bufferPtr[index + 2] = blue
    }
}


let format = vImage_CGImageFormat(bitsPerComponent: 8,
                               bitsPerPixel: 8 * 3,
                               colorSpace: CGColorSpaceCreateDeviceRGB(),
                               bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.none.rawValue))!

let image = try! buffer.createCGImage(format: format)

buffer.free()