how to display MTKView with rgba16Float MTLPixelFormat

Question

I have an MTKView set to use MTLPixelFormat.rgba16Float. I'm having display issues which can be best described with the following graphic:

So the intended UIColor becomes washed out, but only while it is being displayed in MTKView. When I convert the drawable texture back to an image for display in a UIView via CIIMage, I get back the original color. Here is how I create that output:

let colorSpace = CGColorSpaceCreateDeviceRGB()
let kciOptions = [kCIImageColorSpace: colorSpace,
                  kCIContextOutputPremultiplied: true,
                  kCIContextUseSoftwareRenderer: false] as [String : Any]  
let strokeCIImage = CIImage(mtlTexture: metalTextureComposite, options: kciOptions)!.oriented(CGImagePropertyOrientation.downMirrored)
let imageCropCG = cicontext.createCGImage(strokeCIImage, from: bbox, format: kCIFormatABGR8, colorSpace: colorSpace) 

Other pertinent settings:

uiColorBrushDefault: UIColor = UIColor(red: 0.92, green: 0.79, blue: 0.18, alpha: 1.0)
self.colorPixelFormat = MTLPixelFormat.rgba16Float
renderPipelineDescriptor.colorAttachments[0].pixelFormat = self.colorPixelFormat

// below is the colorspace for the texture which is tinted with UIColor
let colorSpace = CGColorSpaceCreateDeviceRGB()
let texDescriptor = MTLTextureDescriptor.texture2DDescriptor(pixelFormat: MTLPixelFormat.rgba8Unorm, width: Int(width), height: Int(height), mipmapped: isMipmaped)
target = texDescriptor.textureType
texture = device.makeTexture(descriptor: texDescriptor)

Some posts have hinted at sRGB being assumed somewhere, but no specifics as to how I can disable it.

I'd like the color that I display on MTKView to match the input (as close to it as possible anyway) and still be able to convert that texture into something I can display in an ImageView. I've tested this on an iPad Air and a new iPad Pro. Same behavior. Any help would be appreciated.

Answer 1

So, it looks like you are very close to the complete solution. But what you have is not quite correct. Here is a Metal function that will convert from sRGB to a linear value which you can then write in your Metal shader (I still suggest that you write to a sRGB texture but you can also write to a 16 bit texture). Note that sRGB is not a simple 2.2 gamma curve.

// Convert a non-linear log value to a linear value.
// Note that normV must be normalized in the range [0.0 1.0].

static inline
float sRGB_nonLinearNormToLinear(float normV)
{
  if (normV <= 0.04045f) {
    normV *= (1.0f / 12.92f);
  } else {
    const float a = 0.055f;
    const float gamma = 2.4f;
    //const float gamma = 1.0f / (1.0f / 2.4f);
    normV = (normV + a) * (1.0f / (1.0f + a));
    normV = pow(normV, gamma);
  }

  return normV;
}

Answer 2

The key turned out to be in undoing the gamma correction that is inherently embedded in a UIColor

let colorSRGB = UIColor(red: 0.92, green: 0.79, blue: 0.18, alpha: 1.0)
let rgbaSRGB = colorSRGB.getRGBAComponents()!
let gammapower : CGFloat = 2.2

let r = pow(rgbaSRGB.red, gammapower)
let g = pow(rgbaSRGB.green, gammapower)
let b = pow(rgbaSRGB.blue, gammapower)
let a = pow(rgbaSRGB.alpha, gammapower)

let colorNoGamma: UIColor = UIColor(red: CGFloat(r), green: CGFloat(g), blue: CGFloat(b), alpha: CGFloat(a))

Once I pass colorNoGamma to be applied in an MKTView with MTLPixelFormat.rgba16Float, the results will match a UIView displaying colorSRGB. It makes sense once you think through it...Thanks to @MoDJ for leading me in the right path.

Note that, on the MTKView side, I can keep the texture settings as originally defined, namely:

let texDescriptor = MTLTextureDescriptor.texture2DDescriptor(pixelFormat: MTLPixelFormat.rgba8Unorm, width: Int(width), height: Int(height), mipmapped: isMipmaped)
target = texDescriptor.textureType

And, if I want to convert the currentDrawable into a texture to be displayed in a UIImageView, then I want to make sure I don't apply ANY colorspace settings. I.e:

let strokeCIImage = CIImage(mtlTexture: metalTextureComposite, options: [:])!.oriented(CGImagePropertyOrientation.downMirrored)
let imageCropCG = cicontext.createCGImage(strokeCIImage, from: box, format: kCIFormatABGR8, colorSpace: colorSpace)  // kCIFormatRGBA8 gives same result. Not sure what's proper
let layerStroke = CALayer()
layerStroke.frame = bbox
layerStroke.contents = imageCropCG

Note the argument options: [:] means I pass no colorspace/premultiplication/renderer settings as I did in the original post where I defined kciOptions