MTLTexture from MTLBuffer issue

Question

I have results of compute shader stored in MTLBuffer. Each element of MTLBuffer is a UInt16. I am trying to pass the results to a fragment shader that displays the result by interpreting the elements of MTLBuffer as color intensity between 0 to 255. I create an MTLTexture out of this MTLBuffer using following code and pass the texture to shader. But I think something is not correct in the process as the output is not correct. I am not sure if it is an issue with pixel format and/or format conversions.

    let textureDescriptor = MTLTextureDescriptor()
    textureDescriptor.textureType = .type2D
    textureDescriptor.pixelFormat = .r16Uint
    textureDescriptor.width = bufferWidth
    textureDescriptor.height = 256
    textureDescriptor.usage = [.shaderRead, .shaderWrite]


    let texture = buffer?.makeTexture(descriptor: textureDescriptor, offset: 0, bytesPerRow: bufferWidth*MemoryLayout<UInt16>.stride)

and here is fragment shader code,

   fragment half4 fragmentShaderDisplay (MappedVertex in [[ stage_in ]],
                                              texture2d<ushort, access::sample> lumaTexture  [[ texture(0) ]]
                                              )
{
    constexpr sampler s(t_address::clamp_to_edge, r_address::clamp_to_edge, min_filter::linear, mag_filter::linear);

    float luma = lumaTexture.sample(s, in.textureCoordinate).r/255.0;

    luma = clamp(0.0, luma, 1.0);

   return half4(luma, luma, luma, 1.h);
}

Answer 1

In what sense do you want to interpret the buffer values as between 0 and 255? Their inherent range is 0 to 65535 and the range for a floating-point component would typically be 0.0 to 1.0. Neither of those is 0 to 255.

If you simply divide by 65535.0 instead of 255.0, you'll get what you seem to want, a value between 0.0 and 1.0.

Also, your call to clamp() seems wrong. Given the order of arguments you've written, you're clamping the constant value 0.0 to be between luma and 1.0. I think you want to clamp luma to be between 0.0 and 1.0.

As it happens, the way you've written things will mostly work. If luma is <= 0.0, 0.0 will be between luma and 1.0 and will be returned unmodified. If 0.0 < luma <= 1.0, then 0.0 is below the range, so clamp() will return the lower bound of the range, which is luma. The problem comes if luma > 1.0. In that case, the results of clamp() are undefined, as per the documentation.

Now, values greater than 1.0 shouldn't happen if you're dividing by the proper divisor. Indeed, there's no need to clamp at all.