Low cost Image to NSData conversion for feeding SCNTechnique's sampler2D inputs

Question

Sometimes the only way to pass precious data from CPU to GPU is by hiding it in textures.

I tried to trick SCNTechnique and simply pass [NSData dataWithBytes:length:] or a CGDataProviderRef containing my neatly prepared raw pixel data bytes, but SceneKit is smart enough to detect my sinister attempts.

But I did not give up, and found a loophole:

  [_sceneView.technique setValue: UIImagePNGRepresentation(encodeInSinglePixelUIImage(pos.x, pos.y)) forKey:@"blob_pos_"];

Encoding and decoding single pixel PNGs at 60fps on a mobile device is something you can afford, on an iPhone X it just costs 2ms and keeps your palm a little bit warmer. However I do not need any heat-generating features till november, so I was wondering if there's a cool alternative to this method.

Answer 1

The most efficient way I found is constructing floating point RGB TIFFs. It's still not super fast, consuming 0.7ms on the iPhone X, but a lot faster than the PNG method.

Having a float texture also have the benefits of direct float transfer, that is, no encoding to multiple uint8 RGBA values on the CPU and reconstructing floats on the GPU.

Here's how:

NSData * tiffencode(float x, float y)
{
    const uint8_t tags = 9;
    const uint8_t headerlen = 8+2+tags*12+4;
    const uint8_t width = 1;
    const uint8_t height = 1;
    const uint8_t datalen = width*height*3*4;
    static uint8_t tiff[headerlen+datalen] = {
        'I', 'I', 0x2a, 0, //little endian/'I'ntel
        8, 0, 0, 0, //index of metadata
        tags, 0,
        0x00, 1,  4, 0,  1, 0, 0, 0,  width, 0, 0, 0,   //width
        0x01, 1,  4, 0,  1, 0, 0, 0,  height, 0, 0, 0,  //height
        0x02, 1,  3, 0,  1, 0, 0, 0,  32, 0, 0, 0,      //bits per sample(s)
        0x06, 1,  3, 0,  1, 0, 0, 0,  2, 0, 0, 0,       //photometric interpretation: RGB
        0x11, 1,  4, 0,  1, 0, 0, 0,  headerlen, 0, 0, 0,//strip offset
        0x15, 1,  3, 0,  1, 0, 0, 0,  3, 0, 0, 0,       //samples per pixel: 3
        0x16, 1,  4, 0,  1, 0, 0, 0,  height, 0, 0, 0,  //rows per strip: height
        0x17, 1,  4, 0,  1, 0, 0, 0,  datalen, 0, 0, 0, //strip byte length
        0x53, 1,  3, 0,  1, 0, 0, 0,  3, 0, 0, 0,       //sampleformat: float
        0, 0, 0, 0, //end of metadata

        //RGBRGB.. pixeldata here
    };

    float *rawData = tiff+headerlen;
    rawData[0] = x;
    rawData[1] = y;

    NSData *data = [NSData dataWithBytes:&tiff length:sizeof(tiff)];
    return data;
}

Useful TIFF links I used:

http://www.fileformat.info/format/tiff/corion.htm

http://paulbourke.net/dataformats/tiff/

https://www.fileformat.info/format/tiff/egff.htm

https://www.awaresystems.be/imaging/tiff/tifftags/sampleformat.html