Interoperate MTKView (MetalKit) and SceneKit

Question

I am using MetalKit and have a complex rendering pipeline. The results are rendered into a MTKView.

Now I would like to feed the contents of the MTKView to a SCNScene and using a SCNCamera to perform post-process effects like HDR.

How is this possible?

I do not want general directions. I want specific calls if possible.

Answer 1

You should ideally perform your post-process as part of your Metal rendering pipeline. The procedure you are suggesting requires unnecessary resources since you will be rendering a 2D plane in 3D in SceneKit just to apply some HDR effects.

Nevertheless, you can achieve what you want by rendering your Metal pipeline output to a texture and then simply applying it to a plane in SceneKit.

First assign your texture:

plane.materials.first?.diffuse.contents = offscreenTexture

Then override your SceneKit rendering to your Metal rendering loop:

func renderer(_ renderer: SCNSceneRenderer, willRenderScene scene: SCNScene, atTime time: TimeInterval) {
    doRender()
}

Then perform your Metal rendering with the texture as target, and once that is done you render your SceneKit scene:

func doRender() {
    //rendering to a MTLTexture, so the viewport is the size of this texture
    let viewport = CGRect(x: 0, y: 0, width: CGFloat(textureSizeX), height: CGFloat(textureSizeY))

    //write to offscreenTexture, clear the texture before rendering using green, store the result
    let renderPassDescriptor = MTLRenderPassDescriptor()
    renderPassDescriptor.colorAttachments[0].texture = offscreenTexture
    renderPassDescriptor.colorAttachments[0].loadAction = .clear
    renderPassDescriptor.colorAttachments[0].clearColor = MTLClearColorMake(0, 1, 0, 1.0); //green
    renderPassDescriptor.colorAttachments[0].storeAction = .store

    let commandBuffer = commandQueue.makeCommandBuffer()

    // reuse scene1 and the current point of view
    renderer.scene = scene1
    renderer.pointOfView = scnView1.pointOfView
    renderer.render(atTime: 0, viewport: viewport, commandBuffer: commandBuffer, passDescriptor: renderPassDescriptor)

    commandBuffer.commit()
}`

Full example project:

https://github.com/lachlanhurst/SceneKitOffscreenRendering