How to determine size of SCNPlane to cover entire SKScene

45 Views Asked by At

I am developing an iOS application for a "green screen" using ARKit. That is display an image as a background in a ARFaceTrackingConfiguration AR session (similar to the "green screen" functionality available in other apps such as zoom). I want the image to remain visible all the time and expand to the edges of the screen regardless of the movement of the device. I am using ARKit and SceneKit to accomplish this. In summary here is what I am doing:

  • Add an ARSCNView to the view controller
  • Create an SKScene with the same size (expressed in points) as the ARSCNView
  • Create an SKSpriteNode and add it as child to the SKScene
  • Create an SCNPlane of a specific size (expressed in meters)
  • Create an SCNNode and set its geometry to be the the SCNPlane created in the previous step
  • Add the SCNNode to the ARCSNView

Complete code to accomplish this is below:

    @IBOutlet var arSceneView: ARSCNView!
    var virtualBackgroundNode = SCNNode()
    ...

    func addGreenScreenScene() {
        let skScene = createSKScene()
        let plane = createScnPlane()
        
        virtualBackgroundNode.geometry = plane
        
        let material = SCNMaterial()
        material.diffuse.contents = skScene
        virtualBackgroundNode.geometry?.materials = [material]
        
        if let cameraPosition = arSceneView.pointOfView?.scale {
            let position = SCNVector3(0, 0, cameraPosition.z - 13)
            os_log("camera position=\(cameraPosition.x) \(cameraPosition.y) \(cameraPosition.z)")
            os_log("arSceneView frame=\(self.arSceneView.frame.width) \(self.arSceneView.frame.height)")
            virtualBackgroundNode.position = position
        }

        arSceneView.pointOfView?.addChildNode(virtualBackgroundNode)
    }

    func createSKScene() -> SKScene? {
        guard let image = getImage() else {
            return nil
        }
        
        let skScene = SKScene(size: CGSize(width: arSceneView.frame.width,
                                           height: arSceneView.frame.height))
        let texture = SKTexture(image: image)
        let spriteNode = SKSpriteNode(texture:texture)
        spriteNode.position = CGPoint(x: skScene.size.width / 2.0,
                                      y: skScene.size.height / 2.0)
        spriteNode.size = skScene.size
        spriteNode.yScale = -1.0
        skScene.scaleMode = .aspectFit
        skScene.addChild(spriteNode)
        
        return skScene
    }
    
    func createScnPlane() -> SCNPlane {
        // TODO: Determine the correct size for this
        // such that it covers the entire frame of
        // arSceneView.
        return SCNPlane(width: 12, height: 18)
    }
    
    func getImage() -> UIImage? {
        return UIImage(named: photoInfo.imageName)
    }

The problem I am having is how to calculate the correct size of the SCNPlane (in method createScnPlane) so that it spans across the entire frame of the ARSCNView.

Disclaimer: I am new to AR development. Is there a different way to develop "green screen" functionality?

0

There are 0 best solutions below