Merge CAShapeLayer into CVPixelBuffer

Question

I'm capturing the output of a playing video using AVPlayerItemVideoOutput.copyPixelBuffer

I'm able to convert the pixel buffer into a CIImage, then render it back into a pixel buffer again, and then an AVAssetWriter writes the buffer stream out to a new movie clip successfully.

The reason I'm converting to CIImage is I want to do some manipulation of each frame. (So far I don't understand how to manipulate pixel buffers directly).

In this case I want to overlay a "scribble" style drawing that the user does with their finger. While the video plays, they can draw over it. I'm capturing this drawing successfully into a CAShapeLayer.

The code below outputs just the overlay CAShapeLayer successfully. When I try to reincorporate the original frame by uncommenting the lines shown, the entire process bogs down drastically and drops from 60fps to an unstable 10fps or so on an iPhone 12. I get stable 60fps in all cases except when I uncomment that code.

What's the best way to incorporate the shape layer into this stream of pixel buffers in 60fps "real time"?

Note: some of this code is not finalized -- setting bounds correctly, etc. However this is not related to my question and I'm aware that has to be done. The rotation/translation are there to orient the shape layer -- this all works for now.

func addShapesToBuffer(buffer: CVPixelBuffer, shapeLayer: CAShapeLayer) -> CVPixelBuffer? {
    let coreImage = CoreImage.CIImage.init(cvImageBuffer: buffer)
    let newBuffer = getBuffer(from: coreImage)
    CVPixelBufferLockBaseAddress(newBuffer!, [])

    let rect = CGRect(origin: CGPoint.zero, size: CGSize(width: 800, height: 390))
    
    shapeLayer.shouldRasterize = true
    shapeLayer.rasterizationScale = UIScreen.main.scale
    shapeLayer.backgroundColor = UIColor.clear.cgColor
    
    let renderer = UIGraphicsImageRenderer(size: rect.size)

    let uiImageDrawing = renderer.image {
        context in

//            let videoImage = UIImage(ciImage: coreImage)

//            videoImage.draw(in: rect)

        let cgContext = context.cgContext
        cgContext.rotate(by: deg2rad(-90))
        cgContext.translateBy(x: -390, y: 0)
        return shapeLayer.render(in: cgContext)
    }

    let ciContext = CIContext()
    let newImage = CIImage(cgImage: uiImageDrawing.cgImage!)
    ciContext.render(_: newImage, to: newBuffer!)
    

    CVPixelBufferUnlockBaseAddress(newBuffer!, [])
    
    return newBuffer

}