How can I convert a CMSampleBuffer with image data to a format suitable for sending over a network connection?

Question

I want to send frames of a video stream over a network connection, so I have implemented the AVCaptureVideoDataOutputSampleBufferDelegate function:

func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection)

How should I convert the CMSampleBuffer to Data as the NWConnection function:

func send(content: Data?, contentContext: NWConnection.ContentContext = default, isComplete: Bool = default, completion: NWConnection.SendCompletion)

that I'm using for networking expects Data for its content parameter?

Answer 1

You presumably want to compress the video frames before sending them over the network, because uncompressed video frames might require more bandwidth than you have available. And you'll want to use the hardware compressor for speed.

You can access the hardware compressor and decompressor using the VideoToolbox framework.

You should watch WWDC 2014 session 513, “Direct Access to Video Encoding and Decoding”. Here's a quote from the introduction:

And accompanying that, there's the case where you have a stream of images coming in from the camera or someplace else and you'd like to compress those but get direct access to those compressed sample buffers so that you can send them out over the network or do whatever you like with them.

You can find a transcript of the session at ASCIIwwdc.

Answer 2

Please Try it. It is working at me.

        guard let cvBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
        return
    }
    
    //get a CIImage out of the CVImageBuffer
    let ciImage = CIImage(cvImageBuffer: cvBuffer)
    
    //get UIImage out of CIImage
    let uiImage = UIImage(ciImage: ciImage)

    //get frames which is image data format
    let frames = uiImage.jpegData(compressionQuality: 0.5)!