In Swift, how do you remove background from a video?

Question

I'm playing with SwiftUI and have an intro video playing, but wondering how I can remove the background of a video so it will appear transparent

If I replace the AVPlayerItem it does display and plays (with a black background) If I attempt to add the videoComposition (even with no instructions) it doesn't display anything other than a black box.

I'm not sure if my CGSize is correct, or the CMTime is correct.

Here is the code:

import AVKit
import SwiftUI

struct VideoPlayerView: View {
    @State var player = AVPlayer(playerItem: AVPlayerItem(url: Bundle.main.url(forResource: "MyVideo", withExtension: "mov")!))
    
    init() {
        removeBackgroundFromVideo()
    }
    
    func removeBackgroundFromVideo() {
        let asset = AVURLAsset(url: Bundle.main.url(forResource: "MyVideo", withExtension: "mov")!)
        let videoComposition = AVMutableVideoComposition()
        let item = AVPlayerItem(asset: asset)
        let instruction = AVMutableVideoCompositionInstruction()
        instruction.backgroundColor = UIColor.clear.cgColor
        instruction.timeRange = CMTimeRange(start: CMTime(seconds:0,preferredTimescale: CMTimeScale(1)), duration: CMTime(seconds:8,preferredTimescale:CMTimeScale(1)))
        videoComposition.instructions = [instruction]
        videoComposition.renderSize = CGSize(width:1000,height:1000)
        videoComposition.frameDuration = CMTime(seconds: 8, preferredTimescale: 1)
        //item.videoComposition = videoComposition
        player.replaceCurrentItem(with: item)
        //player.currentItem?.videoComposition = videoComposition
    }
    
    var body: some View {
        VStack {
            VideoPlayer(player: player)
                .frame(width: 500,height: 500)
                .onAppear {
                    player.play()
                }
        }
    }
}

Answer 1

In Swift, you can use the AVFoundation framework to work with videos. Removing the background from a video involves a more complex process, often requiring computer vision and machine learning techniques. If you have a pre-trained model for background removal, you can integrate it into your Swift application. to more shorts

import AVFoundation
guard let videoURL = Bundle.main.url(forResource: "your_video", withExtension: "mp4") else {
    return
}

let asset = AVAsset(url: videoURL)
let videoTrack = asset.tracks(withMediaType: .video).first

Create an AVMutableComposition to work with the video.

let composition = AVMutableComposition()
let videoCompositionTrack = composition.addMutableTrack(withMediaType: .video, preferredTrackID: kCMPersistentTrackID_Invalid)

guard let exportURL = // URL for the exported video file else { return }

let exportSession = AVAssetExportSession(asset: composition, presetName: AVAssetExportPresetHighestQuality)
exportSession?.outputURL = exportURL
exportSession?.outputFileType = .mp4

exportSession?.exportAsynchronously {
    if exportSession?.status == .completed {
        // Handle successful export
    } else if let error = exportSession?.error {
        // Handle export error
        print("Export failed with error: \(error)")
    }
}