MLKit iOS Face Detection Not Working with CoreImage object

142 Views Asked by At

G'day, I've been trying to use MLKit for iOS for face detection task. I referred their code samples through codelab and sample app. So right out of the box, the MLKit seems to be working alright. I however ran into a problem where if the image comes as CoreImage object, the resulting VisionImage will not work with MLKit (i.e. not detecting any faces in it, no crashes or errors presented). Here is an example code.


/// Processes an Image and detects faces in it
/// - Parameter coreImage: Image Taken Out of Core Image Pipeline
func detectFaces(coreImage: CIImage) {
    let ciImage = coreImage // image taken out of a core image pipeline
    guard let cgImage = getCGImageFromCIImage(ciImage) else { return }
    
    let uiImage = UIImage(cgImage: cgImage)
    let visionImage = VisionImage(image: uiImage)
    faceDetector.process(visionImage) { result, error in
        if let error = error {
            print(error.localizedDescription)
            return
        }
        
        guard let result = result else {
            print("no features")
            return
        }
        
        print(result.count) // <<<< ALWAYS RESULTS IN 0
        
        // Other relevant processing later
    }
}

private func getCGImageFromCIImage(_ ciImage: CIImage) -> CGImage? {
    let context = CIContext()
    if let cgImage = context.createCGImage(ciImage.clampedToExtent(), from: ciImage.extent) {
        return cgImage
    }
    return nil
}

private lazy var faceDetectorOption: FaceDetectorOptions = {
    let option = FaceDetectorOptions()
    option.performanceMode = .accurate
    option.isTrackingEnabled = false
    return option
}()

private lazy var faceDetector = FaceDetector.faceDetector(options: faceDetectorOption)

The code above does not crash or produce any error. It simply outputs 0 results. However if I bypass the CoreImage pipeline's work and pass in the original UIImage MLKit outputs the desired result. The core image pipeline is required for the work I'm doing as it's doing a crop and rotate based on the user's input from UI.

Any ideas for what might be the problem is really appreciated.

I tried passing the core image through CoreML's CIDetector and it does detect faces without an issue. (As per project's requirement's I need to process the image through MLKit, not CoreML)

0

There are 0 best solutions below