Incorrect Frames by AVAssetImageGenerator.generateCGImagesAsynchronously

Question

As per the doc, generateCGImagesAsynchronously takes in an array of NSValue and generates the frames from the video and given time and return it as a callback.

I generate a list of values using following code

    var values : [NSValue] = []
    let frameDuration = CMTimeMake(1, timeScale)
    for i in 0..<duration{
        let lastFrameTime = CMTimeMake(Int64(i), timeScale)
        let presentationTime = (i == 0) ? lastFrameTime : CMTimeAdd(lastFrameTime, frameDuration)

        values.append(NSValue(time: presentationTime))

        //Next two lines of codes are just to cross check the output
        let image = try! imageGenerator.copyCGImage(at: presentationTime, actualTime: nil)
        let imageUrl = FileManagerUtil.getTempFileName(parentFolder: FrameExtractor.EXTRACTED_IMAGE, fileNameWithExtension: "\(Constants.FRAME_SUFFIX)\(i)\(".jpeg")")
    }

As you can see in the code above, I cross checked the result by using synchronous method and I can confirm that the values array hold the correct reference of time.

But when same array is passed to generateImageAsynchronously method, I get duplicate of same frame 10 times for different time stamps. That is if my video is of 10 seconds, then I get 300 frames (with 30 fps) but frames of 1st second repeated 10 times each. It something like returning frame for time 0.1 seconds when requested for 1 second.

P.S: Though synchronous method is working fine, it is taking twice the time taken by the asynchronous method. May be because it is returning same frames. But I need it working to check the actual time usages.

Answer 1

Not sure why, but when calling the function from main thread I get the bug. But when I call it from background thread, then it started working.

Not sure about the explanation, but below Swift4 code seems to be working and is much faster when compared to individual frames extraction like let image = imageGenerator.copyCGImage(at: time, actualTime: nil)

DispatchQueue.global(qos: .background).async {
    imageGenerator.generateCGImagesAsynchronously(forTimes: localValue, completionHandler: ({ (startTime, generatedImage, endTime, result, error) in
        //Save image to file or perform any task
    }))
}

Answer 2

You need to set requestedTimeToleranceBefore and requestedTimeToleranceAfter to your desired accuracy, for example .zero but this might incur on extra delay and cost to generate the images.

For example:

imageGenerator.requestedTimeToleranceBefore = .zero
imageGenerator.requestedTimeToleranceAfter = .zero

This way you would get the exact frame at that time.