Why getFrameAtIndex faster than getFrameAtTime()

Question

I'm trying to get all frames (one by one) from a video, right now I'm using this :

    int framesCount = videoInformation.getFramesCount();
    for (int i = 0; i < framesCount; i++) {
         Bitmap currentFrame = mediaMetadataRetriever.getFrameAtIndex(i);

For 5 sec video, it's take 13 seconds to extract all frames (159 frames).

Since getFrameAtIndex(i) requires API 28, I tried to use getFrameAtTime()

    long timeS = videoInformation.getDuration() / videoInformation.getFps();
    int framesCount = videoInformation.getFramesCount();
    for (int i = 0; i < framesCount; i++) {
          Bitmap currentFrame = (mediaMetadataRetriever.getFrameAtTime(i*timeS*1000, MediaMetadataRetriever.OPTION_CLOSEST_SYNC));

For the same 5 sec video, it's take almost 48sec to get all frames.

Answer 1

I cant speak for the native implementations for both, as both of these functions call their respective native functions, however, if you look at the getFrameAtIndex, you can see that it:

Has access to the total framecount by reading the metadata, and retrieves a frame using an index, similar to Arrays.

int frameCount = Integer.parseInt(extractMetadata(MediaMetadataRetriever.METADATA_KEY_VIDEO_FRAME_COUNT));

getFrameAtIndex() calls getFramesAtIndexInternal()

   private @NonNull List<Bitmap> getFramesAtIndexInternal(
            int frameIndex, int numFrames, @Nullable BitmapParams params) {
        if (!"yes".equals(extractMetadata(MediaMetadataRetriever.METADATA_KEY_HAS_VIDEO))) {
            throw new IllegalStateException("Does not contain video or image sequences");
        }
        int frameCount = Integer.parseInt(
                extractMetadata(MediaMetadataRetriever.METADATA_KEY_VIDEO_FRAME_COUNT));
        if (frameIndex < 0 || numFrames < 1
                || frameIndex >= frameCount
                || frameIndex > frameCount - numFrames) {
            throw new IllegalArgumentException("Invalid frameIndex or numFrames: "
                + frameIndex + ", " + numFrames);
        }
        return _getFrameAtIndex(frameIndex, numFrames, params);
    }

while getFrameAtTime is not really index based, but rather seeks/searches until it matches a Frame timestamp with the required time you passed, or the closest one:

public @Nullable Bitmap getFrameAtTime(long timeUs, @Option int option) {
    if (option < OPTION_PREVIOUS_SYNC ||
        option > OPTION_CLOSEST) {
        throw new IllegalArgumentException("Unsupported option: " + option);
    }

    return _getFrameAtTime(timeUs, option, -1 /*dst_width*/, -1 /*dst_height*/, null);
}

https://github.com/aosp-mirror/platform_frameworks_base/blob/master/media/java/android/media/MediaMetadataRetriever.java