Get all frames from video using MediaMetadataRetriever

Question

I'm trying to get all frames from a video using MediaMetadataRetriever and getFrameAt() method. Using FFMPEG, I've got the following information :

1. Video frame counts : 234 frames

2. Video duration in milliseconds : 90000

   int counter = 0;
   long mVideoDuration = 9000;
   for (long i = 0; i < mVideoDuration * 1000; i += 1000) {
       Bitmap thumbnail = mMediaMetadataRetriever.getFrameAtTime(i);
   }
   

The above code does not work, it's loop for almost 8944 times, and this is too much, I don't want to get all these frames, I just want to get the 234 frames.

getFrameAtIndex() WORKS VERY WELL, but due to API level compatibility, I would like to make it work with getFrameAt() method

Answer 1

To get the frames with getFrameeAtTime() youd have to know the time of every frame in the video. You could roughly calculate that if u already know the frame count by getting the frametime (duration/frames), taking the math.ceil() method on that and taking that as your incrementor. however, youd have to check if the last frame gets read since u can't always get the exact frame time, bc they tend to have alot of decimal places wich can lead to wrong representation as float. A 60fps video for example would have a frame time of 16+1/3 ms. Wich has infinite deecimal places. In addition this method wouldn't even work at all if you have a video with too high fps since in that case the round up done by math.ceil() can lead to a big enough margin to skip frames. To be precise this method only works if:

(math.ceil(frameTime)-frameTime)⋅frameCount<frameTime

I hope, by now u got that it's a pretty bad idea and u should probably use another way to get the individual frames. But still, heres an implementation:

int lastTime;
long mVideoDuration = 9000;
mVideoDuration*=1000;
int frameTime=math.ceil(mVideoDuration/frameCount)
for(int i =0, i<=mVideoDuration, i+=frameTime){
    Bitmap thumbnail = mMediaMetadataRetriever.getFrameAtTime(i);
    lastTime=i;
}
if (lastTime<mVideoDuration){
   mMediaMetadataRetriever.getFrameAtTime(mVideoDuration);
}

Of couse this is already assuming that,

(math.ceil(frameTime)-frameTime)⋅frameCount<frameTime

is given. And i can't guarantee that the declaframeTime will result the correct value since i dont know exactly how decimal dividing is handleed internally and its entirely possible that is causes problems due to floating point overflow

Answer 2

you can use getFramesAtIndex(int frameIndex, int numFrames) instead of getFrameAt(), it will return list of Bitmaps containing the requested video frames.

This method is similar to getFramesAtIndex(int, int, MediaMetadataRetriever.BitmapParams) except that the default for MediaMetadataRetriever.BitmapParams will be used. Params: frameIndex – 0-based index of the first video frame to retrieve. The frame index must be that of a valid frame. The total number of frames available for retrieval can be queried via the METADATA_KEY_VIDEO_FRAME_COUNT key. numFrames – number of consecutive video frames to retrieve. Must be a positive value. The stream must contain at least numFrames frames starting at frameIndex. Returns: An list of Bitmaps containing the requested video frames. The returned array could contain less frames than requested if the retrieval fails. Throws: IllegalStateException – if the container doesn't contain video or image sequences. IllegalArgumentException – if the frameIndex or numFrames is invalid, or the stream doesn't contain at least numFrames starting at frameIndex. See Also: getFrameAtIndex(int, MediaMetadataRetriever.BitmapParams), getFrameAtIndex(int), getFramesAtIndex(int, int, MediaMetadataRetriever.BitmapParams)

regarding to numFrames, you can get number of frames this way:

m_mediaMetadataRetriever = new MediaMetadataRetriever();
m_mediaMetadataRetriever.setDataSource(getApplicationContext(), your_video_path);
int numFrames = Integer.parseInt(m_mediaMetadataRetriever.extractMetadata(MediaMetadataRetriever.METADATA_KEY_VIDEO_FRAME_COUNT));