Android: playing many video simultaneously

Question

I am developing a chat and we have high quality emoticons with extension mp4 (file size of about 300kb). GIF format is not used because of the poor quality and limited colors (256).
I need to display the files in the ListView as cyclic video. Now I'm trying to do this using TextureView and MediaCodec classes. Sources can be found at https://github.com/google/grafika.
The problem is that when you try to play more than 4 video simultaneously, an error occurs IllegalStateException at android.media.MediaCodec.dequeueOutputBuffer. I think this happens because of the large memory consumption, on my device (HTC ONE M7) while playing four videos, the processor is loaded more than 60% !
How can I solve this problem? Maybe I need to use third party codecs?
Or the idea of using video to display smileys is bad and I need to give it up and use something like GIF ?

Answer 1

There is a limit on the number of simultaneous decoders, if for no other reason than at some point you'll exceed the maximum bandwidth of the hardware. On some devices I've seen it switch to software decoding after two hardware decoders are configured. AFAIK there's no enforced behavior here.

One possible solution to your problem is to have a single multiplexed video, where you have all of your emoticons in a single .mp4 file. Play that into a SurfaceTexture, which is then used as a "sprite sheet". This approach requires that all animations have roughly the same number of frames, so you may have to adjust some or just pad out the sequence.

Update: according to this link, the 'M' release is scheduled to add MediaCodecInfo.CodecCapabilities.getMaxSupportedInstances(), which provides "a hint for the max number of the supported concurrent codec instances." Doesn't really help with your issue, but at least it'd give you a number. Hopefully the API will take the video resolution(s) into account.