Android MediaExtractor: Slow Speed when reading mp3 file

Question

So, I am trying to extract raw PCM data from compressed audio files "in one go", i.e. reading the entire file as fast as possible and outputting the result (no "streaming"). I've got a working async-based implementation using MediaExtractor and MediaCodec, but it's rather slow - it takes ~50 seconds for a 6 MB mp3 file. I've seen other apps do this in <3 seconds (not sure if it's the same API, of course).

Trying to find out what the issue is, I noticed that already simply reading the file using MediaExtractor takes about 6 to 10 seconds without any processing, like so:

MediaExtractor extractor = new MediaExtractor();    
extractor.setDataSource("/path/to/mp3/file");
extractor.selectTrack(0);
ByteBuffer fileInputBuffer= ByteBuffer.allocate(1024*1048);
while (extractor.readSampleData(fileInputBuffer, 0) >= 0) {
  extractor.advance();
}

This code pretty much directly follows the example on the official MediaExtractor web site - but why does it take so long? Is the API only intended for streaming/listening? Any way to speed this up?

The device is an Oculus Quest. Reading in the file itself using plain Java methods takes under a second, so it's not the disk access.

Any ideas? Thanks.