MediaMuxer unable to make MP4s that are streamable

Question

I'm editing an MP4 on Android using MediaExtractor to fetch audio and video tracks then creating a new file using MediaMuxer. It works fine. I can play the new MP4 on the phone (and other players) but am unable to stream the file on the web. When I stop the MediaMuxer it generates a log message

"The mp4 file will not be streamable."

I looked at the underlying native code (MPEG4Writer.cpp) and it would appear that the writer is having trouble calculating the needed moov box size. It tries to guess using some heuristic if a bit rate is not supplied as a parameter to the writer. The problem is the MediaMuxer doesn't provider the ability to set MPEG4Writer's parameters. Am I missing something or am I stuck looking a some other means of generating the file (or header)? Thanks.

Answer 1

In MPEG4Writer.cpp:

// The default MIN_MOOV_BOX_SIZE is set to 0.6% x 1MB / 2,
// where 1MB is the common file size limit for MMS application.
// The default MAX _MOOV_BOX_SIZE value is based on about 3
// minute video recording with a bit rate about 3 Mbps, because
// statistics also show that most of the video captured are going
// to be less than 3 minutes.

This is a bad assumption on how MediaMuxer might be used. We are recording a max of 15 seconds of higher res video and MIN_MOOV_BOX_SIZE is way too small. So to make the file streamable I have to rewrite the file to move the moov header before mdat and patch up some offsets. Here is my code. It's not great. Error paths aren't handled correctly and it makes assumptions about the order of the boxes.

public void fastPlay(String srcFile, String dstFile)  {
    RandomAccessFile inFile = null;
    FileOutputStream outFile = null;
    try {
        inFile = new RandomAccessFile(new File(srcFile), "r");
        outFile = new FileOutputStream(new File(dstFile));
        int moovPos = 0;
        int mdatPos = 0;
        int moovSize = 0;
        int mdatSize = 0;
        byte[] boxSizeBuf = new byte[4];
        byte[] pathBuf = new byte[4];
        int boxSize;
        int dataSize;
        int bytesRead;
        int totalBytesRead = 0;
        int bytesWritten = 0;

        // First find the location and size of the moov and mdat boxes
        while (true) {
            try {
                boxSize = inFile.readInt();
                bytesRead = inFile.read(pathBuf);
                if (bytesRead != 4) {
                    Log.e(TAG, "Unexpected bytes read (path) " + bytesRead);
                    break;
                }
                String pathRead = new String(pathBuf, "UTF-8");
                dataSize = boxSize - 8;
                totalBytesRead += 8;
                if (pathRead.equals("moov")) {
                    moovPos = totalBytesRead - 8;
                    moovSize = boxSize;
                } else if (pathRead.equals("mdat")) {
                    mdatPos = totalBytesRead - 8;
                    mdatSize = boxSize;
                }
                totalBytesRead += inFile.skipBytes(dataSize);
            } catch (IOException e) {
                break;
            }
        }

        // Read the moov box into a buffer. This has to be patched up. Ug.
        inFile.seek(moovPos);
        byte[] moovBoxBuf = new byte[moovSize]; // This shouldn't be too big.
        bytesRead = inFile.read(moovBoxBuf);
        if (bytesRead != moovSize) {
            Log.e(TAG, "Couldn't read full moov box");
        }

        // Now locate the stco boxes (chunk offset box) inside the moov box and patch
        // them up. This ain't purdy.
        int pos = 0;
        while (pos < moovBoxBuf.length - 4) {
            if (moovBoxBuf[pos] == 0x73 && moovBoxBuf[pos + 1] == 0x74 &&
                    moovBoxBuf[pos + 2] == 0x63 && moovBoxBuf[pos + 3] == 0x6f) {
                int stcoPos = pos - 4;
                int stcoSize = byteArrayToInt(moovBoxBuf, stcoPos);
                patchStco(moovBoxBuf, stcoSize, stcoPos, moovSize);
            }
            pos++;
        }

        inFile.seek(0);
        byte[] buf = new byte[(int) mdatPos];
        // Write out everything before mdat
        inFile.read(buf);
        outFile.write(buf);

        // Write moov
        outFile.write(moovBoxBuf, 0, moovSize);

        // Write out mdat
        inFile.seek(mdatPos);
        bytesWritten = 0;
        while (bytesWritten < mdatSize) {
            int bytesRemaining = (int) mdatSize - bytesWritten;
            int bytesToRead = buf.length;
            if (bytesRemaining < bytesToRead) bytesToRead = bytesRemaining;
            bytesRead = inFile.read(buf, 0, bytesToRead);
            if (bytesRead > 0) {
                outFile.write(buf, 0, bytesRead);
                bytesWritten += bytesRead;
            } else {
                break;
            }
        }
    } catch (IOException e) {
        Log.e(TAG, e.getMessage());
    } finally {
        try {
            if (outFile != null) outFile.close();
            if (inFile != null) inFile.close();
        } catch (IOException e) {}
    }
}

private void patchStco(byte[] buf, int size, int pos, int moovSize) {
    Log.e(TAG, "stco " + pos + " size " + size);
    // We are inserting the moov box before the mdat box so all of
    // offsets in the stco box need to be increased by the size of the moov box. The stco
    // box is variable in length. 4 byte size, 4 byte path, 4 byte version, 4 byte flags
    // followed by a variable number of chunk offsets. So subtract off 16 from size then
    // divide result by 4 to get the number of chunk offsets to patch up.
    int chunkOffsetCount = (size - 16) / 4;
    int chunkPos = pos + 16;
    for (int i = 0; i < chunkOffsetCount; i++) {
        int chunkOffset = byteArrayToInt(buf, chunkPos);
        int newChunkOffset = chunkOffset + moovSize;
        intToByteArray(newChunkOffset, buf, chunkPos);
        chunkPos += 4;
    }
}

public static int byteArrayToInt(byte[] b, int offset)
{
    return   b[offset + 3] & 0xFF |
            (b[offset + 2] & 0xFF) << 8 |
            (b[offset + 1] & 0xFF) << 16 |
            (b[offset] & 0xFF) << 24;
}

public void intToByteArray(int a, byte[] buf, int offset)
{
    buf[offset] = (byte) ((a >> 24) & 0xFF);
    buf[offset + 1] = (byte) ((a >> 16) & 0xFF);
    buf[offset + 2] = (byte) ((a >> 8) & 0xFF);
    buf[offset + 3] = (byte) (a  & 0xFF);
}

Answer 2

Currently MediaMuxer does not create streamable MP4 files

You can try Intel INDE on https://software.intel.com/en-us/intel-inde and Media Pack for Android which is a part of INDE, tutorials on https://software.intel.com/en-us/articles/intel-inde-media-pack-for-android-tutorials. It has a sample that shows how to use media pack to create and stream files over the network

For example for camera streaming it have sample CameraStreamerActivity.java

public void onCreate(Bundle icicle) {

    capture = new CameraCapture(new AndroidMediaObjectFactory(getApplicationContext()), progressListener);

    parameters = new StreamingParameters();
    parameters.Host = getString(R.string.streaming_server_default_ip);
    parameters.Port = Integer.parseInt(getString(R.string.streaming_server_default_port));
    parameters.ApplicationName = getString(R.string.streaming_server_default_app);
    parameters.StreamName = getString(R.string.streaming_server_default_stream);

    parameters.isToPublishAudio = false;
    parameters.isToPublishVideo = true;
}

public void startStreaming() {
    configureMediaStreamFormat();
    capture.setTargetVideoFormat(videoFormat);
    capture.setTargetAudioFormat(audioFormat);
    capture.setTargetConnection(prepareStreamingParams());
    capture.start();
}

In addition there are simular samples for files streaming or game process capturing and streaming