So I've decided to try and make my own client for IIS's Smooth Streaming as a learning exercise and I'm struggling to decode the mp4 fragments.
I have the following fragment: http://www.mediafire.com/download/n32660d920g291c/part+1.mp4
Which when I try to decode using ffmpeg:
C:\>ffplay.exe -f h264 -vcodec h264 -g 52 "part 1.mp4"
ffplay version N-58502-g738ebb4 Copyright (c) 2003-2013 the FFmpeg developers built on Nov 27 2013 22:09:34 with gcc 4.8.2 (GCC)
configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetype --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis--enable-libvpx --enable-libwavpack --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
libavutil 52. 55.100 / 52. 55.100
libavcodec 55. 44.100 / 55. 44.100
libavformat 55. 21.102 / 55. 21.102
libavdevice 55. 5.102 / 55. 5.102
libavfilter 3. 91.100 / 3. 91.100
libswscale 2. 5.101 / 2. 5.101
libswresample 0. 17.104 / 0. 17.104
libpostproc 52. 3.100 / 52. 3.100
[h264 @ 0000000001e58260] non-existing PPS referenced 0B f=0/0
[h264 @ 0000000001e58260] non-existing PPS 25 referenced
[h264 @ 0000000001e58260] decode_slice_header error
[h264 @ 0000000001e58260] Invalid mix of idr and non-idr slices
[h264 @ 0000000001e58260] sps_id out of range
Last message repeated 4 times
[h264 @ 0000000001e58260] non-existing PPS referenced
[h264 @ 0000000001e58260] sps_id out of range
Last message repeated 4 times
[h264 @ 0000000001e58260] slice type too large (24) at 0 0
[h264 @ 0000000001e58260] decode_slice_header error
[h264 @ 0000000001e58260] no frame!
[h264 @ 0000000001e4fa00] decoding for stream 0 failed
[h264 @ 0000000001e4fa00] Could not find codec parameters for stream 0 (Video: h264): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
part 1.mp4: could not find codec parameters
After reading I found the PPS/SPS comes from the CodecPrivateData in the manifest that IIS gives me, which I have as:
000000016764001eacd940a03da10005161501312d000f162d960000000168ebe3cb22c0
Binary: http://www.mediafire.com/download/hm3av6h3i1g7ojf/CodecPrivateData
Which I think is annex b 00000001[SPS]00000001[PPS] but I have no idea what to do with it, I've tried putting it at the beginning of the file but get the same error, I also tried taking just the data in the mdat and sending that to ffmpeg but I get
[h264 @ 0000000001da8a80] sps_id out of range 0KB sq= 0B f=0/0
[h264 @ 0000000001da8a80] non-existing PPS referenced
[h264 @ 0000000001da8a80] sps_id out of range
[h264 @ 0000000001da8a80] slice type too large (24) at 0 0
[h264 @ 0000000001da8a80] decode_slice_header error
[h264 @ 0000000001da8a80] no frame!
[h264 @ 0000000001d9fa00] decoding for stream 0 failed
[h264 @ 0000000001d9fa00] Could not find codec parameters for stream 0 (Video: h264): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
part 1 - Copy.mp4: could not find codec parameters
So I then tried putting the CodecPrivateData just before the mdat section and I then get this
[h264 @ 0000000001d38a80] sps_id out of range 0KB sq= 0B f=0/0
Last message repeated 1 times
[h264 @ 0000000001d38a80] slice type too large (24) at 0 0
[h264 @ 0000000001d38a80] decode_slice_header error
[h264 @ 0000000001d38a80] no frame!
[h264 @ 0000000001d2fa00] decoding for stream 0 failed
[h264 @ 0000000001d2fa00] Could not find codec parameters for stream 0 (Video: h264 (High)): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
part 1 - Copy.mp4: could not find codec parameters
I seem to be missing something to get this to decode and I'm just not sure what so I'm hoping someone knowledgeable with h264 can help me out! Ultimately I want to get this working in DirectShow but even getting it to work in ffmpeg would be a massive help!
mp4 fragments are not decodable on their own. If you wish to convert it to an mp4, you need to recreate the moov atom in its entirety. This includes making a avc1 atom that contains the extradata (sps/pps) in avcc format.