FFMpeg Convert AVI to MP4 - Audio / Video Out of Sync

Question

I have an application that lets users record video from their webcams. The users record 3 clips, which are then combined into one video for playback.

I am having issues with the audio and video on the third clip sometimes being out of sync after converting to MP4.

Here's how the process works now-

Combine AVI video files and insert transition videos in between using Mencoder to combine avi files.

mencoder.exe -oac copy -ovc copy -idx -o test.avi c:\temp\*.avi 

I get this error:

Muxer frame buffer cannot allocate memory!

1 duplicate frame(s)! 
1 duplicate frame(s)! 
1 duplicate frame(s)!
1 duplicate frame(s)! 
1 duplicate frame(s)! 
1 duplicate frame(s)!

However, the video is combined to avi and this AVI plays back without any audio sync issues.

Reindex using mencoder

mencoder.exe -idx TEST.AVI 

The video still plays fine at this point as an AVI.

Convert the AVI to MP4 using ffmpeg batch file:

set FFMPEG_DATADIR=c:\Presets
ffmpeg.exe -i "test.avi" -s 640x480 -y -strict experimental -acodec aac \
   -ab 128k -ac 2 -ar 48000 -vcodec libx264 -vpre medium -vpre ipod640 -r 24 \
   -g 48 -b 520000 -threads 64 "out.mp4"

This all has to be done programmatically (no video editing GUI software) using command line tools in Windows.

Here are sample avi and mp4 files. The audio / video sync happens around the 1:03 mark. AVI doesn't get out of sync, while the mp4 does.

http://trtemp.s3.amazonaws.com/new.avi

http://trtemp.s3.amazonaws.com/new.mp4

Does anyone have any suggestions to fix this?

Answer 1

A couple of things:

• I don't find the source composite audio sync to be exactly perfect. It would be interesting to also see the original three source clips. A general rule is to always start with the highest quality source possible.
• Can we assume the source three clips also have 48kHz 16-bit stereo audio too? If there is a conversion going on there then maybe this is an issue.
• FFmpeg's internal aac CODEC is experimental, and that is why you have to have that -strict experimental switch. Since you are having audio sync issues, this is an obvious candidate problem area. Consider using libfaac if available. (You very well may have to build your own FFmpeg compile.)
• Using -threads 64 is not going to make it go faster no matter how many cores you have. Follow the generally accepted convention of # cores * 3 / 2. So, if you have 4 cores, the mast is 4 * 3 / 2 = 6. My own experiments with the -threads shows diminishing rates of return with each thread over 4-6 which zeros out (no benefit whatsoever) over 12-16 threads. At best, 64 threads will no have no impact. At worst, it will be slower or degrading the quality of your result.
• Experiment with the -copyts option. "Copy timestamps from input to output."
• Experiment with the -async switch.

Audio sync method. "Stretches/squeezes" the audio stream to match the timestamps, the parameter is the maximum samples per second by which the audio is changed. -async 1 is a special case where only the start of the audio stream is corrected without any later correction.

Answer 2

I had similar audio sync problems with avi or dv --> mp4 with mencoder under linux. Simple solution was to extract audio first, then convert avi to mp4 and as last step mux both tracks together. could be done with a shellscript avi2mp4.sh like this:

#!/bin/bash
# convert avi files to mp4 with correct sync of video and audio.
# usage: avi2mp4 <sourcefile> <targetfile>
audiopart=$2.audiotmp.mp3
videopart=$2.videotmp.mp4
mencoder -of rawaudio -oac lavc -lavcopts acodec=mp2:abitrate=128 -ovc copy -o $audiopart $1 
mencoder  -oac mp3lame -ovc x264 -vf yadif=0:harddup -o $videopart  $1 
mencoder -oac copy -ovc copy -audiofile $audiofile -o $2 $videopart
rm $audiopart
rm $videopart