My work would require me to encode a few thousand movies in a few days. Each movie needs to be encoded in 3 different formats. I use ffmpeg to output these formats in parallel with a single read of the input source as detailed here: http://ffmpeg.org/trac/ffmpeg/wiki/Creating%20multiple%20outputs
In addition, I am using GNU Parallel to encode from multiple video files in parallel. We have four blade servers of different configurations (48, 32, 16 and 16 cores) encoding videos in parallel. Ideally, we should be able to encode 112 videos in parallel.
However, it seems that encoding completes faster on machines with lesser cores. I have 16 completed encodes on the 16 core servers in around 4 hours, while it takes close to 10 hours for 48 encodes to complete on the 48 core system. What could be the bottleneck? A typical encode command is as follows:
ffmpeg -i sample.mpg -y -vcodec libx264 -vprofile baseline -level 30 -acodec libfdk_aac -ab 128k -ac 2 -b:v 500K -threads 1 encoded/sample_enc.mp4
Any pointers highly appreciated. Thanks!
1 encoding split over several n core is ok but n encoding in parallel 1 per core would cause a bandwidth saturation ...
try to activate 4 thread for each transcoder to speed the throughput of 1 video and limit to 2 or 3 the number of video you encode in parallel but that depend of the memory you have , of your bandwidth and of the video (SD vs HD for example)