How can I tile videos/create a video montage?

Question

I have four videos that I would like to tile in a 2x2 fashion to make a new video. Is there a way I can do this easily, preferably free and under Linux? I am willing to program a moderate amount, perhaps in order to interact with some library, but unwilling to write an entire video-processing program myself. You may assume that the input and output videos are in whatever commonly-occurring format is most convenient.

An analogue of the gm montage command (for images) would be fantastic.

Answer 1

I am currently using GStreamer for a similar project (lecture-capture) myself. You are probably looking for the videomixer element. Check out this example: Video 4-way split screen gstreamer pipeline (script is located here).

GStreamer works perfectly fine on Windows too. You may want to check out the GStreamer WinBuilds if you are interested.

Example
Here's a basic script that works for me on Windows (it doesn't have the backslashes because I use the gst_parse_launch call from C code to parse the pipeline description):

  v0. ! queue
      ! decodebin
      ! ffmpegcolorspace
      ! videoscale
      ! video/x-raw-yuv,width=320,height=240
      ! videobox right=-320 bottom=-240
      ! ffmpegcolorspace
      ! vmix.sink_0
  v1. ! queue   
      ! decodebin
      ! ffmpegcolorspace
      ! videoscale
      ! video/x-raw-yuv,width=320,height=240
      ! videobox bottom=-240
      ! ffmpegcolorspace
      ! vmix.sink_1
  v2. ! queue   
      ! decodebin
      ! ffmpegcolorspace
      ! videoscale
      ! video/x-raw-yuv,width=320,height=240
      ! videobox right=-240
      ! ffmpegcolorspace
      ! vmix.sink_2
  v3. ! queue   
      ! decodebin
      ! ffmpegcolorspace
      ! videoscale
      ! video/x-raw-yuv,width=320,height=240
      ! ffmpegcolorspace
      ! vmix.sink_3
  vmix. ! queue 
        ! ffmpegcolorspace
        ! dshowvideosink
  filesrc location="c:/test.mpg" name="v0"
  filesrc location="c:/test.mpg" name="v1"
  filesrc location="c:/test.mpg" name="v2"
  filesrc location="c:/test.mpg" name="v3"
  videomixer name=vmix
             sink_0::xpos=0   sink_0::ypos=0   sink_0::zorder=0
             sink_1::xpos=320 sink_1::ypos=0   sink_1::zorder=1
             sink_2::xpos=0   sink_2::ypos=240 sink_2::zorder=2
             sink_3::xpos=320 sink_3::ypos=240 sink_3::zorder=3

Answer 2

The following ffmpeg command will do exactly what the questioner wanted:

ffmpeg -i input1.mp4 -i input2.mp4 -i input3.mp4 -i input4.mp4 -filter_complex \
'[0:v]pad=iw*2:ih*2:0:0[int2];[int2][1:v]overlay=0:H/2[int3];[int3][2:v]overlay=W/2:0[int4];[int4][3:v]overlay=W/2:H/2[out]' \
-map [out] -c:v libx264 -crf 23 -preset veryfast output.mp4

First, the pad filter doubles the size of the first input video, leaving the original video in the top-left corner. The serial overlay filters then place the other inputs over the the black padding added by the pad filter.

If the videos are of different resolutions, the command will require some modification.

Answer 3

One possible solution would be to describe the layout of your video montage with SMIL, a multimedia markup language. This requires a text editor for writing your SMIL document and a SMIL video player (e.g., Ambulant, Quicktime or Realplayer) for displaying it.

Answer 4

This sounds like the sort of problem that AviSynth was designed to solve.

AviSynth is essentially a scripting language for manipulating video streams. A text file describes that operations you want to apply to one or more input video streams. The text file is handed to the AviSynth engine, which provides a virtual .AVI file that manipulates the source streams one frame at a time as you fetch them.

Combine AviSynth with a separate tool that reads from the virtual .AVI file and writes to a new file to save the changes.