Real-time video compositing using HD footage, controlled by script

Question

I have been asked to construct a system for showing non-linear artworks on an HD screen, I need to be able to mix multiple streams of video together (ie using alpha maps) in real time and I also need to be able to script the sequence so I can mix and match different clips depending on the situation and have it play this sequence endlessly without the need for user interactions.

I have looked at various VJ software, Flash video, Using FFMPEG to decode the frames and mixing them myself, streaming videos through vlc and many other things but I don't feel like I have found the perfect solution so I am turning to you ladies and gentlemen to suggest any combinations of software/technology I might not have thought of.

I'm sure there is a solution out there somewhere, and perhaps someone here with direct experience of something similar who could advise me on what the solutions and pitfalls they encountered were.

Thanks for your time everyone.

Answer 1

I still have to check this in action but looks MLT (http://www.mltframework.org/twiki/bin/view/MLT/) promising. Aseptically Melt looks interesing: http://www.mltframework.org/twiki/bin/view/MLT/MltMelt.

Answer 2

You mentioned Flash video, but to what degree have you investigated using Flash ActionScript controlling flv videos to achieve your required results? I've been involved in some things like this, and I believe it should be possible even with 1080p videos - so long as you supply hefty enough hardware.