[Ardour-Dev] ardour3 video timeline

[Robin Gareus]

On 06/28/2010 01:38 AM, Patrick Shirkey wrote:
> On 06/28/2010 09:20 AM, Robin Gareus wrote:
>> Picking up an old subject.. here's what I set out for:
>>
>> Outline:
>>   1) add a video-timeline to ardour3
>>   2) allow to launch xjadeo from ardour3-menu
>>   3) remember state between sessions.
>>
>> Constraints:
>>   - one video-track per session.
>>   - video-file decoding happens outside of ardour (video-server)
>>   - but the ardour session remembers all relevant settings.
>>   - KISS
>>
>> Implementation:
>> 2+3) can be similar to what qjadeo does: All xjadeo settings (incl
>> window position&  attributes) can be remote-controlled. A "video-import"
>> dialog could invoke mencoder or ffmpeg on initial video-open.
>>
>> As for 1):
>> The trickiest part (at least for me) is the ardour3 GUI. For the
>> video-frame-cache/server there is an existing code-base and xjadeo works
>> on all platforms that ardour does.
>>
>> The first idea to implement the image-timeline like it had been in
>> ardour2/CMT proved unsuitable: in ardour3 TimeAxisView and StreamView
>> both rely RouteTimeAxis[View] (and eventually a Route) to display a
>> track on the canvas. The two options (write all required functions for a
>> StreamView without route or make the video-track a special route) seem
>> overkill if not wrong (because it'd require changes to libardour, and a
>> video-timeline is GUI only).
>>
>> A IMHO suitable solution is to use the ruler/marker bar for the
>> video-timeline. A first-shot at coding this looks rather promising:
>>
>> curl
>> "http://rg42.org/gitweb/?p=ardour3.git;a=commitdiff_plain;hp=master;h=videotl"
>>
>> | patch -p1
>> ./waf configure --videotimeline
>>
>> It currently displays "dummy-frames" (black-frames with video-frameno
>> rendered into them) and looks like this:
>> http://rg42.org/_media/wiki/a3vtl1.png
>> http://rg42.org/_media/wiki/a3ctl2.png
>>
>> Please review. Would this concept be eligible to become part of ardour3?
>>
>>    
> 
> Nice work Robin!

Thanks for the encouragement.

> I think it will be very useful for post production work. My only concern
> is that it doesn't necessarily allow extension for the inevitable future
> request for video editing capabilities...

I've been pondering this beforehand. The image-timeline present in
ardour2 does allow editing but it's more a curse than a blessing.
besides ardour3 broke that code completely.

The problem is a more general one: for video-editing you'll want a
different tool-box. "move" and "trim" are probably the only ones in
common, not to mention you want a different GUI layout.

Oh and it can be extended. While the current number of "rulers" is
hardcoded, they can be created dynamically and handle actions :)

> However if it is viable to place the time line in the editor panel at
> some future date when the single track system is stable and fully
> implemented then it seems like a very good solution to get the ball
> rolling.

I dare say that much of the essential code (video-backend and timeline
calc) can be re-used. But it'll be a major effort to implement the GUI.
I actually tried to do that before I came up with the current ruler
solution: I've hacked > 2000 LOC [Time|Stream|Image|Axis|View] classes
last week without even displaying a single video-track before I realized
that it could not be done cleanly without patching libardour.

The current prototype is < 500 LOC and displays something. Adding calls
to a video-decoder and frame-cache is easy.
The session-management and setup & option dialogs - while sounding
trivial - are much more work.

The main reason to keep it simple was to get sth. that is usable for
more than 3 persons on this world (aics + ardour2) soon.

> I understand that there is already Blender and several other very good
> video editing systems but the same defence was used for midi editing and
> now we have ardour3. so it's probably best to plan for the eventual
> possiblity that ardour will become a fully featured video editing system
> too once this step has been integrated and proven stable.

Well, Midi is at least audio-related but Video-editing is whole
different subject. In a professional workflow one does mix the two.

The current algorithm that aligns the video provides for time-offsets
and concatenation of videos in a single track (playlist). That's not
editing but it can in principle _read_ simple EDL.

In fact I see this video-timeline as a handy addition. The main focus is
to improve interoperability with xjadeo and session management.

My support for a FLOSS NLE is on lumiera.

AFAIK lumiera only implements minimal audio features but already has
jack-transport sync on their feature list. IMHO a lumiera session export
to ardour (or even tighter integration) will be the way to go.

> Cheers.

robin