[Ardour-Users] "Extract LTC from audio and align video" menu option missing from "Transcode/Import Video file" popup

[Chris Caudle]

This is the message where I described the settings I used attempting to
duplicate Robert's problem with timeline alignment:

http://lists.ardour.org/pipermail/ardour-users-ardour.org/2018-April/028750.html

This was with a wav file recorded on Robert's Zoom F8 recorder, which
places a time stamp in the BWF header metadata, and a video file recorded
with a Moto Android phone, which had an audio track sourced from the
timecode output of the Zoom recorder.

A recap of what I did:
...
[imported BWF 8 channel audio file]
I imported the file you created,
180404-T004.WAV, and when I selected import at file timestamp it was moved
to 6:05:27 as expected.
...
[imported video with timecode as first audio track]
[if not clear from this description, I kept the default of
import/transcode video to session]

In the session/openvideo dialog, after selecting the file and the second
dialog opens, I select extract audio, click the checkbox for extract LTC
and align video, and also the checkbox for debug mode.  That should log
the output from ffmpeg so you can see if there are any errors transcoding
the video, or extracting the audio, but I do not see as many messages from
ffmpeg as I expected showing up in the ardour log.

After importing the video, it is lined up about a minute later than the
audio in the timeline.
Checking the Ardour log window, I see these messages:
[WARNING]: Failed to set session-framerate: '29.917' does not have a
corresponding option setting in Ardour.
[WARNING]: Video file's framerate is not equal to Ardour session
timecode's framerate: '29.917' vs '30'
[INFO]: Align video-start to 1055787966 [samples]

I have my session set to 48k, so 10557887966/48000 is around 367 minutes, or
about 6 hours and 6 minutes, so that matches where it is place in the
timeline.
What I don't see is any message about timecode count or any indication of
how that sample count was derived.

I'm also not sure where that 29.917 frame rate came from.

Looking at the video file info with ffmpeg -i I see  this information
about the video stream:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'VID_20180405_060536.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2018-04-05T12:06:20.000000Z
    com.android.version: 7.0
  Duration: 00:00:42.29, start: 0.000000, bitrate: 17200 kb/s
    Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv,
bt709), 1920x1080, 16984 kb/s, SAR 1:1 DAR 16:9, 29.89 fps, 29.92 tbr,
90k tbn, 180k tbc (default)
    Metadata:
      creation_time   : 2018-04-05T12:06:20.000000Z
      handler_name    : VideoHandle

Note the frame rate of 29.89 frames per second.  That does not match a
common video frame rate ( I would expect either 30 frames per second, or
29.979 frames per second, but not 29.89).
Perhaps that is just a round off error because the video is short, but it
also doesn't match the 29.917 value that Ardour printed.

So I went to Session/RemoveVideo, then Session/Properties, and in the
Timecode frames-per-second setting change to 29.97 so that the session is
manually set to 29.97 (you cannot enter arbitrary values, so there is no
way to see what happens if I set the session frame rate to the 29.917 that
the video is detected as having).

I went back to Session/OpenVideo, find your file again, and uncheck the
"Adjust session framerate to match video framerate" selection box.

After that in the Ardour log I still see this:
[WARNING]: Video file's framerate is not equal to Ardour session
timecode's framerate: '29.917' vs '29.97'
[INFO]: Align video-start to 1055787966 [samples]

The sample value used for alignment is still the same, changing the
session frame rate from 30fps to 29.97fps did not change that.

--------------------
<end recap of mail from 8 April>



On Sat, April 21, 2018 2:52 pm, Robin Gareus wrote:
> Did you select "Reference from current location" without transcoding the
> video? and if so, why?

As noted above, when I duplicated the problem that Robert found I kept the
default of "Import/Transcode Video to Session." as well as the option to
extract audio, and extract LTC from audio and align video.

> On 04/20/2018 06:49 PM, robertlazarski wrote:
>> The phone problem with Ardour was indeed because it was a variable rate
>> issue. I was able to fix it with this command. And then Ardour was fine
>> with it.

After Robert's success using Handbrake, I used this command to perform the
equivalent with ffmpeg:
ffmpeg -i VID_20180405_060536.mp4 -r 29.97 VID_20180405_060536_fixerate.mp4

And also confirmed that after transcoding to fixed frame rate Ardour did
indeed import to the correct location on the timeline.

> There should be no need for this.

I agree, but it seems there is a difference between intent and actual
behavior.  Robert found the problem originally, provided example files
from a download site, and I was able to duplicate.  Discussed in detail on
the mailing list from 2 April.

> When you import a video in Ardour, it is transcoded to a proxy video:

What source file has the function(s) which convert from timecode value to
sample number?
That seems like the most likely place to look for why video frame rate
should affect the sample number where the video gets placed on the
timeline.
I have not found the correct location to look in the code yet, but based
on log messages it appears that although the timeline is displayed as
hour/minute/second/frame format, internally Ardour is placing elements
based on audio sample number.

-- 
Chris Caudle