[FFmpeg-user] Syncing clips based on audio waveform?

[Bouke (VideoToolShed)]

----- Original Message ----- 
From: "Arthur Wait" <artwait at gmail.com>
To: <ffmpeg-user at ffmpeg.org>
Sent: Friday, March 01, 2013 5:38 PM
Subject: [FFmpeg-user] Syncing clips based on audio waveform?


> Hi all,
>
> This is coming from out of left field I imagine, but I'm wondering if
> anyone knows of a filter (or chain of filters) which would allow me to
> align/sync two video clips based on matches in their audio waveforms.

I'm guessing you're after field recording syncing...
it is done in a commercial application called Plural Eyes.
The original developer has sold it, and now it's owned (and improved / 
maintained) by Red Giant IIRC.
With improved, i mean it sucks less, but it still is dog slow, and has only 
success when both sources are very much alike.
For double system shooting it is absolutely unreliable (It is now advertised 
as a 'Fix it in post' product.)

>From my (limited) point of view, waveform is not the best way, as noise will 
drive you crazy.
Finding peaks would be better, but you'll find peaks in one source that is 
not in the other.
(Like tapping on mics, other close sounds.)
Then, there is no guarantee both sources run at exactly the same speed.
A drift of some 20 Msecs over a long take is not uncommon.
Not really a problem for post, but you need to compensate in your filter.
Then there is the trouble of not knowing for sure what takes belong 
together.
(Of course no problem in a lab, but for stuff coming out of the field, 
Murphy has a strong case.)

hth,
Bouke (who is a timecode freak)

> I'm shooting events with multiple cameras and would like to either trim 
> the
> clips to a precise, matching starting frame or generate a metadata 
> "offset"
> marker indicating how far off one clip is from the other in time. Then I
> can trim the clips to match.
>
> Is this possible in ffmpeg (or anywhere else for that matter!)?
>
> Thanks very much!
>
> Art
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
>
>