[FFmpeg-user] Stuck on adelay/amix sync problem..

Tue Dec 19 16:35:21 EET 2017

On 12/19/17, Mark Corner <mcorner at gmail.com> wrote:
> Sorry, forgot to include the mailing list...  See below.

Is out of sync from start or happens later?

>
>
> On December 19, 2017 at 9:16:02 AM, Mark Corner (mcorner at gmail.com) wrote:
>
> Ok, I tried this:
>
> ffmpeg -y -i RT2568dcb62e48c44c4ac61094c4377553 -i
> RTb5d9c3c0a59e98061159ddb310587b71 -i RT955f8ed764b5fda55a4dcd23cfed673d
>  -filter_complex
> "[0]adelay=2542|2542[t0];[1]adelay=4880|4880[t1];[2]adelay=986042|986042[t2];[t0]asetpts=N/SR/TB[t0s];[t1]asetpts=N/SR/TB[t1s];[t2]asetpts=N/SR/TB[t2s];[t2s][t1s][t0s]amix=inputs=3"
> -strict -2 -c:a opus -strict -2 output_audio.mka
>
> Sadly, that doesn't work either....  The problem is definitely here in the
> combining of audio (at least I know where the problem is) because if I
> listen to the conversation I can tell it is out of sync and they are
> talking on top of each other (this is a recording from a Twilio video
> chat).
>
> Any other ideas?  Does it matter that they are different bitrates?
>
> Here is the input data from ffprobe:
>
> Input #0, matroska,webm, from 'RT2568dcb62e48c44c4ac61094c4377553':
>   Metadata:
>     encoder         : GStreamer matroskamux version 1.8.1.1
>     creation_time   : 2017-12-14T20:00:39.000000Z
>   Duration: 00:16:19.11, start: 2.542000, bitrate: 38 kb/s
>     Stream #0:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
>     Metadata:
>       title           : Audio
>
> Input #0, matroska,webm, from 'RTb5d9c3c0a59e98061159ddb310587b71':
>   Metadata:
>     encoder         : GStreamer matroskamux version 1.8.1.1
>     creation_time   : 2017-12-14T20:00:41.000000Z
>   Duration: 00:32:49.71, start: 4.880000, bitrate: 41 kb/s
>     Stream #0:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
>     Metadata:
>       title           : Audio
>
> Input #0, matroska,webm, from 'RT955f8ed764b5fda55a4dcd23cfed673d':
>   Metadata:
>     encoder         : GStreamer matroskamux version 1.8.1.1
>     creation_time   : 2017-12-14T20:17:02.000000Z
>   Duration: 00:16:30.87, start: 986.042000, bitrate: 36 kb/s
>     Stream #0:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
>     Metadata:
>       title           : Audio
>
>
>
>
> On December 19, 2017 at 7:35:38 AM, Paul B Mahol (onemda at gmail.com) wrote:
>
> On 12/19/17, Mark Corner <mcorner at gmail.com> wrote:
>> Thanks! I tried this at least:
>>
>> ffmpeg -y -i RT2568dcb62e48c44c4ac61094c4377553 -i
>> RTb5d9c3c0a59e98061159ddb310587b71 -i RT955f8ed764b5fda55a4dcd23cfed673d
>> -filter_complex
>>
> "[0]adelay=25|25[t0];[1]adelay=2363|2363[t1];[2]adelay=983525|983525[t2];[t0]asetpts=PTS-STARTPTS[t0s];[t1]asetpts=PTS-STARTPTS[t1s];[t2]asetpts=PTS-STARTPTS[t2s];[t2s][t1s][t0s]amix=inputs=3"
>
> PTS-STARTPTS will not fix your issue
> asetpts=N/SR/TB might.
>
>> -strict -2 -c:a opus -strict -2 output_audio.mka
>>
>> And that still has the same problem of out of sync audio.
>>
>> I do get lots of these kinds of warnings:
>> [opus @ 0x7f8855019400] Queue input is backward in timespeed=18.1x
>>
>>
>> Just to demonstrate one of my points, these two commands are identical,
>> expect the order of streams to amix. They produce results that are 20
>> seconds different in length! This happens with asetpts enabled as well.
>>
>> t0 first:
>> ffmpeg -y -i RT2568dcb62e48c44c4ac61094c4377553 -i
>> RTb5d9c3c0a59e98061159ddb310587b71 -filter_complex
>> "[0]adelay=25|25[t0];[1]adelay=2363|2363[t1];[t0][t1]amix=inputs=2" -c:a
>> opus -strict -2 output_audio2.mka
>>
>> size= 23881kB time=00:33:11.14 bitrate= 98.3kbits/s speed=20.5x
>> video:0kB audio:23198kB subtitle:0kB other streams:0kB global headers:0kB
>> muxing overhead: 2.946144%
>> [opus @ 0x7fb674801800] Average Intensity Stereo band: 0.7
>> [opus @ 0x7fb674801800] Dual Stereo used: 0.00%
>>
>> t1 first:
>> ffmpeg -y -i RT2568dcb62e48c44c4ac61094c4377553 -i
>> RTb5d9c3c0a59e98061159ddb310587b71 -filter_complex
>> "[0]adelay=25|25[t0];[1]adelay=2363|2363[t1];[t1][t0]amix=inputs=2" -c:a
>> opus -strict -2 output_audio2.mka
>>
>> size= 23881kB time=00:32:52.06 bitrate= 99.2kbits/s speed= 20x
>> video:0kB audio:23198kB subtitle:0kB other streams:0kB global headers:0kB
>> muxing overhead: 2.945853%
>> [opus @ 0x7fb403803000] Average Intensity Stereo band: 0.7
>> [opus @ 0x7fb403803000] Dual Stereo used: 0.00%
>>
>>
>>
>>
>> On December 19, 2017 at 3:36:47 AM, Paul B Mahol (onemda at gmail.com)
>> wrote:
>>
>> On 12/19/17, Mark Corner <mcorner at gmail.com> wrote:
>>> I am having trouble with amix producing different results with different
>>> input orders and only one input to amix staying in sync with the video.
>>>
>>> I am starting with three video and three audio files, vp8 and opus
>>> encoded. There is a video and audio that go together and are about 32
>>> minutes long. Then there are two pairs of audio and video, each about 16
>>> minutes long.
>>>
>>> First I combine the three videos using an overlay:
>>> ffmpeg -y -itsoffset 0 -i
>>>
>>
> /tmp/RM359a097bf459c327e34954525862d2b7/RTd9075ac090a32a21e71269e6b08f4053-reproces
>>> sed.mkv -itsoffset 2.364 -i
>>>
>>
> /tmp/RM359a097bf459c327e34954525862d2b7/RT21a59c19d91a7644b8c76d3fd12c3b32-reproc
>>> essed.mkv -itsoffset 983.503 -i
>>>
>>
> /tmp/RM359a097bf459c327e34954525862d2b7/RT4d34aa2bae28a73f8079a6e1b65b3aa4-re
>>> processed.mkv -filter_complex
>>> "[0]pad=iw*2:ih[t0];[t0][1]overlay=W/2:0[t1];[t1][2]overlay=0:0[t2]"
>>> -map
>>> [t2]
>>> -an -c:v libvpx -crf 23 -b:v 1M -cpu-used 3 -threads 8
>>> /tmp/RM359a097bf459c327e34954525862d2b7/output_video.
>>> webm
>>>
>>> That works fine.
>>>
>>> Then I combine the audio (this is where the problem is):
>>> ffmpeg -y -i
>>>
> /tmp/RM359a097bf459c327e34954525862d2b7/RT2568dcb62e48c44c4ac61094c4377553
>>> -i /tmp/RM359a097bf45
>>> 9c327e34954525862d2b7/RTb5d9c3c0a59e98061159ddb310587b71 -i
>>> /tmp/RM359a097bf459c327e34954525862d2b7/RT955f8ed
>>> 764b5fda55a4dcd23cfed673d -filter_complex
>>> "[0]adelay=25|25[t0];[1]adelay=2363|2363[t1];[2]adelay=983525|9835
>>> 25[t2];[t2][t1][t0]amix=inputs=3" -strict -2 -c:a libopus
>>> /tmp/RM359a097bf459c327e34954525862d2b7/output_audi
>>> o.mka
>>>
>>> And then merge the audio and video together:
>>> ffmpeg -y -i /tmp/RM359a097bf459c327e34954525862d2b7/output_audio.mka -i
>>> /tmp/RM359a097bf459c327e34954525862d2b7/output_video.webm -c:v copy -c:a
>>> copy /tmp/RM359a097bf459c327e34954525862d2b7/full.webm
>>>
>>> The problem is that the audio and video are only in sync for what ever
>>> is
>>> first in the amix filter.... So in the above example, the t2 audio input
>>> will be in sync, but not the t1 or t0. If I change the order to
>>> something
>>> like [t1][t2][t0], then the t1 audio is in sync, but none of the others.
>>> In fact the length of the output_audio.mka will differ based on the
>>> order
>>> of inputs to amix.... As far as I understand, the order to amix
>>> shouldn't
>>> matter at all.
>>>
>>> Thanks for any advice you can offer.
>>
>> Perhaps you need to use asetpts filter after each adelay to reset pts
>> to sane value?
>> _______________________________________________
>> ffmpeg-user mailing list
>> ffmpeg-user at ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>>
>