[FFmpeg-user] Stuck on adelay/amix sync problem..

Mark Corner mcorner at gmail.com
Tue Dec 19 16:35:31 EET 2017 

Oh, I think I got it….

I put in a resample=async and it seems like it works!  I need to whittle
the filter down to see which parts are necessary.

ffmpeg -y -i RT2568dcb62e48c44c4ac61094c4377553 -i
RTb5d9c3c0a59e98061159ddb310587b71 -i RT955f8ed764b5fda55a4dcd23cfed673d
 -filter_complex
"[0]aresample=async=10000[0r];[1]aresample=async=10000[1r];[2]aresample=async=10000[2r];[0r]adelay=25|25[t0];[1r]adelay=2363|2363[t1];[2r]adelay=983525|9835
25[t2];[t0]asetpts=N/SR/TB[t0s];[t1]asetpts=N/SR/TB[t1s];[t2]asetpts=N/SR/TB[t2s];[t2s][t1s][t0s]amix=inputs=3"
-strict -2 -c:a opus -strict -2 output_audio.mka

It is ever so slightly out of sync with the video, but it is really, really
close!



On December 19, 2017 at 9:16:49 AM, Mark Corner (mcorner at gmail.com) wrote:

Sorry, forgot to include the mailing list…  See below.


On December 19, 2017 at 9:16:02 AM, Mark Corner (mcorner at gmail.com) wrote:

Ok, I tried this:

ffmpeg -y -i RT2568dcb62e48c44c4ac61094c4377553 -i
RTb5d9c3c0a59e98061159ddb310587b71 -i RT955f8ed764b5fda55a4dcd23cfed673d
 -filter_complex
"[0]adelay=2542|2542[t0];[1]adelay=4880|4880[t1];[2]adelay=986042|986042[t2];[t0]asetpts=N/SR/TB[t0s];[t1]asetpts=N/SR/TB[t1s];[t2]asetpts=N/SR/TB[t2s];[t2s][t1s][t0s]amix=inputs=3"
-strict -2 -c:a opus -strict -2 output_audio.mka

Sadly, that doesn’t work either….  The problem is definitely here in the
combining of audio (at least I know where the problem is) because if I
listen to the conversation I can tell it is out of sync and they are
talking on top of each other (this is a recording from a Twilio video chat).

Any other ideas?  Does it matter that they are different bitrates?

Here is the input data from ffprobe:

Input #0, matroska,webm, from 'RT2568dcb62e48c44c4ac61094c4377553':
  Metadata:
    encoder         : GStreamer matroskamux version 1.8.1.1
    creation_time   : 2017-12-14T20:00:39.000000Z
  Duration: 00:16:19.11, start: 2.542000, bitrate: 38 kb/s
    Stream #0:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
    Metadata:
      title           : Audio

Input #0, matroska,webm, from 'RTb5d9c3c0a59e98061159ddb310587b71':
  Metadata:
    encoder         : GStreamer matroskamux version 1.8.1.1
    creation_time   : 2017-12-14T20:00:41.000000Z
  Duration: 00:32:49.71, start: 4.880000, bitrate: 41 kb/s
    Stream #0:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
    Metadata:
      title           : Audio

Input #0, matroska,webm, from 'RT955f8ed764b5fda55a4dcd23cfed673d':
  Metadata:
    encoder         : GStreamer matroskamux version 1.8.1.1
    creation_time   : 2017-12-14T20:17:02.000000Z
  Duration: 00:16:30.87, start: 986.042000, bitrate: 36 kb/s
    Stream #0:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
    Metadata:
      title           : Audio




On December 19, 2017 at 7:35:38 AM, Paul B Mahol (onemda at gmail.com) wrote:

On 12/19/17, Mark Corner <mcorner at gmail.com> wrote:
> Thanks! I tried this at least:
>
> ffmpeg -y -i RT2568dcb62e48c44c4ac61094c4377553 -i
> RTb5d9c3c0a59e98061159ddb310587b71 -i RT955f8ed764b5fda55a4dcd23cfed673d
> -filter_complex
>
"[0]adelay=25|25[t0];[1]adelay=2363|2363[t1];[2]adelay=983525|983525[t2];[t0]asetpts=PTS-STARTPTS[t0s];[t1]asetpts=PTS-STARTPTS[t1s];[t2]asetpts=PTS-STARTPTS[t2s];[t2s][t1s][t0s]amix=inputs=3"

PTS-STARTPTS will not fix your issue
asetpts=N/SR/TB might.

> -strict -2 -c:a opus -strict -2 output_audio.mka
>
> And that still has the same problem of out of sync audio.
>
> I do get lots of these kinds of warnings:
> [opus @ 0x7f8855019400] Queue input is backward in timespeed=18.1x
>
>
> Just to demonstrate one of my points, these two commands are identical,
> expect the order of streams to amix. They produce results that are 20
> seconds different in length! This happens with asetpts enabled as well.
>
> t0 first:
> ffmpeg -y -i RT2568dcb62e48c44c4ac61094c4377553 -i
> RTb5d9c3c0a59e98061159ddb310587b71 -filter_complex
> "[0]adelay=25|25[t0];[1]adelay=2363|2363[t1];[t0][t1]amix=inputs=2" -c:a
> opus -strict -2 output_audio2.mka
>
> size= 23881kB time=00:33:11.14 bitrate= 98.3kbits/s speed=20.5x
> video:0kB audio:23198kB subtitle:0kB other streams:0kB global headers:0kB
> muxing overhead: 2.946144%
> [opus @ 0x7fb674801800] Average Intensity Stereo band: 0.7
> [opus @ 0x7fb674801800] Dual Stereo used: 0.00%
>
> t1 first:
> ffmpeg -y -i RT2568dcb62e48c44c4ac61094c4377553 -i
> RTb5d9c3c0a59e98061159ddb310587b71 -filter_complex
> "[0]adelay=25|25[t0];[1]adelay=2363|2363[t1];[t1][t0]amix=inputs=2" -c:a
> opus -strict -2 output_audio2.mka
>
> size= 23881kB time=00:32:52.06 bitrate= 99.2kbits/s speed= 20x
> video:0kB audio:23198kB subtitle:0kB other streams:0kB global headers:0kB
> muxing overhead: 2.945853%
> [opus @ 0x7fb403803000] Average Intensity Stereo band: 0.7
> [opus @ 0x7fb403803000] Dual Stereo used: 0.00%
>
>
>
>
> On December 19, 2017 at 3:36:47 AM, Paul B Mahol (onemda at gmail.com) wrote:
>
> On 12/19/17, Mark Corner <mcorner at gmail.com> wrote:
>> I am having trouble with amix producing different results with different
>> input orders and only one input to amix staying in sync with the video.
>>
>> I am starting with three video and three audio files, vp8 and opus
>> encoded. There is a video and audio that go together and are about 32
>> minutes long. Then there are two pairs of audio and video, each about 16
>> minutes long.
>>
>> First I combine the three videos using an overlay:
>> ffmpeg -y -itsoffset 0 -i
>>
>
/tmp/RM359a097bf459c327e34954525862d2b7/RTd9075ac090a32a21e71269e6b08f4053-reproces
>> sed.mkv -itsoffset 2.364 -i
>>
>
/tmp/RM359a097bf459c327e34954525862d2b7/RT21a59c19d91a7644b8c76d3fd12c3b32-reproc
>> essed.mkv -itsoffset 983.503 -i
>>
>
/tmp/RM359a097bf459c327e34954525862d2b7/RT4d34aa2bae28a73f8079a6e1b65b3aa4-re
>> processed.mkv -filter_complex
>> "[0]pad=iw*2:ih[t0];[t0][1]overlay=W/2:0[t1];[t1][2]overlay=0:0[t2]" -map
>> [t2]
>> -an -c:v libvpx -crf 23 -b:v 1M -cpu-used 3 -threads 8
>> /tmp/RM359a097bf459c327e34954525862d2b7/output_video.
>> webm
>>
>> That works fine.
>>
>> Then I combine the audio (this is where the problem is):
>> ffmpeg -y -i
>>
/tmp/RM359a097bf459c327e34954525862d2b7/RT2568dcb62e48c44c4ac61094c4377553
>> -i /tmp/RM359a097bf45
>> 9c327e34954525862d2b7/RTb5d9c3c0a59e98061159ddb310587b71 -i
>> /tmp/RM359a097bf459c327e34954525862d2b7/RT955f8ed
>> 764b5fda55a4dcd23cfed673d -filter_complex
>> "[0]adelay=25|25[t0];[1]adelay=2363|2363[t1];[2]adelay=983525|9835
>> 25[t2];[t2][t1][t0]amix=inputs=3" -strict -2 -c:a libopus
>> /tmp/RM359a097bf459c327e34954525862d2b7/output_audi
>> o.mka
>>
>> And then merge the audio and video together:
>> ffmpeg -y -i /tmp/RM359a097bf459c327e34954525862d2b7/output_audio.mka -i
>> /tmp/RM359a097bf459c327e34954525862d2b7/output_video.webm -c:v copy -c:a
>> copy /tmp/RM359a097bf459c327e34954525862d2b7/full.webm
>>
>> The problem is that the audio and video are only in sync for what ever is
>> first in the amix filter.... So in the above example, the t2 audio input
>> will be in sync, but not the t1 or t0. If I change the order to something
>> like [t1][t2][t0], then the t1 audio is in sync, but none of the others.
>> In fact the length of the output_audio.mka will differ based on the order
>> of inputs to amix.... As far as I understand, the order to amix shouldn't
>> matter at all.
>>
>> Thanks for any advice you can offer.
>
> Perhaps you need to use asetpts filter after each adelay to reset pts
> to sane value?
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>

More information about the ffmpeg-user mailing list