[FFmpeg-devel] [PATCH] fftools/ffmpeg: fix video frame lost sync for the dual input ts files with audio

[Paul B Mahol]

On 7/10/19, Limin Wang <lance.lmwang at gmail.com> wrote:
> On Tue, Jul 09, 2019 at 07:49:23PM +0200, Michael Niedermayer wrote:
>> On Mon, Jul 08, 2019 at 06:34:12PM +0800, lance.lmwang at gmail.com wrote:
>> > From: Limin Wang <lance.lmwang at gmail.com>
>> >
>> > How to reproduce the problem(use two ts files with audio stream):
>> > ffmpeg -i left_w_a.ts -i right_w_a.ts -filter_complex "hstack=inputs=2"
>> > -f null -
>> >
>> > With above command, the audio stream of the second input will be
>> > discarded default, however the start_time is
>> > initalized from the audio stream, so the old code will try to correct
>> > the ts offset in the condition which will
>> > cause the video frame out of sync and report some dts error like below:
>> > [null @ 0x7fa78c000c00] Application provided invalid, non monotonically
>> > increasing dts to muxer in stream 0: 43 >= 43
>> > [null @ 0x7fa78c000c00] Application provided invalid, non monotonically
>> > increasing dts to muxer in stream 0: 44 >= 44
>> > [null @ 0x7fa78c000c00] Application provided invalid, non monotonically
>> > increasing dts to muxer in stream 0: 45 >= 45
>> > [null @ 0x7fa78c000c00] Application provided invalid, non monotonically
>> > increasing dts to muxer in stream 0: 46 >= 46
>> > [null @ 0x7fa78c000c00] Application provided invalid, non monotonically
>> > increasing dts to muxer in stream 0: 47 >= 47
>> >
>> > Or you can try with below command to reproduce the issue:
>> > ffmpeg -i input_w_a.ts -i input_w_a.ts -filter_complex "ssim=f=ssim.log"
>> > -f null -
>> >
>> > You'll get below ssim result which isn't expected result for the two
>> > input is same, the SSIM should be 1.00000
>> > SSIM Y:0.777353 (6.523822) U:0.964087 (14.447539) V:0.966739 (14.780661)
>> > All:0.840039 (7.959872)
>> >
>> > With the patch applied, the above testing command can get expected
>> > result.
>> >
>> > other solution are: use the setpts=PTS-STARTPTS filter or skip all audio
>> > stream to avoid the timestamp adjustment.
>> >
>> > Signed-off-by: Limin Wang <lance.lmwang at gmail.com>
>> > ---
>> >  fftools/ffmpeg.c | 5 ++++-
>> >  1 file changed, 4 insertions(+), 1 deletion(-)
>>
>> this breaks fate-mpeg2-ticket186
>>
>
> For the fate-mpeg2-ticket186 case, the new code wasn't triggered the
> timestamp
> adjustment for the audio is skiped and have one stream only. So we'll get
> one
> extra video frame in the beginning than old code, I have checked the next
> 350 frames are bitexact. Can I update the tests/ref/fate/mpeg2-ticket186
> to fix it?
>

Yes, if change is correct, update relevant files.

> below is the extra one frame bit info:
> +0,          0,          0,        1,   152064, 0x5c464d94
>
>
>
>> [...]
>> --
>> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>>
>> When you are offended at any man's fault, turn to yourself and study your
>> own failings. Then you will forget your anger. -- Epictetus
>
>
>
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".