[FFmpeg-devel] [PATCH] fftools/ffmpeg: fix video frame lost sync for the dual input ts files with audio

[Limin Wang]

On Tue, Jul 09, 2019 at 07:49:23PM +0200, Michael Niedermayer wrote:
> On Mon, Jul 08, 2019 at 06:34:12PM +0800, lance.lmwang at gmail.com wrote:
> > From: Limin Wang <lance.lmwang at gmail.com>
> > 
> > How to reproduce the problem(use two ts files with audio stream):
> > ffmpeg -i left_w_a.ts -i right_w_a.ts -filter_complex "hstack=inputs=2" -f null -
> > 
> > With above command, the audio stream of the second input will be discarded default, however the start_time is 
> > initalized from the audio stream, so the old code will try to correct the ts offset in the condition which will
> > cause the video frame out of sync and report some dts error like below:
> > [null @ 0x7fa78c000c00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 43 >= 43
> > [null @ 0x7fa78c000c00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 44 >= 44
> > [null @ 0x7fa78c000c00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 45 >= 45
> > [null @ 0x7fa78c000c00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 46 >= 46
> > [null @ 0x7fa78c000c00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 47 >= 47
> > 
> > Or you can try with below command to reproduce the issue:
> > ffmpeg -i input_w_a.ts -i input_w_a.ts -filter_complex "ssim=f=ssim.log" -f null -
> > 
> > You'll get below ssim result which isn't expected result for the two input is same, the SSIM should be 1.00000
> > SSIM Y:0.777353 (6.523822) U:0.964087 (14.447539) V:0.966739 (14.780661) All:0.840039 (7.959872)
> > 
> > With the patch applied, the above testing command can get expected result.
> > 
> > other solution are: use the setpts=PTS-STARTPTS filter or skip all audio stream to avoid the timestamp adjustment.
> > 
> > Signed-off-by: Limin Wang <lance.lmwang at gmail.com>
> > ---
> >  fftools/ffmpeg.c | 5 ++++-
> >  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> this breaks fate-mpeg2-ticket186
> 

For the fate-mpeg2-ticket186 case, the new code wasn't triggered the timestamp
adjustment for the audio is skiped and have one stream only. So we'll get one 
extra video frame in the beginning than old code, I have checked the next
350 frames are bitexact. Can I update the tests/ref/fate/mpeg2-ticket186
to fix it?

below is the extra one frame bit info: 
+0,          0,          0,        1,   152064, 0x5c464d94 



> [...]
> -- 
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> 
> When you are offended at any man's fault, turn to yourself and study your
> own failings. Then you will forget your anger. -- Epictetus



> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".