[FFmpeg-devel] [PATCH] lavf: inspect more frames when the time base is unreliable

[Anssi Hannula]

23.976 fps H.264 Matroska files are usually detected as 24 fps
(r_frame_rate) by av_find_stream_info(). Fix that by inspecting the
timestamps of 30 frames instead of just 20 for video codecs with
unreliable time base.

---

This seems the most straight-forward and safe fix for this issue, with
the unfortunate cost of the 10 additional frames that have to be
demuxed.

Another (or additional) option could be to improve the heuristics
themselves, if someone more experienced with this timing stuff wants to
give it a try (or to provide hints). I guess one such way would be to
fall back to codec time base early if it looks accurate based on the
first few timestamps, but indeed I have no idea if it would break
something else. Input welcome.

For the record, here you can see the progression of the cumulative error
of the two competing rates when detecting the frame rate:

0.042000
for 24.000000, at 0.042000, error 0.000333, cumulative 0.000000111
for 23.976024, at 0.042000, error 0.000292, cumulative 0.000000085
0.041000
for 24.000000, at 0.041000, error -0.000667, cumulative 0.000000444
for 23.976024, at 0.041000, error -0.000708, cumulative 0.000000502
0.083000
for 24.000000, at 0.083000, error -0.000333, cumulative 0.000000556
for 23.976024, at 0.083000, error -0.000417, cumulative 0.000000675
0.125000
for 24.000000, at 0.125000, error 0.000000, cumulative 0.000000556
for 23.976024, at 0.125000, error -0.000125, cumulative 0.000000691
0.167000
for 24.000000, at 0.167000, error 0.000333, cumulative 0.000000667
for 23.976024, at 0.167000, error 0.000167, cumulative 0.000000719
0.208000
for 24.000000, at 0.208000, error -0.000333, cumulative 0.000000778
for 23.976024, at 0.208000, error -0.000542, cumulative 0.000001012
0.250000
for 24.000000, at 0.250000, error 0.000000, cumulative 0.000000778
for 23.976024, at 0.250000, error -0.000250, cumulative 0.000001075
0.292000
for 24.000000, at 0.292000, error 0.000333, cumulative 0.000000889
for 23.976024, at 0.292000, error 0.000042, cumulative 0.000001076
0.333000
for 24.000000, at 0.333000, error -0.000333, cumulative 0.000001000
for 23.976024, at 0.333000, error -0.000667, cumulative 0.000001521
0.375000
for 24.000000, at 0.375000, error 0.000000, cumulative 0.000001000
for 23.976024, at 0.375000, error -0.000375, cumulative 0.000001661
0.417000
for 24.000000, at 0.417000, error 0.000333, cumulative 0.000001111
for 23.976024, at 0.417000, error -0.000083, cumulative 0.000001668
0.459000
for 24.000000, at 0.459000, error 0.000667, cumulative 0.000001556
for 23.976024, at 0.459000, error 0.000208, cumulative 0.000001712
0.500000
for 24.000000, at 0.500000, error 0.000000, cumulative 0.000001556
for 23.976024, at 0.500000, error -0.000500, cumulative 0.000001962
0.542000
for 24.000000, at 0.542000, error 0.000333, cumulative 0.000001667
for 23.976024, at 0.542000, error -0.000208, cumulative 0.000002005
0.584000
for 24.000000, at 0.584000, error 0.000667, cumulative 0.000002111
for 23.976024, at 0.584000, error 0.000083, cumulative 0.000002012
0.625000
for 24.000000, at 0.625000, error 0.000000, cumulative 0.000002111
for 23.976024, at 0.625000, error -0.000625, cumulative 0.000002403
0.667000
for 24.000000, at 0.667000, error 0.000333, cumulative 0.000002222
for 23.976024, at 0.667000, error -0.000333, cumulative 0.000002514
0.709000
for 24.000000, at 0.709000, error 0.000667, cumulative 0.000002667
for 23.976024, at 0.709000, error -0.000042, cumulative 0.000002516
0.750000
for 24.000000, at 0.750000, error 0.000000, cumulative 0.000002667
for 23.976024, at 0.750000, error -0.000750, cumulative 0.000003078
0.792000
for 24.000000, at 0.792000, error 0.000333, cumulative 0.000002778
for 23.976024, at 0.792000, error -0.000458, cumulative 0.000003288

Without the patch the detection stops at this point. Error for 24.000 is
still smaller and it is wrongly selected.

Raising the frame count to 30 allows the cumulative error for 24.000 to
raise significantly when compared to 23.976, therefore ensuring the
correct selection:

0.834000
for 24.000000, at 0.834000, error 0.000667, cumulative 0.000003222
for 23.976024, at 0.834000, error -0.000167, cumulative 0.000003316
0.876000
for 24.000000, at 0.876000, error 0.001000, cumulative 0.000004222
for 23.976024, at 0.876000, error 0.000125, cumulative 0.000003332
0.917000
for 24.000000, at 0.917000, error 0.000333, cumulative 0.000004333
for 23.976024, at 0.917000, error -0.000583, cumulative 0.000003672
0.959000
for 24.000000, at 0.959000, error 0.000667, cumulative 0.000004778
for 23.976024, at 0.959000, error -0.000292, cumulative 0.000003757
1.001000
for 24.000000, at 1.001000, error 0.001000, cumulative 0.000005778
for 23.976024, at 1.001000, error 0.000000, cumulative 0.000003757
1.042000
for 24.000000, at 1.042000, error 0.000333, cumulative 0.000005889
for 23.976024, at 1.042000, error -0.000708, cumulative 0.000004259
1.084000
for 24.000000, at 1.084000, error 0.000667, cumulative 0.000006333
for 23.976024, at 1.084000, error -0.000417, cumulative 0.000004432
1.126000
for 24.000000, at 1.126000, error 0.001000, cumulative 0.000007333
for 23.976024, at 1.126000, error -0.000125, cumulative 0.000004448
1.168000
for 24.000000, at 1.168000, error 0.001333, cumulative 0.000009111
for 23.976024, at 1.168000, error 0.000167, cumulative 0.000004476
1.209000
for 24.000000, at 1.209000, error 0.000667, cumulative 0.000009556
for 23.976024, at 1.209000, error -0.000542, cumulative 0.000004769

 libavformat/utils.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/libavformat/utils.c b/libavformat/utils.c
index 4f51c26..8227c91 100644
--- a/libavformat/utils.c
+++ b/libavformat/utils.c
@@ -2257,7 +2257,7 @@ int av_find_stream_info(AVFormatContext *ic)
                 break;
             /* variable fps and no guess at the real fps */
             if(   tb_unreliable(st->codec) && !(st->r_frame_rate.num && st->avg_frame_rate.num)
-               && st->info->duration_count<20 && st->codec->codec_type == AVMEDIA_TYPE_VIDEO)
+               && st->info->duration_count<30 && st->codec->codec_type == AVMEDIA_TYPE_VIDEO)
                 break;
             if(st->parser && st->parser->parser->split && !st->codec->extradata)
                 break;
-- 
1.7.3