[FFmpeg-devel] How to correctly handle Matroska timestamps

[Andreas Rheinhardt]

MattKC:
> ==Summary==
> 
> I've been implementing ffmpeg/libav into an application, and have run
> into timing-related issues when playing back Matroska/MKV videos created
> by my application in VLC. I tried asking about this on the libav-user
> mailing list and received no response so I thought I'd try here. While
> it's true that there may be issues with VLC's implementation too, I've
> noticed specifically that videos generated by FFmpeg do not cause these
> issues, while my application and the examples in "doc/examples" (which
> my code is modeled from) do, which makes me think this code is missing
> something crucial.
> 
> ==Symptoms==
> 
> During playback, VLC's playback time will appear to stay frozen at 0:00
> while the video and audio plays normally. It will then jump only in
> increments of a few seconds (0:03, then 0:07, then 0:11, etc.) This
> behavior occurs with videos produced both by my application and by the
> FFmpeg examples programs "transcoding.c" and "remuxing.c".
> 
> The fact that this occurs with "remuxing.c" is particularly interesting
> since that would seem to imply it's an issue with the format/container
> writing specifically. If "remuxing.c" is given a correct MKV file (i.e.
> one that does not exhibit this issue) produced by FFmpeg as its input,
> and outputs to a second MKV file, that second MKV will demonstrate this
> issue even though the first one didn't, indicating that something is
> breaking during the remuxing process despite no decoding/encoding
> occurring.
> 
> The worst demonstration of this issue that I've run into is if my
> application writes any audio packets before writing the first video
> packet (with "av_interleaved_write_frame"), the first few seconds of
> video (until that first 0:03) will be considered completely blank by
> VLC. The fact that it will pick up after 0:03, the same time the timer
> will make its first update on other MKV files, leads me to believe it's
> the exact same root cause, just a different presentation of it.
> 
> This issue only occurs on MKV files. MP4/MOV appear to have no issues
> whatsoever.
> 
> ==Conclusion==
> 
> Considering FFmpeg produces correct MKV files, I don't believe this to
> be a bug in libav (though it could be considered a bug in the examples).
> I'm assuming the examples and my program are missing something or doing
> something incorrectly.
> 
> My code can be seen here, however the examples will almost certainly be
> easier to read and test:
> https://github.com/olive-editor/olive/blob/master/app/codec/ffmpeg/ffmpegencoder.cpp
> 
> 
> Thanks in advance for any help, I'm a little confused about this whole
> issue.
> 
If it happens with our examples, too (as you said), then you should open
an issue on trac (for the examples). Be sure to add samples that allow
to reproduce it.
It would probably be enlightening to use mkvinfo to look at the
generated files. The files are probably badly interleaved (your
description makes me suspect that the generated files have lots of audio
packets at the beginning before the first video packet).
(A quick look showed that the transcoding example does not use
pkt_timebase for decoding, as does your code; furthermore, your code
sets the duration of subtitle packets wrong, as the end_display_time is
in a different timebase than the subtitle's pts.)

- Andreas