[Libav-user] Encoding with variable frame rate

[Robert Krüger]

Hi,

On Mon, May 20, 2013 at 5:33 PM, Brad O'Hearne
<brado at bighillsoftware.com> wrote:
>
> On May 20, 2013, at 6:01 AM, Robert Krüger <krueger at lesspain.de> wrote:
>
>> On Mon, May 20, 2013 at 2:06 PM, Carl Eugen Hoyos <cehoyos at ag.or.at> wrote:
>>> Robert Krüger <krueger at ...> writes:
>>>
>>>> On Mon, May 20, 2013 at 12:11 PM, Carl Eugen Hoyos wrote:
>>>>> FFmpeg's mov muxer does not support vfr.
>
>>>> What happens when one feeds the muxer with packets
>>>> that have timestamps with differing offsets?
>
> Robert,
>
> For what its worth, this is essentially the problem I recently had to tackle. You may have seen some recent discussions about pts/dts and the nature of time-base. While there are some things that can be done if you control a capture mechanism, if you don't control that mechanism (i.e. receiving capture data from a third-party API), in general, there is no such thing as a fixed frame rate.
>
> In my use-case, I am using QTKit as the capture mechanism, which while you can set the desired frame-rate, doesn't deliver that specified FPS per se. It delivers pts timestamp, dts timestamp, a timescale, and a duration. That is sufficient data and logically all that is needed to properly encode the captured video. But if you specify that you want 30 FPS, and set FFmpeg's time_base.den to 30, but then QTKit decides to deliver you 15FPS with each frame having double the 30FPS duration (1/15 of a sec instead of 1/30), this is a problem because FFmpeg must have its time_base.den frames per second. If you don't give it that, your video will play back at an unexpected speed -- if you give it 15FPS for example, it will take 2 seconds in the source data's time to reach 30 FPS, but FFmpeg processes that in one second, and hence your video will end up playing back at double the speed.
>
> This was my dilemma -- capture is by definition a variable frame rate (it may end up coming at a fixed frame rate, but that isn't guaranteed, so you have to plan for variable), but FFmpeg must have fixed frame rate. So what I had to do was make variable frame rate look like a fixed frame rate. In short, you have to skip encoding frames entirely if they come too quickly (i.e. a higher frame rate than time_base.den), and you have to add frames if they are received too slowly (i.e. lower frame rate than time_base.den) by encoding the same frame more than once. You also have to convert your pts / dts received to the frame count relative again to the time_base.
>
> It isn't pretty. But it worked. I hope that helps.
>
are you talking about using the ffmpeg as a command line application?
I am not sure because I am mostly interested in the capabilities of
the muxer (e.g. used programatically in another application)? could
you elaborate what exactly you mean by "but FFmpeg must have fixed
frame rate"?