[FFmpeg-devel] [PATCH 1/3] lavc/encode: make sure frame timebase matches encoder, when set

[Anton Khirnov]

Quoting Marton Balint (2022-10-05 20:54:46)
> 
> 
> On Tue, 4 Oct 2022, Anton Khirnov wrote:
> 
> > Quoting Marton Balint (2022-09-28 21:54:11)
> >>
> >>
> >> On Wed, 28 Sep 2022, Anton Khirnov wrote:
> >>
> >>> AVFrame.time_base has been added recently, but is currently not used for
> >>> anything. Prepare for its use in encoders by rejecting frames where
> >>> time_base is set, but differs from the AVCodecContext one.
> >>
> >> How is that not an API break? Users can encode AVFrames with anything in
> >> the AVFrame->time_base right now, if you change that behaviour, that will
> >> surely break some code. That is why it was explicitly documented that
> >> it will be ignored by encoders by default.
> >
> > Why would there be anything in that field? No code we have currently
> > sets that field or does anything with it.
> 
> It is a public field which was explicitly documented to be ignored by 
> filters or encoders. The user could store any data in it, because the 
> documentation of the field ensured it will not be a problem.
> 
> If you read back the old threads which added AVFrame->time_base 
> you will find the reasoning behind the original comments, in fact,
> you suggested the actual wording for the documentation of the field, and 
> now you want now to change the semantics of the field which contradicts 
> the existing documentation... Usually we introduce a new field and 
> deprecate the old if we want to do something like this.

Honestly I do not remember why I wrote it that way and it now seems like
a mistake to me. We did not add this field as random-number storage for
our callers, we added it to use it in the libraries.

> One could argue that this break is "small" enough, to not dance around it, 
> but I don't really see the benefit of the change in the first place. So 
> the real question is why do you want to start using AVFrame->time_base in 
> encoders, and what is the feature which is undoable with the current 
> AVCodecContext->time_base?

Where I'm going with this is duration handling in encoders. libavcodec
currently disregards AVFrame.duration completely, while properly it
should set output AVPacket.duration from its corresponding
AVFrame.duration. The problem with that is that we cannot just use
AVFrame.duration, because there is no guarantee that the caller is aware
of it and set in the correct timebase. E.g. ffmpeg.c currently does not
touch AVFrame.duration, so whatever value it acquired along the way
(from the decoder or some filter) will linger there in whatever timebase
it was originally in, which is not necessarily equal to the encoder
timebase.

So my idea was to use the timebase being set as an indicator of the
caller being duration-aware. But on more thought, this seems rather
obscure and an explicit option is probably better.

-- 
Anton Khirnov