[FFmpeg-devel] [RFC] 5 year plan & Inovation

[Stefano Sabatini]

On date Wednesday 2024-04-17 17:24:02 +0100, Andrew Sayers wrote:
> On 17/04/2024 14:58, Michael Niedermayer wrote:
[...]
> I've occasionally tried getting into ffmpeg for over a decade now, and have
> lurked here for the past few months as I put in the effort to grok it.
> On behalf of people who could contribute but don't, I'd like to suggest
> ffmpeg focus on *learnability*.
> 

> Whenever I've tried to learn ffmpeg, I've always been rebuffed by
> documentation that seems needlessly hard to use.  I understand some of
> these reflect deeper issues - for example there's a reason the words
> "ffmpeg" and "libav" are used ambiguously, even though it makes it
> hard to differentiate between the library and the command-line tool.

> But other issues seem like quick wins - for example I've lost count of
> all the times I typed two functions into Google, spent hours trying to
> make them work together, then finally realised I was looking at the
> documentation for 3.0 in one tab and 5.0 in the other.  Surely you can
> just add a line to the top of the documentation like "click here to see
> the trunk version of this file"?

Functions are documented in doxygen, so they depend on the major.minor
version, while you seem to refer to the FFmpeg version. Also on the
website we usually only have the latest mainline documentation, so I
don't understand how can you have different versions in different tabs
(unless you didn't update that tab since months/years).

If you mean that we should ship documentatation for the latest
supported releases, in addition to latest mainline, I tend to agree.

> Here's a small example to demonstrate the larger issue -
> what does it mean for something to be a "context"?
> When I started learning how to write ffmpeg code, I read through the
> docs and saw various things calling themselves "context" structs, but
> never found a link to explain what that meant.  If I was a young
> developer, I would probably have just assumed it was standard
> programming jargon I was too dumb to know, and walked away to find
> something more my speed.  But I'm old and stubborn and have nothing
> better to do right now, so I kept going...
> 
> I tried to learn by going through the examples, but the nearest thing
> to an explanation was e.g. `transcode.c` making up a new type and
> calling it a `FilteringContext`.  I ignored the AVClass documentation
> for a long time because the name made me think it was some kind of
> GObject-style C-with-classes thing.  It was only when I noticed that
> it says it describes "the class of an AVClass context structure" that
> I realised what I was looking at.  And it was only when I convinced
> myself that the documentation for AVOptions was using

> "AVOptions-enabled struct" to mean the same thing as "AVClass context
> structure" that I felt able to disregard the `FilteringContext`.  So
> my current opinion is that "AVOptions-enabled struct", "AVClass
> context structure" and "context structure" are different terms for the
> same thing - but now I've said that publicly, I will no doubt find an
> "SwrClass context structure" or something tomorrow.

In general, a "context" in the FFmpeg jargon is usually a data
structure providing the context/state/configuration for a given
operation, which can be muxing, demuxing, decoding, encoding,
filtering etc.

You need to fill the "context" with the configuration parameters and
with the data needed for the specific operation.

In general, when setting up a context, you also want some facilities
to avoid to repeat logic again and again:
- you want to provide means to send log messages
- you want an interface to query, set, and get options
- you want a "private" internal context, with options/parameters
  specific for a particular instance of a generic context. For example
  you might want to set specific options which only apply to a given
  encoder (these are so-called "private" options)

This is done through the AVClass structure, which being generic is
used in various parts of FFmpeg.

The AVOption interface wass added later, so depending on your usage,
you might be directly setting a field in the context of set an option
through the AVOption programming interface.

Currently this is the documentation for AVCodecContext:

/**
 * main external API structure.
 * New fields can be added to the end with minor version bumps.
 * Removal, reordering and changes to existing fields require a major
 * version bump.
 * You can use AVOptions (av_opt* / av_set/get*()) to access these fields from user
 * applications.
 * The name string for AVOptions options matches the associated command line
 * parameter name and can be found in libavcodec/options_table.h
 * The AVOption/command line parameter names differ in some cases from the C
 * structure field names for historic reasons or brevity.
 * sizeof(AVCodecContext) must not be used outside libav*.
 */
typedef struct AVCodecContext {

I agree this might be improved/updated, "main external API structure"
is indeeed pretty vague and it misses many references to other part of
the API (in particular the AVOption one defined in
libavutil). Probably sending a patch to expand/extend/clarify this
would be a good starting point.

> To bring this back to ffmpeg development - I made a note to write a
> patch saying they were synonyms and linking that explanation from
> e.g. AVCodecContext, but the job has been rotting away in my todo list
> for a month now waiting for me to be "sure I've got it right" (i.e. to
> give up a perfect excuse for procrastination).  To be blunt, on some
> level it feels like I've put more in than I've got out of that problem,
> and haven't been able to psych myself up to submit a patch that could
> get me laughed out of the room (or worse, politely ignored).  If the
> project was otherwise easy-to-learn, I would have felt a much stronger
> sense of obligation to pay that back.

This looks like the usual pattern: experienced API developers don't
feel the need to improve the docs, so they have no real incentives to
do so; outsiders might need to improve it, but this means to
understand the intricacies of the framework, and by the time they got
it they probably don't need it anymore.

Also writing good documentation - as writing good prose - is *hard*,
but in general bad is better than nothing.