[FFmpeg-devel] [PATCH v2 1/3] doc: Explain what "context" means

[Stefano Sabatini]

On date Saturday 2024-04-20 23:17:57 +0100, Andrew Sayers wrote:
> On Sat, Apr 20, 2024 at 06:48:32PM +0200, Stefano Sabatini wrote:
> > On date Saturday 2024-04-20 13:19:41 +0100, Andrew Sayers wrote:
> > > Based largely on the explanation by Stefano Sabatini:
> > > https://ffmpeg.org/pipermail/ffmpeg-devel/2024-April/325854.html
> > > ---
> > >  doc/jargon.md | 169 ++++++++++++++++++++++++++++++++++++++++++++++++++
> > >  1 file changed, 169 insertions(+)
> > >  create mode 100644 doc/jargon.md
> > > 
> > > diff --git a/doc/jargon.md b/doc/jargon.md
> > > new file mode 100644
> > > index 0000000000..f967b5c8bc
> > > --- /dev/null
> > > +++ b/doc/jargon.md
> > > @@ -0,0 +1,169 @@
> > > +# Jargon
> > > +
> > > +Terms used throughout the code that developers may need to know.
> > > +
> > > + at anchor context
> > > +
> > 
> > > +## Context
> > > +
> > 
> > > +A design pattern that stores the context (e.g. configuration) for a series
> > > +of operations in a "context" structure, and moves other information with
> > > +a longer or shorter lifetime elsewhere.
> > 
> > I'd skip the mention of a design pattern since this is about the
> > jargon.
> > 
> > So a simplified variant would be:
> > 
> > A "context" is a a structure used to store information
> > (e.g. configuration and/or internal state) for a series of operations
> > working on the same data.
> 
> I think there's a pattern to the problems I'm having in this thread -
> *anchoring effects*.
> 
> If you ask someone "is 5 a big number?" then "is 5 thousand a big number?",
> they'll probably say "yes" to the second question.  But if you ask them
> "is 5 billian a big number?" then "is 5 thousand a big number?", they'll
> probably say "no".  In each case, their concept of "bigness" has been
> anchored by the first question you asked.
> 
> When I originally tried to learn FFmpeg back in the day, I got nowhere with my
> default OOP mindset.  It wasn't until I thought to read the examples with a
> procedural mindset that it started making any sense, and I think that has
> *anchored* my mental model of FFmpeg to a mindset that made it hard to think
> deeply about its object-oriented bits.
> 
> Yesterday I would have agreed this was just one piece of jargon that needed
> pinning down.  But if other people have similarly mis-anchored themselves,
> this question might need to be a bit easier for them to find.
> 
[...]
> > About the internal "private" context, this is mostly relevant for
> > FFmpeg development, and not really useful for API users (basically
> > they don't even need to know about the private data).
> > 
> > For example all they need to know is that for AVCodecContext generic
> > options they can set the fields in the context itself, or use
> > AVOptions, but they can only use AVOptions for "private" options.
> > 
> > We are not still enforcing the use of AVOption to set all options,
> > although we might want in the future.
> 

> I think you're saying that "context structure" is synonymous with "context",
> and is FFmpeg's term for a common style of C structure; but that other projects
> might use a different word, or write that style of struct without naming it at
> all?

Correct, althought this style is pretty common in plain C and there
are some commonly used conventions (e.g. the first parameter of the
related functions is usually the "context") but there is no common
jargon.

Examples:
https://github.com/freeswitch/sofia-sip/blob/master/libsofia-sip-ua/nua/nua_client.h
https://github.com/GNOME/glib/blob/main/gio/gsettings.h
https://code.videolan.org/videolan/x264/-/blob/master/x264.h?ref_type=heads

The meaning of "context" in FFmpeg maps pretty well on the meaning of
the English term (provides the context for a given operation needing
to work on the same data and with a changing state).

> If so, I'd argue it's important to give people a non-FFmpeg-specific
> *anchor*, but that we should expand the later FFmpeg-specific example, so they
> have an idea of how it's used around here.
> 

> A quick grep of the source suggests that "private context" is an accepted
> synonym for "internal context".  And it sounds like it fulfils the same purpose
> as C++ "private" access.  If both statements are true, then yes it doesn't need
> to go in the example, and the whole topic can be cut down to a line like "the
> main context is for public members, the private context is for private members".
> Sound good?

The "internal context" was added to specify options which are not
already covered in the "generic" options. Initially the options were
only specified in the "global" public context, as fields in the
generic structure (e.g. AVCodecContext for encoding/decoding).

One of the problems was that we needed to query the options - for
example so that you can print all the options with ffmpeg -help
encoder=libx264, so AVOptions were added to query those fields through
some form of rudimetary "introspection" (the AVOptions API implies
that there is a struct containing the offsets of the fields, together
with other metadata - a description, a type, and other data to perform
validation).

Later as more codecs/formats were added, there was the need to support
options only specified for specific elements. Such options were stored
in a private context.

In summary:
> the main context is for public members, the private context is for private members

This is correct with the clarification that members correspond to the
options the user can set/get on the affected component (encoder,
decoder, muxers, demuxers, filters etc) using the AVOptions API.

> 
> If we have public and private members, and then AVOption members are a third
> thing, the document should probably address the natural assumption that they're
> equivalent to C++ "protected" members (i.e. not fully private to the class, but
> not fully open to the public).  How about "It might help to think of
> AVOption-accessible public members as having 'protected' access, in that you
> should access them through the AVOptions API unless you know what you're
> doing.  This rule isn't always followed in practice, especially in older code"?

On the other hand, I'm worried to adopt the OOP terminology, as this
might convey the wrong impression that there is a complete OOP
internal implementation or that there is a perfect mapping with OOP
concepts. Also a pure C programmer might not be familiar with the OOP
terminology, and this might be confusing to her.

For example, the main reason for private options was not really to
make them protected, but rather to make it possible to set
component-specific (aka private) options. For setting generic options,
it's still pretty common to set the fields directly, although this has
several disadvantages (no validation is performed, and this is not
robust to API changes - e.g. in case the field is moved in the
structure).

Private options fields direct access is technically not possible
(i.e. you cannot set an option for the x264 encoder by directly
accessing the private x264 context, since this is an opaque field), so
this is not even a concern.

[...]
> > > +FFmpeg itself uses the context design pattern to solve many problems.
> > > +You can use this pattern anywhere it would be useful, and may want to use
> > > +AVClass and @ref avoptions "AVOptions" if they're relevant to your situation.
> > 
> > But again, I'm confused by this since it's confusing two levels:
> > internal API development and API usage. When you write "may want to
> > use" it seems to refer to the former, but the user should not really
> > care about this (unless he wants to know how the internal
> > implementation works).
> > 
> > In fact, while one user might want to use the FFmpeg API as a generic
> > development toolkit (and therefore create its own custom API with
> > AVClass and AVOptions) I don't think this is really very common.
> 

> I think this is another anchoring problem on my part.  The AVOptions docs[1]
> describe how to add AVOptions in accessible language that made me think it was
> aimed at ordinary programmers who happen to use FFmpeg.  Would it be better for
> the line below "Implementing AVOptions" on that page to say something like:
> 
>  This section describes how to add AVOptions capabilities to a struct.

> +It is intended for developers of new FFmpeg libraries, but use outside of FFmpeg
> +is also possible.

Yes, but I'd say instead:

It is intended for developers of FFmpeg/internal FFmpeg development,
...

(since usually FFmpeg is extended by adding new API - e.g. new headers
or new functions - not necessarily libraries).

> 
> If so, I'll make a separate patch for that and rewrite the document to match.

Thanks.