[FFmpeg-devel] [RFC] 5 year plan & Inovation

[Andrew Sayers]

On 17/04/2024 14:58, Michael Niedermayer wrote:
> Hi all
>
> The pace of inovation in FFmpeg has been slowing down.
> Most work is concentarted nowadays on code refactoring, and adding
> support for new codecs and formats.
>
> Should we
> * make a list of longer term goals
> * vote on them
> * and then together work towards implementing them
> ?
>
> (The idea here is to increase the success of larger efforts
>   than adding codecs and refactoring code)
> It would then also not be possible for individuals to object
> to a previously agreed goal.
> And it would add ideas for which we can try to get funding/grants for
>
> (larger scale changes need consensus first that we as a whole want
>   them before we would be able to ask for funding/grants for them)
>
> Some ideas and why they would help FFmpeg:
>
> * Switch to a plugin architecture
>      (Increase the number of developers willing to contribute and reduce
>       friction as the team and community grows)
> * ffchat
>      (expand into realtime chat / zoom) this would
>      bring in more users and developers, and we basically have almost
>      all parts for it already but some people where against it
> * client side / in browser support
>      (expand towards webapps, webpages using ffmpeg client side in the browser)
>      bring in more users and developers, and it will be costly for us
>      if we let others take this area as its important and significant
> * AI / neural network filters and codecs
>      The future seems to be AI based. Future Filters and Codecs will use
>      neural networks. FFmpeg can be at the forefront, developing these
> * [your idea here]
>
> thx
>
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".


I've occasionally tried getting into ffmpeg for over a decade now, and have
lurked here for the past few months as I put in the effort to grok it.
On behalf of people who could contribute but don't, I'd like to suggest
ffmpeg focus on *learnability*.

Whenever I've tried to learn ffmpeg, I've always been rebuffed by
documentation that seems needlessly hard to use.  I understand some of
these reflect deeper issues - for example there's a reason the words
"ffmpeg" and "libav" are used ambiguously, even though it makes it
hard to differentiate between the library and the command-line tool.
But other issues seem like quick wins - for example I've lost count of
all the times I typed two functions into Google, spent hours trying to
make them work together, then finally realised I was looking at the
documentation for 3.0 in one tab and 5.0 in the other.  Surely you can
just add a line to the top of the documentation like "click here to see
the trunk version of this file"?

Here's a small example to demonstrate the larger issue -
what does it mean for something to be a "context"?

When I started learning how to write ffmpeg code, I read through the
docs and saw various things calling themselves "context" structs, but
never found a link to explain what that meant.  If I was a young
developer, I would probably have just assumed it was standard
programming jargon I was too dumb to know, and walked away to find
something more my speed.  But I'm old and stubborn and have nothing
better to do right now, so I kept going...

I tried to learn by going through the examples, but the nearest thing
to an explanation was e.g. `transcode.c` making up a new type and
calling it a `FilteringContext`.  I ignored the AVClass documentation
for a long time because the name made me think it was some kind of
GObject-style C-with-classes thing.  It was only when I noticed that
it says it describes "the class of an AVClass context structure" that
I realised what I was looking at.  And it was only when I convinced
myself that the documentation for AVOptions was using
"AVOptions-enabled struct" to mean the same thing as "AVClass context
structure" that I felt able to disregard the `FilteringContext`.  So
my current opinion is that "AVOptions-enabled struct", "AVClass
context structure" and "context structure" are different terms for the
same thing - but now I've said that publicly, I will no doubt find an
"SwrClass context structure" or something tomorrow.

To bring this back to ffmpeg development - I made a note to write a
patch saying they were synonyms and linking that explanation from
e.g. AVCodecContext, but the job has been rotting away in my todo list
for a month now waiting for me to be "sure I've got it right" (i.e. to
give up a perfect excuse for procrastination).  To be blunt, on some
level it feels like I've put more in than I've got out of that problem,
and haven't been able to psych myself up to submit a patch that could
get me laughed out of the room (or worse, politely ignored).  If the
project was otherwise easy-to-learn, I would have felt a much stronger
sense of obligation to pay that back.

 	- Andrew Sayers