[FFmpeg-devel] Project orientation

James Almer jamrial at gmail.com
Sat Jul 4 18:37:25 EEST 2020


On 7/4/2020 11:43 AM, Nicolas George wrote:
> Hi.
> 
> When I first started progressively to contribute do FFmpeg, the project
> was far from mature, but it was very much alive.
> 
> There was attempts at implementing new and better codecs, directly in
> lavc's code base. There were attempts at designing new formats with
> useful features.
> 
> Even outside the specialized work on specific codecs and formats, there
> was creativity. At the API and framework level, there was work being
> done to do things in a smarter way, either more convenient or more
> efficient or both.
> 
> For example, the format negotiation in liabvfilter (which was mostly
> designed before I got involved: I am not singing my own praise here),
> with its pointers to pointers to pointers, is not run-of-the-mill code.
> Even if parts of it are made of common algorithms, the whole is very
> specific to FFmpeg, and its design was creative. Creativity was
> necessary later to extend it to support audio.
> 
> Nowadays, not so much. There is work in making highly-optimized
> decoders, this work is impressive and creative, there is no doubt about
> it. But as far as I can see, that is mostly all there is going on. The
> rest seems to be rather basic: fixing bugs (and things that are not
> bugs, like harmless integer overflows), implementing support for
> features that were decided and designed elsewhere.
> 
> And it is not just that it is not happening: there is genuine opposition
> to creativity: things that are not already justified externally, things
> that do not look like well-known patterns, are met with scepticism and
> eventually rejected.
> 
> At this point, we should ask ourselves:
> 
> Is it what we want the project to be, or is it just what we have let it
> become?
> 
> I do not think this evolution is the result of a deliberate choice. I
> think it is more the result of the stress of success and the stress of a
> fork. Success can stifle creativity, because creativity implies the risk
> of failure: the project has become advert to risk.
> 
> It has evolved that way, but are we fine with it?
> 
> I personally am not.
> 
> I find the new ambiance boring.
> 
> Creativity is the reason I practice development, and the reason I do not
> practice it professionally: my creativity cannot be put on a schedule.
> 
> My skills are not for micro-optimizing codec code: I cannot help on
> these tasks. If I am allowed to analyze this myself, I would say that my
> skills lie in general organization: making sure that the right code gets
> called at the right time, finding more convenient ways of doing tasks,
> etc.
> 
> I have several ideas for the project. Some are not directly related to
> multimedia at all; rather, the are invented for FFmpeg precisely, for
> FFmpeg's exact needs and ways of doing things. They relate to options
> and introspection, to inter-thread scheduling and asynchronous
> operation, to error reporting to the application, to handling of strings
> and serialization, to partial configurations of filters, to avoiding
> global state and allowing multiple instances, etc.
> 
> I have shown the first steps of some of my ideas (AVWriter a few months
> ago, avrefcount_template.h more recently), and the outcome was rather
> disheartening and discouraging: "it's too complex" (of course: I am
> putting all the complexity in one place so that the rest of the code can
> be simple, if you look at just that part it looks complex!), "why don't
> you just" do thing the usual way? (because I am trying to find a better
> way than the usual way.) It seems my fellow developers do not look beyond
> the immediate strangeness of the proposal to see the promised benefits,
> but remember: strangeness is just lack of familiarity, it goes away fast
> all by itself, and all that remains are the benefits.
> 
> These proposals would probably be better met if they were complete: if I
> proposed a patch series that eliminates escaping hell or gets
> non-blocking operation working all at once, it would be easier to get it
> accepted. But it is too big a task, especially with the rest of the code
> moving under my feet, with conflicts at each rebase.
> 
> Lacking that, they require a little trust: trust enough to look, beyond
> the immediate strangeness, where I am going and to try to see what I
> want to achieve, and see that it is achievable. But for now, I have seen
> more doubt and dismissal than trust and enthusiasm. And the projects are
> too big, I am not prepared to fight an uphill battle to get accepted
> every small step.
> 
> If I cannot express my creativity by developing for FFmpeg, I will find
> other projects, mine or existing, to express it. I suspect others have
> orbited away from the projects for similar reasons.
> 
> So, I ask one last time: What kind of project do you want FFmpeg to be?
> 
> Sorry for the interruption, you can resume your normal occupation.
> 
> Regards,

Just a few quick Saturday morning thoughts that don't cover your entire
email and may be just me rambling, but some of this could be linked to
the completely different world we're living in today than in say 2010,
regarding what is expected of multimedia projects. Nowadays almost
everything is streaming, and almost everything is mobile focused.
Getting a 1% speed up in one hot loop in h264dec that could mean the
difference between realtime or dropped frames in some ARM chip is
immensely more important and demanded than a smoother experience running
a filter chain, or a decoder for a 20 years old console game that would
aid with emulation efforts. And yes, even if those chips have a hw decoder.

Similarly, the HEVC patent pool disaster completely ruined any chance
for ffhevc to ever see any kind of real development, and the situation
indirectly (or not) resulted in what would have been the internal AV1
decoder becoming a standalone library: It required a extremely
permissive license for the sake of fast and widespread adoption that
ffmpeg could not provide with LGPL.
It's the first time a very important codec is not present internally in
our codebase, and that itself is proof things are definitely not the
same as they used to. I can't imagine ffmpeg without an internal h264
decoder ever making it anywhere a decade ago.
Of course anyone could port it today, but does anyone capable want to?
So far it doesn't look like.

Another thing worth mentioning is a lack of new blood. Despite
participating in GSoC for a long while, i can't name a student that
stuck around after the fact. Mind, there are new devs that started
contributing for other reasons, but perhaps not enough?


More information about the ffmpeg-devel mailing list