[FFmpeg-devel] On libavfilter: A Summary of Issues

Derek Buitenhuis derek.buitenhuis at gmail.com
Sun Oct 7 01:38:36 CEST 2012 

So I've been asked to try and detail all the problems with libavfilter
I've heard or experience (many from a "certain IRC channel"). What
follows is my attempt to summarize them, and is very  likely an incomplete
list that will certainly clash some people's ideologies.

This email has sprung from a lot of "Y U NO USE LIBAVFILTER???" questions
from FOSS-y sorts of people regarding people's use of Avisynth[1] and
the new hot stuff, VapourSynth[2]. As such, you can expect a lot
of comparisons.

Ill-Defined Goals and Use-Cases
------------------------------

I can really sum up the normal reaction of people who read about
libavfilter with this quote:

[17:11] < Daemon404> hey
[17:11] < Daemon404> im sourcing a list of issues against libavfilter
[17:11] < Daemon404> anyone care to contribute?
[17:12] < Kovensky> idk
[17:12] < Kovensky> nobody uses it
[17:12] <@Myrsloik> 1. what is it?
[17:12] < Kovensky> I have no idea how to use it either
[17:12] <@Myrsloik> 2. why should I complain?
[17:12]  * Myrsloik even wrote a filter for it once
[17:14] < Zor> "<@Myrsloik> 1. what is it?" pretty much summarizes
               everyone's impression of lavfi

Is it supposed to be an extensible filtering framework that 3rd
party devs write filters for and use for a lot of video processing,
like Avisynth and VapourSynth? Is it supposed to be something to
mainly be used as a C API from others' applications? Just ffmpeg.c?
Whats the goal and direction of libavfilter as library? I think
it could really benefit from some sort of public statement, declaration
or documentation of this.


It's a Monolithic Library
-------------------------

This is a massive dealbreaker for any sort of filtering framework,
especially if you want to write your own filters or work in any sort
of dynamic way. Having to recompile is a massive hinderance and
introduces annoying compatability issues for anyone wishing to
use libavfilter and one of its filters, or a filter they've written
themselves.

Ideally there would be a -stable- and -long term- API and ABI for
user written plugins. This, and I cannot stress this enough, is
-essential-. If I write a plugin in 2012, and distro a binary under
a commercial license, or simply just a freeware license, I should
be able to load it in 2015. This is a something FOSS in general
is VERY bad at all, and FFmpeg in particular. One of the reasons
Avisynth has done so well to this way is that:

1) Users may write closed source plugins.
2) They still work with AviSynth today. You can even still load 2.0
   plugins in 2.5 and 2.6, if you are so inclined.

Also, to clarify, when I say dynamic, I do not mean linking. I mean
completely dynamic. LoadLibrary() and dlopen().

The other issue that the whole monolithic library thing creates is
that any user who wants to distribute a plugin written for it
-must- abide by the LGPL, at the very least. See 1) above.

Basically, a monolithic library is not very flexible and has limited
use.

**PREEMPTIVE STRIKE: Before anyone brings up how bad gstreamer is,
                     please note its not gstreamer's dynamic nature
                     that makes it so horrible.


There's No Real Scripting (No, Really)
--------------------------------------

Don't kid ourselves. The "scripting" in libavfilter is not really scripting.
There is no real language, or syntax, just a graph and a lackluster eval. On
top of that you must specify it all in the cli arguments rather than script
files, and the syntax is absolutely atrocious. To quote our own docs:

    ffmpeg -i opening.mkv -i episode.mkv -i ending.mkv -filter_complex \
      '[0:0] [0:1] [0:2] [1:0] [1:1] [1:2] [2:0] [2:1] [2:2]
       concat=n=3:v=1:a=2 [v] [a1] [a2]' \
      -map '[v]' -map '[a1]' -map '[a2]' output.mkv

Yeah. No.

One of the great appealing aspects of Avisynth and VapourSynth is that users
who aren't necessarily programmers, let alone C programmers, can fairly easily
extend it via scripted functions. Users should not have to write C plugins
to do every little trivial task. One benefit of VapourSynth is that you
can bind it to any language, as well.


No Documentation for the API
----------------------------

The usage of filters is documented fairly well. No problem there. The using
of libavfilter itself, however, really isn't well documented. Examples are
not documentation.

At least the public API has an example. Want to write a filter for libavfilter?
Good luck! Hope you like grep and trying to find similar commits to copy.


The Public API is Overly Complex (Some Even Say Terrible)
---------------------------------------------------------

I'm going to compare again to Avisynth here. Its API for using it, and for
writing plugins for it is super simple. It has a very low bar for entry.
Get a buffer and metadata. Do stuff with it. Return a buffer, or skip. It's not
nearly this simple in libavformat. Just look at what should be an
extremely simple filter of no more than a few lines: vf_select.c. 443 lines.

Relevant Quote:

[18:51] <@TheFluff> the interface is so terrible that I'd gladly take ANY other
                    solution whatever it is regardless of how bad it is


It Offers Nothing Of Value; Filters or Otherwise
------------------------------------------------

Every filter in libavfilter is available in some other, better framework,
and as been for a very long time. Usually in an easier-to-use capacity. Most
of libavfilter consists of ancient mplayer plugins, or similar. They're no
longer useful in 2012, either. Almost every single plugin in libavfilter has
better working alternatives in other frameworks. To name some of the obvious
ones:

- mvtools (No, exposing the motion estimation in libavcodec is not a valid
  alternaive.)
- Anything by tritical (TIVTC, EEDI3, etc)
- FieldHints
- SeparateFields
- SelectEvery
- AddGrain
- QTGMC (Yes it's script. A good one. See an earlier point.)
- DitherTools
- Non-ancient copies of gradfun, and pals.
- Many, many more. See Doom9.

So, why, as a third party developer, should I use libavfilter for anything,
or write any plugins? What benefits does it offer over anything else?

[18:51] <@TheFluff> the interface is so terrible that I'd gladly take ANY other
                    solution whatever it is regardless of how bad it is


Conclusion
---------

I am certain I have missed some problems. Others may respond with theirs.

This entire email may seem like a bitch-fest, but I think it's better to get
the common complaints out there than ignore the two ton pink elephant in the
room.

My final question is: What is the point of libavfilter existing in the first
place, besides Not In House Synrome?

Trolls, flames, and perhaps constructive discussion is welcome.

- Derek

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 895 bytes
Desc: OpenPGP digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20121006/131d9ec1/attachment.asc>

More information about the ffmpeg-devel mailing list