[FFmpeg-devel] [PATCH] avdevice/avdevice: Deprecate AVDevice Capabilities API

Mark Thompson sw at jkqxz.net
Wed Jan 27 23:32:31 EET 2021


On 27/01/2021 14:36, Nicolas George wrote:
> Mark Thompson (12021-01-26):
>> Even after such a merge, the libavdevice API is still a problem.
> 
> I will re-center the discussion on this alone.
> 
> And I ask, simply: what problem exactly?

See for example the list of suggestions I made for improving the API a few days ago.

On 20/01/2021 12:41, Mark Thompson wrote:
 > * Handle frames as well as packets.
 >    * Including hardware frames - DRM objects from KMS/V4L2, D3D surfaces from Windows desktop duplication (which doesn't currently exist but should).
 > * Clear core option set - currently almost everything is set by inconsistent private options; things like pixel/sample format, frame/sample rate, geometry and hardware device should be common options to all.
 > * Asynchronicity - a big annoyance in current recording scenarios with the ffmpeg utility is that both audio and video capture block, and do so on the same thread which results in skipped frames.
 > * Capability probing - the existing method of options which log the capabilities are not very useful for API users.

> You, who AFAIK, do not maintain anything in libavdevice and have never
> used it in a project, say there is a problem with its API. I, who
> maintain parts of libavdevice and have been using it in projects for
> years, say there is no problem, there are ways to enhance it, but on the
> whole it works well enough.
> 
> Who might be right? Maybe the person with the most experience, don't you
> think?

How are we to determine who has the most relevant experience, so that we avoid appealing to a false authority?

For example, we could ask who has made the most changes to the library in the last few years:

$ git log --since 2015 libavdevice/ | grep "^Author:" | grep 'Nicolas\|Mark' | sort | uniq -c | sort -n
       9 Author: Mark Thompson <sw at jkqxz.net>
       7 Author: Nicolas George <george at nsup.org>

Or maybe we could go by lines of code?

$ git log --since 2015 --author='sw at jkqxz.net' --patch libavdevice/ | diffstat -s
  9 files changed, 827 insertions(+), 121 deletions(-)
$ git log --since 2015 --author='george at nsup.org' --patch libavdevice/ | diffstat -s
  6 files changed, 73 insertions(+), 61 deletions(-)

Or maybe these are completely meaningless measures which can be cherry-picked to show whatever answer we want and we should instead consider the merits of the proposals involved rather than trying to dismiss the person making them?

> Therefore, I will cut short this discussion:
> 
> If you want to know what my ideas are to enhance the API of libavdevice,
> just ask, and if you'll read the answer I'll be happy to explain at
> lengths.

Please will you explain what your ideas are about how to enhance the API of libavdevice.

Even if we disagree about exactly where such changes should be implemented, I would very much welcome hearing about the improvements you would like to make underneath.

> If you have a diagnostic of an actual problem, please share it.
> 
> If you have a detailed plan to make libavdevice's API significantly
> better, then please explain it, I am keenly interested.
> 
> Another point: remember that in this project, we have a policy of no new
> API without user: we do not add a new function or API just for the fun
> of it; we add a new function or API when and if it allows to add a
> feature, to simplify existing code, etc.
> 
> So, if you invent a new design for devices, and you implement an useful
> new feature thanks to this new design, or even better, several useful
> new features, then excellent.

Indeed.  The most obvious use-case for much of what I have said is of course the ffmpeg utility itself - the ability to avoid pointless copies when working with devices and to be able to record video and audio at the same time without weird interactions would both be useful features.

I should note that my original intent in engaging with this discussion was to gather thoughts from other members of the project wrt this sort of improvement before doing significant work on it, to avoid proceeding down a path which wouldn't go anywhere useful.

> But if you propose to just add a grand new design for devices, and add a
> wrapper so that existing devices can be used unchanged, and another
> wrapper so that existing applications can still use the old API, and
> that's it, then you are making it worse.
> 
> So, since AFAIK, you do not have plans to add new useful features to
> devices, or to libavfilter as a whole, my advice is: for now, leave it
> alone, until you have given it much more thought.
> 
> (In return, I will continue to refrain from criticizing the design of
> the VAAPI stuff.)

I would prefer that you do not refrain from offering constructive criticism of "the VAAPI stuff", or anything else that I work on, should you have any.

Thanks,

- Mark


More information about the ffmpeg-devel mailing list