[FFmpeg-devel] [PATCH] avdevice/avdevice: Deprecate AVDevice Capabilities API

[Nicolas George]

Mark Thompson (12021-01-27):
> See for example the list of suggestions I made for improving the API a few days ago.

I intended to reply to them, but the conversation never came back to it
before now.

> * Handle frames as well as packets.

Already possible.

>   * Including hardware frames - DRM objects from KMS/V4L2, D3D
>   surfaces from Windows desktop duplication (which doesn't currentl y
>   exist but should).

I do not know if it is possible, you know this stuff better than me.

> * Clear core option set - currently almost everything is set by
> inconsistent private options; things like pixel/sample format,
> frame/sample rate, geometry and hardware device should be common
> options to all.

That would be good, but can be done with any API.

> * Asynchronicity - a big annoyance in current recording scenarios with
> the ffmpeg utility is that both audio and video capture block, and do
> so on the same thread which results in skipped frames.

This is a problem for devices, protocols and demuxers, and even filters,
and is therefore not an argument for a separate device API.

Please know that I am working on fixing this issue. This is the event
loop stuff. But it is a complex task.

> * Capability probing - the existing method of options which log the
> capabilities are not very useful for API users.

This is true, but it is not a problem with the API, just a missing
feature. See below for details about what I mean.

> How are we to determine who has the most relevant experience, so that
> we avoid appealing to a false authority?

Please accept my apology. I mistakenly assumed that you were speaking
from the same of lack of experience as the other people who want to
destroy libavdevice.

Did you use libavdevice in projects, not through ffmpeg.c, as well?

> Please will you explain what your ideas are about how to enhance the
> API of libavdevice.
> 
> Even if we disagree about exactly where such changes should be
> implemented, I would very much welcome hearing about the improvements
> you would like to make underneath.

First, I need to say that I agree with you that intuitively libavdevice
needs a dedicated API: devices are not the same thing as muxers and
demuxers, they have special need.

BUT...

But it means yet another f...ing API, and that is not good either. It
requires learning another API, special code, etc. Or it will be used
only through wrappers, and as such useless in itself.

Devices have special needs, but at the core, they are components that
produce or consume frames.

We already have APIs for components that produce or consume frames. We
already have several of them in fact.

So, my thoughts on devices is:

Let us choose the most nimble and extensible such API, and extend it as
needed.

This is, in essence, object-oriented design: instead of inventing a
completely new interface for devices, let the interface for devices
inherit from an existing interface, choosing the best one.

How does it sound for now?

> Indeed.  The most obvious use-case for much of what I have said is of
> course the ffmpeg utility itself - the ability to avoid pointless
> copies when working with devices and to be able to record video and

That would be nice, but I wonder how urgent it is.

> audio at the same time without weird interactions would both be useful
> features.

This would be very useful, but it is not specific to devices. We need to
be able to capture from the sound device and from a networked webcam
too.

If you try to fix this with a device API, you will be missing the bigger
picture.

> I should note that my original intent in engaging with this discussion
> was to gather thoughts from other members of the project wrt this sort
> of improvement before doing significant work on it, to avoid
> proceeding down a path which wouldn't go anywhere useful.

My advice is: before starting to work on code, discuss concrete details
of your project.

> I would prefer that you do not refrain from offering constructive
> criticism of "the VAAPI stuff", or anything else that I work on,
> should you have any.

I will refrain still, because I do not know anything about them.

Regards,

-- 
  Nicolas George
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20210201/328c66f5/attachment.sig>