[FFmpeg-devel] [PATCH 5/8] lavd: add device capabilities API

[Lukasz Marek]

On 4 April 2014 19:28, Michael Niedermayer <michaelni at gmx.at> wrote:
>
> >  /**
> > @@ -228,6 +231,128 @@ int avdevice_dev_to_app_control_message(struct
> AVFormatContext *s,
> >                                          void *data, size_t data_size);
> >
> >  /**
> > + * Following API allows user to probe device capabilities (supported
> codecs,
> > + * pixel formats, sample formats, resolutions, channel counts, etc).
> > + * It is build on top op AVOption API.
> > + * Queried capabilities allows to set up converters of video or audio
> > + * parameters that fit to the device.
> > + *
> > + * List of capablities that can be queried:
> > + *  - Capabilities valid for both audio and video devices:
> > + *    - codec:          supported audio/video codecs.
> > + *                      type: AV_OPT_TYPE_INT (AVCodecID value)
> > + *    - format:         supported pixel/sample formats.
> > + *                      type: AV_OPT_TYPE_INT (AVPixelFormat or
> AVSampleFormat value)
> > + *  - Capabilities valid for audio devices:
> > + *    - sample_rate:    supported sample rates.
> > + *                      type: AV_OPT_TYPE_INT
> > + *    - channels:       supported number of channels.
> > + *                      type: AV_OPT_TYPE_INT
> > + *    - channel_layout: supported channel layouts.
> > + *                      type: AV_OPT_TYPE_INT64
> > + *  - Capabilities valid for audio devices:
> > + *    - window_size:    supported window sizes (describes size of the
> window size presented to the user).
> > + *                      type: AV_OPT_TYPE_IMAGE_SIZE
> > + *    - frame_size:     supported frame sizes (describes size of
> provided video frames).
> > + *                      type: AV_OPT_TYPE_IMAGE_SIZE
> > + *    - fps:            supported fps values
> > + *                      type: AV_OPT_TYPE_RATIONAL
> > + *
> > + * Value of the capability may be set by user using av_opt_set()
> function
> > + * and AVDeviceCapabilitiesQuery object. Following queries will
> > + * limit results to the values matching already set capabilities.
> > + * For example, setting a codec may impact number of formats or fps
> values
> > + * returned during next query. Setting invalid value may limit results
> to zero.
> > + *
> > + * Example of the usage basing on opengl output device:
> > + *
>
> > + * @code
> > + *  AVFormatContext *oc = NULL;
> > + *  AVDeviceCapabilitiesQuery *caps = NULL;
> > + *  AVOptionRanges *ranges;
> > + *  int ret;
> > + *
> > + *  if ((ret = avformat_alloc_output_context2(&oc, NULL, "opengl",
> NULL)) < 0)
> > + *      goto fail;
>
>
> > + *  if (avdevice_capabilities_create(&caps, oc, NULL) < 0)
> > + *      goto fail;
>
> can this be called multiple times ?
> if not, why is a free function needed ?
> caps could just be kept track of and freed in avformat_free_context()
>

I simple case there is no need to call it more than once.
I don't know if there is a useful use case that would need more.

> + *
> > + *  //query codecs
> > + *  if (av_opt_query_ranges(&ranges, caps, "codec",
> AV_OPT_MULTI_COMPONENT_RANGE)) < 0)
> > + *      goto fail;
> > + *  //pick codec here and set it
> > + *  av_opt_set(caps, "codec", AV_CODEC_ID_RAWVIDEO, 0);
> > + *
> > + *  //query format
> > + *  if (av_opt_query_ranges(&ranges, caps, "format",
> AV_OPT_MULTI_COMPONENT_RANGE)) < 0)
> > + *      goto fail;
> > + *  //pick format here and set it
> > + *  av_opt_set(caps, "format", AV_PIX_FMT_YUV420P, 0);
> > + *
> > + *  //query and set more capabilities
> > + *
> > + * fail:
> > + *  //clean up code
> > + *  avdevice_capabilities_free(&query, oc);
> > + *  avformat_free_context(oc);
>
> naively i would have thought it could be done like this:
>
> if ((ret = avformat_alloc_output_context2(&oc, NULL, "opengl", NULL)) < 0)
>     goto fail;
>
> if (av_opt_query_ranges(&ranges, oc, "codec",
> AV_OPT_MULTI_COMPONENT_RANGE)) < 0)
>     goto fail;
> av_opt_set(caps, "codec", AV_CODEC_ID_RAWVIDEO, 0);
>
> if (av_opt_query_ranges(&ranges, oc, "format",
> AV_OPT_MULTI_COMPONENT_RANGE)) < 0)
>     goto fail;
> av_opt_set(caps, "format", AV_PIX_FMT_YUV420P, 0);
>
> but maybe this has some issue, can you explain why a seperate
> "external" struct wth alloc and apply functions is needed ?
>

This is similar to one of previous versions I posted.
http://ffmpeg.org/pipermail/ffmpeg-devel/2014-February/153977.html

I don't remember now all problems it caused, but you need add AVOptions
that are probed with this API to device options.
They will be visible to the user and may confuse. Also some devices have
some of this options already, sometimes under different name etc.
So this is one problem I remember.

Also, when new caps are added they would need updates everywhere, now just
query method need update (not very significant, but still easier to
maintain)

One solution I see now is to add AVDeviceCapabilitiesQuery * to
AVFormatContext and allocate it on first use and free when context is freed
(as you pointed above).
Add new method - maybe in libavdevice space that wraps av_opt_query_ranges
but replace object, something like

av_opt_query_capability_ranges(AVOptionRanges **ranges, AVFormatContext
*oc, ...)
{
    if (!oc->caps)
       avdevice_capabilities_create(&oc->caps)
    return av_opt_query_ranges(ranges, oc->caps, ...)
}


Iam not agaist it at all if its neeedd, i am just trying to keep the
> API simple if its not
> and the difference here would be 2-3 function calls


I may try the solution I just posted above.