[FFmpeg-devel] [PATCH] doc/filters: add documentation to all existing OpenCL filters, except tonemap filter

[Mark Thompson]

On 05/07/18 15:11, Danil Iashchenko wrote:
> add documentation to all existing OpenCL filters
> ---
>> I like this patch, the other thing, is it have any plan to update
>> openCL part like VAAPI (https://trac.ffmpeg.org/wiki/Hardware/VAAPI)
>> in https://trac.ffmpeg.org/wiki/HWAccelIntro, thanks.
> I am going to add it during the next week, thanks!
> 
>> Why not for tonemap_opencl?
> Added.

The title still says "except tonemap", though.

>> Since these filters are distinct, please list options as well, since their software counterparts could conceivably change. Option entries should include default value, range, and semantic.
> Fixed.
> 
>> Why are all examples of filters, that operate upon a single stream, shown within -filter_complex? '-vf' is preferred.
> Fixed.
> 
>> P.S. Once this new section is created, program_opencl entry should be shifted there, in a separate patch.
> Shifted to opencl section.

There is openclsrc as well.

I'm not sure I'm completely in favour of that change, anyway?  The section feels more coherent to me as "OpenCL versions of other filters" rather than "all filters using OpenCL" - program_opencl/openclsrc do something really quite different.  Does anyone else have an opinion on that?

(Also, the "in a separate patch" comment from Gyan is probably sensible if you keep this way around.)

>  doc/filters.texi | 741 +++++++++++++++++++++++++++++++++++++++++++++----------
>  1 file changed, 611 insertions(+), 130 deletions(-)
> 
> diff --git a/doc/filters.texi b/doc/filters.texi
> index fb5f3ee..16a350a 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> ...
> @@ -17422,6 +17292,617 @@ pixel format "yuv422p" @var{hsub} is 2 and @var{vsub} is 1.
>  
>  @c man end VIDEO FILTERS
>  
> + at chapter OpenCL Video Filters
> + at c man begin OPENCL VIDEO FILTERS
> +
> +Below is a description of the currently available OpenCL video filters.
> +
> +To enable compilation of these filters you need to configure FFmpeg with
> + at code{--enable-opencl}.

A pointer to the -init_hw_device/-filter_hw_device options would probably be a good idea here, since all of the examples using hwupload/hwdownload require that.

> + at section avgblur_opencl
> +
> +Apply average blur filter.
> +
> +The filter accepts the following options:
> +
> + at table @option
> + at item sizeX
> +Set horizontal kernel size. By default value is @code{1}.

I know it's copied from the avgblur filter, but this sounds very misleading to me.  It's the radius of the kernel, not the size of the kernel itself - it works on 2*sizeX+1 by 2*sizeY+1 rectangle, not a sizeX by sizeY one.  (Could make non-opencl avgblur clearer in a separate patch too.)

> +
> + at item planes
> +Set which planes to filter. By default all planes are filtered.
> +
> + at item sizeY
> +Set vertical kernel size, if zero it will be same as @code{sizeX}.
> +By default value is @code{0}.
> + at end table
> +
> + at subsection Example
> +
> + at itemize
> + at item
> +Apply average blur filter with sizeX and sizeY set to 3

For an example, I'd say a bit more about exactly what it does - e.g. "Apply average blur filter with horizontal and vertical size of 3, setting each pixel of the output to the average value of the 7x7 region centred on it in the input." (or something like that, maybe a comment about what it does at the edges too).

> + at example
> +-i INPUT -vf "hwupload, avgblur_opencl=3, hwdownload" OUTPUT
> + at end example
> + at end itemize
> +
> + at section boxblur_opencl
> +
> +Apply a boxblur algorithm to the input video.
> +
> +It accepts the following parameters:
> +
> + at table @option
> +
> + at item luma_radius, lr
> + at item luma_power, lp
> + at item chroma_radius, cr
> + at item chroma_power, cp
> + at item alpha_radius, ar
> + at item alpha_power, ap
> +
> + at end table
> +
> +A description of the accepted options follows.
> +
> + at table @option
> + at item luma_radius, lr
> + at item chroma_radius, cr
> + at item alpha_radius, ar
> +Set an expression for the box radius in pixels used for blurring the
> +corresponding input plane.
> +
> +The radius value must be a non-negative number, and must not be
> +greater than the value of the expression @code{min(w,h)/2} for the
> +luma and alpha planes, and of @code{min(cw,ch)/2} for the chroma
> +planes.
> +
> +Default value for @option{luma_radius} is "2". If not specified,
> + at option{chroma_radius} and @option{alpha_radius} default to the
> +corresponding value set for @option{luma_radius}.
> +
> +The expressions can contain the following constants:
> + at table @option
> + at item w
> + at item h
> +The input width and height in pixels.
> +
> + at item cw
> + at item ch
> +The input chroma image width and height in pixels.
> +
> + at item hsub
> + at item vsub
> +The horizontal and vertical chroma subsample values. For example, for the
> +pixel format "yuv422p", @var{hsub} is 2 and @var{vsub} is 1.
> + at end table
> +
> + at item luma_power, lp
> + at item chroma_power, cp
> + at item alpha_power, ap
> +Specify how many times the boxblur filter is applied to the
> +corresponding plane.
> +
> +Default value for @option{luma_power} is 2. If not specified,
> + at option{chroma_power} and @option{alpha_power} default to the
> +corresponding value set for @option{luma_power}.
> +
> +A value of 0 will disable the effect.
> + at end table
> +
> + at subsection Examples
> +
> + at itemize
> + at item
> +Apply a boxblur filter with the luma, chroma, and alpha radius

Again, try to describe what the examples actually do rather than just the options provided.

> +set to 2:
> + at example
> +-i INPUT -vf "hwupload, boxblur_opencl=luma_radius=2:luma_power=1, hwdownload" OUTPUT
> +-i INPUT -vf "hwupload, boxblur_opencl=2:1, hwdownload" OUTPUT
> + at end example
> +
> + at item
> +Set the luma radius to 2, and alpha and chroma radius to 0:
> + at example
> +-i INPUT -vf "hwupload, boxblur_opencl=2:1:cr=0:ar=0, hwdownload" OUTPUT
> + at end example
> +
> + at item
> +Set the luma and chroma radius to a fraction of the video dimension:
> + at example
> +-i INPUT -vf "hwupload, boxblur_opencl=luma_radius=min(h\,w)/10:luma_power=1:chroma_radius=min(cw\,ch)/10:chroma_power=1, hwdownload" OUTPUT
> + at end example
> + at end itemize
> +
> + at section convolution_opencl
> +
> +Apply convolution of 3x3, 5x5, 7x7 or horizontal/vertical up to 49 elements.
> +
> +The filter accepts the following options:
> +
> + at table @option
> + at item 0m
> + at item 1m
> + at item 2m
> + at item 3m
> +Set matrix for each plane.
> +Matrix is sequence of 9, 25 or 49 signed integers in @var{square} mode,
> +and from 1 to 49 odd number of signed integers in @var{row} mode.
> +
> + at item 0rdiv
> + at item 1rdiv
> + at item 2rdiv
> + at item 3rdiv
> +Set multiplier for calculated value for each plane.
> +If unset or 0, it will be sum of all matrix elements.
> +
> + at item 0bias
> + at item 1bias
> + at item 2bias
> + at item 3bias
> +Set bias for each plane. This value is added to the result of the multiplication.
> +Useful for making the overall image brighter or darker. Default is 0.0.
> +
> + at item 0mode
> + at item 1mode
> + at item 2mode
> + at item 3mode
> +Set matrix mode for each plane. Can be @var{square}, @var{row} or @var{column}.
> +Default is @var{square}.
> + at end table
> +
> + at subsection Examples
> +
> + at itemize
> + at item
> +Apply sharpen:
> + at example
> +-i INPUT -vf "hwupload, convolution_opencl=0 -1 0 -1 5 -1 0 -1 0:0 -1 0 -1 5 -1 0 -1 0:0 -1 0 -1 5 -1 0 -1 0:0 -1 0 -1 5 -1 0 -1 0, hwdownload" OUTPUT
> + at end example
> +
> + at item
> +Apply blur:
> + at example
> +-i INPUT -vf "hwupload, convolution_opencl=1 1 1 1 1 1 1 1 1:1 1 1 1 1 1 1 1 1:1 1 1 1 1 1 1 1 1:1 1 1 1 1 1 1 1 1:1/9:1/9:1/9:1/9, hwdownload" OUTPUT
> + at end example
> +
> + at item
> +Apply edge enhance:
> + at example
> +-i INPUT -vf "hwupload, convolution_opencl=0 0 0 -1 1 0 0 0 0:0 0 0 -1 1 0 0 0 0:0 0 0 -1 1 0 0 0 0:0 0 0 -1 1 0 0 0 0:5:1:1:1:0:128:128:128, hwdownload" OUTPUT
> + at end example
> +
> + at item
> +Apply edge detect:
> + at example
> +-i INPUT -vf "hwupload, convolution_opencl=0 1 0 1 -4 1 0 1 0:0 1 0 1 -4 1 0 1 0:0 1 0 1 -4 1 0 1 0:0 1 0 1 -4 1 0 1 0:5:5:5:1:0:128:128:128, hwdownload" OUTPUT
> + at end example
> +
> + at item
> +Apply laplacian edge detector which includes diagonals:
> + at example
> +-i INPUT -vf "hwupload, convolution_opencl=1 1 1 1 -8 1 1 1 1:1 1 1 1 -8 1 1 1 1:1 1 1 1 -8 1 1 1 1:1 1 1 1 -8 1 1 1 1:5:5:5:1:0:128:128:0, hwdownload" OUTPUT
> + at end example
> +
> + at item
> +Apply emboss:
> + at example
> +-i INPUT -vf "hwupload, convolution_opencl=-2 -1 0 -1 1 1 0 1 2:-2 -1 0 -1 1 1 0 1 2:-2 -1 0 -1 1 1 0 1 2:-2 -1 0 -1 1 1 0 1 2, hwdownload" OUTPUT
> + at end example
> + at end itemize
> +
> + at section overlay_opencl
> +
> +Overlay one video on top of another.
> +
> +It takes two inputs and has one output. The first input is the "main"
> +video on which the second input is overlaid.
> +
> +The filter accepts the following options:
> +
> + at table @option
> +
> + at item x
> +Set the x coordinate of the overlaid video on the main video
> +By default value is @code{0}.
> +
> + at item y
> +Set the x coordinate of the overlaid video on the main video
> +By default value is @code{0}.
> +
> + at end table
> +
> + at subsection Example
> +
> + at itemize
> + at item
> +Insert a transparent PNG logo in the bottom left corner of the input
> + at example
> +-i INPUT -i LOGO -filter_complex "[0:v]hwupload[a],[1:v]hwupload[b],[a][b]overlay_opencl[out],[out]hwdownload" OUTPUT
> + at end example
> + at end itemize

There is quite a bit of trickiness wrt formats here - if you have a PNG logo and an RGB input this isn't going to do what you expect.  (Try it!)

That also doesn't end up in the bottom left unless the PNG is padded with transparancy at the top to the same height as the video.

> + at section prewitt_opencl
> +
> +Apply prewitt operator to input video stream.
> +
> +The filter accepts the following option:
> +
> + at table @option
> + at item planes
> +Set which planes will be processed, unprocessed planes will be copied.
> +By default value @code{0xf}, all planes will be processed.
> +
> + at item scale
> +Set value which will be multiplied with filtered result.
> +By default value is @code{1}.
> +
> + at item delta
> +Set value which will be added to filtered result.
> +By default value is @code{0}.
> + at end table
> +
> + at subsection Example
> +
> + at itemize
> + at item
> +Apply prewitt operator with scale set to 2 and delta set to 10
> + at example
> +-i INPUT -vf "hwupload, prewitt_opencl=scale=2:delta=10, hwdownload" OUTPUT
> + at end example
> + at end itemize
> +
> ...
> +
> + at end itemize
> +
> + at section roberts_opencl
> +Apply roberts cross operator to input video stream.
> +
> +The filter accepts the following option:
> +
> + at table @option
> + at item planes
> +Set which planes will be processed, unprocessed planes will be copied.
> +By default value @code{0xf}, all planes will be processed.
> +
> + at item scale
> +Set value which will be multiplied with filtered result.
> +By default value is @code{1}.
> +
> + at item delta
> +Set value which will be added to filtered result.
> +By default value is @code{0}.
> + at end table
> +
> + at subsection Example
> +
> + at itemize
> + at item
> +Apply roberts cross operator with scale set to 2 and delta set to 10
> + at example
> +-i INPUT -vf "hwupload, roberts_opencl=scale=2:delta=10, hwdownload" OUTPUT
> + at end example
> + at end itemize
> +
> + at section sobel_opencl
> +
> +Apply sobel operator to input video stream.
> +
> +The filter accepts the following option:
> +
> + at table @option
> + at item planes
> +Set which planes will be processed, unprocessed planes will be copied.
> +By default value @code{0xf}, all planes will be processed.
> +
> + at item scale
> +Set value which will be multiplied with filtered result.
> +By default value is @code{1}.
> +
> + at item delta
> +Set value which will be added to filtered result.
> +By default value is @code{0}.
> + at end table
> +
> + at subsection Example
> +
> + at itemize
> + at item
> +Apply sobel operator with scale set to 2 and delta set to 10
> + at example
> +-i INPUT -vf "hwupload, sobel_opencl=scale=2:delta=10, hwdownload" OUTPUT
> + at end example
> + at end itemize
> +
> + at section tonemap_opencl
> +Tone map colors from different dynamic ranges.
> +
> + at subsection Options
> +The filter accepts the following options.
> +
> + at table @option
> + at item tonemap
> +Set the tone map algorithm to use.
> +
> +Possible values are:
> + at table @var
> + at item none
> +Do not apply any tone map, only desaturate overbright pixels.
> +
> + at item clip
> +Hard-clip any out-of-range values. Use it for perfect color accuracy for
> +in-range values, while distorting out-of-range values.
> +
> + at item linear
> +Stretch the entire reference gamut to a linear multiple of the display.
> +
> + at item gamma
> +Fit a logarithmic transfer between the tone curves.
> +
> + at item reinhard
> +Preserve overall image brightness with a simple curve, using nonlinear
> +contrast, which results in flattening details and degrading color accuracy.
> +
> + at item hable
> +Preserve both dark and bright details better than @var{reinhard}, at the cost
> +of slightly darkening everything. Use it when detail preservation is more
> +important than color and brightness accuracy.
> +
> + at item mobius
> +Smoothly map out-of-range values, while retaining contrast and colors for
> +in-range material as much as possible. Use it when color accuracy is more
> +important than detail preservation.
> + at end table
> +
> +Default is none.
> +
> + at item param
> +Tune the tone mapping algorithm.
> +
> +This affects the following algorithms:
> + at table @var
> + at item none
> +Ignored.
> +
> + at item linear
> +Specifies the scale factor to use while stretching.
> +Default to 1.0.
> +
> + at item gamma
> +Specifies the exponent of the function.
> +Default to 1.8.
> +
> + at item clip
> +Specify an extra linear coefficient to multiply into the signal before clipping.
> +Default to 1.0.
> +
> + at item reinhard
> +Specify the local contrast coefficient at the display peak.
> +Default to 0.5, which means that in-gamut values will be about half as bright
> +as when clipping.
> +
> + at item hable
> +Ignored.
> +
> + at item mobius
> +Specify the transition point from linear to mobius transform. Every value
> +below this point is guaranteed to be mapped 1:1. The higher the value, the
> +more accurate the result will be, at the cost of losing bright details.
> +Default to 0.3, which due to the steep initial slope still preserves in-range
> +colors fairly accurately.
> + at end table
> +
> + at item desat
> +Apply desaturation for highlights that exceed this level of brightness. The
> +higher the parameter, the more color information will be preserved. This
> +setting helps prevent unnaturally blown-out colors for super-highlights, by
> +(smoothly) turning into white instead. This makes images feel more natural,
> +at the cost of reducing information about out-of-range colors.
> +
> +The default of 2.0 is somewhat conservative and will mostly just apply to
> +skies or directly sunlit surfaces. A setting of 0.0 disables this option.
> +
> +This option works only if the input frame has a supported color tag.
> +
> + at item peak
> +Override signal/nominal/reference peak with this value. Useful when the
> +embedded peak information in display metadata is not reliable or when tone
> +mapping from a lower range to a higher range.
> + at end table

I don't think tonemap and tonemap_opencl are similar enough to just copy this.

> + at section unsharp_opencl
> +
> +Sharpen or blur the input video.
> +
> +It accepts the following parameters:
> +
> + at table @option
> + at item luma_msize_x, lx
> +Set the luma matrix horizontal size. It must be an odd integer between
> + at code{3} and @code{23}. The default value is @code{5}.
> +
> + at item luma_msize_y, ly
> +Set the luma matrix vertical size. It must be an odd integer between @code{3}
> +and @code{23}. The default value is @code{5}.
> +
> + at item luma_amount, la
> +Set the luma effect strength. It must be a floating point number, reasonable
> +values lay between @code{-1.5} and @code{1.5}.
> +
> +Negative values will blur the input video, while positive values will
> +sharpen it, a value of zero will disable the effect.
> +
> +Default value is @code{1.0}.
> +
> + at item chroma_msize_x, cx
> +Set the chroma matrix horizontal size. It must be an odd integer
> +between @code{3} and @code{23}. The default value is @code{5}.
> +
> + at item chroma_msize_y, cy
> +Set the chroma matrix vertical size. It must be an odd integer
> +between @code{3} and @code{23}. The default value is @code{5}.
> +
> + at item chroma_amount, ca
> +Set the chroma effect strength. It must be a floating point number, reasonable
> +values lay between @code{-1.5} and @code{1.5}.
> +
> +Negative values will blur the input video, while positive values will
> +sharpen it, a value of zero will disable the effect.
> +
> +Default value is @code{0.0}.
> +
> + at end table
> +
> +All parameters are optional and default to the equivalent of the
> +string '5:5:1.0:5:5:0.0'.
> +
> + at subsection Examples
> +
> + at itemize
> + at item
> +Apply strong luma sharpen effect:
> + at example
> +-i INPUT -vf "hwupload, unsharp_opencl=luma_msize_x=7:luma_msize_y=7:luma_amount=2.5, hwdownload" OUTPUT
> + at end example
> +
> + at item
> +Apply a strong blur of both luma and chroma parameters:
> + at example
> +-i INPUT -vf "hwupload, unsharp_opencl=7:7:-2:7:7:-2, hwdownload" OUTPUT
> + at end example
> + at end itemize
> +
> + at c man end OPENCL VIDEO FILTERS
> +
>  @chapter Video Sources
>  @c man begin VIDEO SOURCES
>  
> 

- Mark