[FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters

[Xiang, Haihao]

On Mon, 2021-08-30 at 05:52 +0000, Soft Works wrote:
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > Xiang, Haihao
> > Sent: Monday, 30 August 2021 06:20
> > To: ffmpeg-devel at ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters
> > 
> > On Thu, 2021-08-05 at 16:21 +0000, Soft Works wrote:
> > > > -----Original Message-----
> > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > > > Haihao Xiang
> > > > Sent: Thursday, 5 August 2021 10:19
> > > > To: ffmpeg-devel at ffmpeg.org
> > > > Cc: Haihao Xiang <haihao.xiang at intel.com>
> > > > Subject: [FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters
> > > > 
> > > > This patchset clean up scale_qsv and deinterlace_qsv filters, and
> > 
> > take the
> > > > two filters as the special cases of vpp_qsv, so vf_scale_qsv.c
> > 
> > and
> > > > vf_deinterlace_qsv.c can be deleted from FFmpeg. In addition, a
> > 
> > few small
> > > > features are added in this patchset.
> > > > ---
> > > > v5:
> > > > * Rebased this patchset against the latest master branch and
> > 
> > fixed conflicts
> > > > 
> > > > Haihao Xiang (20):
> > > >   lavfi/qsv: use QSVVPPContext as base context in
> > > >     vf_vpp_qsv/vf_overlay_qsv
> > > >   lavfi/scale_qsv: simplify scale_qsv filter
> > > >   lavfi/scale_qsv: don't need variables for constants in FFmpeg
> > > >   lavfi/vpp_qsv: add "a", "dar" and "sar" variables
> > > >   lavfi/vpp_qsv: handle NULL pointer when evaluating an
> > 
> > expression
> > > >   lavfi/vpp_qsv: allow special values for the output dimensions
> > > >   lavfi/vpp_qsv: factorize extra MFX configuration
> > > >   lavfi/vpp_qsv: allow user to set scale_mode with constant
> > > >   lavfi/vpp_qsv: add vpp_preinit callback
> > > >   lavfi/scale_qsv: re-use VPPContext for scale_qsv filter
> > > >   lavfi/vpp_qsv: factor common QSV filter definition
> > > >   lavfi/scale_qsv: add new options for scale_qsv filter
> > > >   lavfi/scale_qsv: add more input / output pixel formats
> > > >   lavfi/qsvvpp: avoid overriding the returned value
> > > >   lavfi/qsvvpp: set PTS for output frame
> > > >   lavfi/vpp_qsv: check output format string against NULL pointer
> > > >   lavfi/deinterlace_qsv: simplify deinterlace_qsv filter
> > > >   lavfi/deinterlace_qsv: re-use VPPContext for deinterlace_qsv
> > 
> > filter
> > > >   lavfi/deinterlace_qsv: add async_depth option
> > > >   lavfi/deinterlace_qsv: add more input / output pixel formats
> > > > 
> > > >  libavfilter/Makefile             |   4 +-
> > > >  libavfilter/qsvvpp.c             |  57 ++-
> > > >  libavfilter/qsvvpp.h             |  11 +-
> > > >  libavfilter/vf_deinterlace_qsv.c | 611 -------------------------
> > 
> > --
> > > >  libavfilter/vf_overlay_qsv.c     |  11 +-
> > > >  libavfilter/vf_scale_qsv.c       | 685 -------------------------
> > 
> > ------
> > > >  libavfilter/vf_vpp_qsv.c         | 473 +++++++++++++--------
> > > >  7 files changed, 347 insertions(+), 1505 deletions(-)  delete
> > 
> > mode 100644
> > > > libavfilter/vf_deinterlace_qsv.c  delete mode 100644
> > > > libavfilter/vf_scale_qsv.c
> > > > 
> > > > --
> > > > 2.17.1
> > > 
> > > Hi Hihao,
> > > 
> > > The general idea of this patch makes sense to me.
> > > 
> > > Currently there are implementation differences between these
> > 
> > filters,
> > > and there are cases where vpp_qsv  doesn't work and I need to use
> > > scale_qsv instead.
> > > 
> > > I have never analyzed the actual reason, but this should be done
> > > before replacing scale_qsv with an aliased vpp_qsv.
> > > 
> > > I'll try to dig out an example..
> > 
> > 
> > Hi Softworkz,
> > 
> > Could you provide the cases when you have time ? I may look into the
> > issues.
> > 
> > Thanks
> > Haihao
> 
> 
> Hi Haihao,
> 
> IIRC think it can be reproduced easily by using the filters without
> parameters,
> where scale_qsv works and vpp_qsv doesn't work.

scale_qsv only supports video memory input and output however vpp_qsv may
support both video memory and system memory, there is a case where scale_qsv
doesn't work but vpp_qsv may work. 

$> ffmpeg -c:v h264_qsv -i input.mp4 -vf scale_qsv -f null -

Impossible to convert between the formats supported by the filter 'graph 0 input
from stream 0:0' and the filter 'auto_scale_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:0
Conversion failed!

After applying this patchset, scale_qsv may work with both video memory and
system memory.

In addition, vpp_qsv may support crop however scale_qsv doesn't support crop.
After applying  this patchset, scale_qsv may support crop too. 

> 
> If it doesn't reproduce, try this in combination with hwupload, e.g.
> 
> - "hwupload at f1=extra_hw_frames=32,vpp_qsv"
> vs.
> - "hwupload at f1=extra_hw_frames=32,scale_qsv"
> 

The above two combinations work for me

Thanks
Haihao