[FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters

[Xiang, Haihao]

On Thu, 2021-09-09 at 14:56 +0000, Xiang, Haihao wrote:
> On Mon, 2021-08-30 at 08:04 +0000, Xiang, Haihao wrote:
> > On Mon, 2021-08-30 at 05:52 +0000, Soft Works wrote:
> > > > -----Original Message-----
> > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > > > Xiang, Haihao
> > > > Sent: Monday, 30 August 2021 06:20
> > > > To: ffmpeg-devel at ffmpeg.org
> > > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters
> > > > 
> > > > On Thu, 2021-08-05 at 16:21 +0000, Soft Works wrote:
> > > > > > -----Original Message-----
> > > > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > > > > > Haihao Xiang
> > > > > > Sent: Thursday, 5 August 2021 10:19
> > > > > > To: ffmpeg-devel at ffmpeg.org
> > > > > > Cc: Haihao Xiang <haihao.xiang at intel.com>
> > > > > > Subject: [FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters
> > > > > > 
> > > > > > This patchset clean up scale_qsv and deinterlace_qsv filters, and
> > > > 
> > > > take the
> > > > > > two filters as the special cases of vpp_qsv, so vf_scale_qsv.c
> > > > 
> > > > and
> > > > > > vf_deinterlace_qsv.c can be deleted from FFmpeg. In addition, a
> > > > 
> > > > few small
> > > > > > features are added in this patchset.
> > > > > > ---
> > > > > > v5:
> > > > > > * Rebased this patchset against the latest master branch and
> > > > 
> > > > fixed conflicts
> > > > > > 
> > > > > > Haihao Xiang (20):
> > > > > >   lavfi/qsv: use QSVVPPContext as base context in
> > > > > >     vf_vpp_qsv/vf_overlay_qsv
> > > > > >   lavfi/scale_qsv: simplify scale_qsv filter
> > > > > >   lavfi/scale_qsv: don't need variables for constants in FFmpeg
> > > > > >   lavfi/vpp_qsv: add "a", "dar" and "sar" variables
> > > > > >   lavfi/vpp_qsv: handle NULL pointer when evaluating an
> > > > 
> > > > expression
> > > > > >   lavfi/vpp_qsv: allow special values for the output dimensions
> > > > > >   lavfi/vpp_qsv: factorize extra MFX configuration
> > > > > >   lavfi/vpp_qsv: allow user to set scale_mode with constant
> > > > > >   lavfi/vpp_qsv: add vpp_preinit callback
> > > > > >   lavfi/scale_qsv: re-use VPPContext for scale_qsv filter
> > > > > >   lavfi/vpp_qsv: factor common QSV filter definition
> > > > > >   lavfi/scale_qsv: add new options for scale_qsv filter
> > > > > >   lavfi/scale_qsv: add more input / output pixel formats
> > > > > >   lavfi/qsvvpp: avoid overriding the returned value
> > > > > >   lavfi/qsvvpp: set PTS for output frame
> > > > > >   lavfi/vpp_qsv: check output format string against NULL pointer
> > > > > >   lavfi/deinterlace_qsv: simplify deinterlace_qsv filter
> > > > > >   lavfi/deinterlace_qsv: re-use VPPContext for deinterlace_qsv
> > > > 
> > > > filter
> > > > > >   lavfi/deinterlace_qsv: add async_depth option
> > > > > >   lavfi/deinterlace_qsv: add more input / output pixel formats
> > > > > > 
> > > > > >  libavfilter/Makefile             |   4 +-
> > > > > >  libavfilter/qsvvpp.c             |  57 ++-
> > > > > >  libavfilter/qsvvpp.h             |  11 +-
> > > > > >  libavfilter/vf_deinterlace_qsv.c | 611 -------------------------
> > > > 
> > > > --
> > > > > >  libavfilter/vf_overlay_qsv.c     |  11 +-
> > > > > >  libavfilter/vf_scale_qsv.c       | 685 -------------------------
> > > > 
> > > > ------
> > > > > >  libavfilter/vf_vpp_qsv.c         | 473 +++++++++++++--------
> > > > > >  7 files changed, 347 insertions(+), 1505 deletions(-)  delete
> > > > 
> > > > mode 100644
> > > > > > libavfilter/vf_deinterlace_qsv.c  delete mode 100644
> > > > > > libavfilter/vf_scale_qsv.c
> > > > > > 
> > > > > > --
> > > > > > 2.17.1
> > > > > 
> > > > > Hi Hihao,
> > > > > 
> > > > > The general idea of this patch makes sense to me.
> > > > > 
> > > > > Currently there are implementation differences between these
> > > > 
> > > > filters,
> > > > > and there are cases where vpp_qsv  doesn't work and I need to use
> > > > > scale_qsv instead.
> > > > > 
> > > > > I have never analyzed the actual reason, but this should be done
> > > > > before replacing scale_qsv with an aliased vpp_qsv.
> > > > > 
> > > > > I'll try to dig out an example..
> > > > 
> > > > 
> > > > Hi Softworkz,
> > > > 
> > > > Could you provide the cases when you have time ? I may look into the
> > > > issues.
> > > > 
> > > > Thanks
> > > > Haihao
> > > 
> > > 
> > > Hi Haihao,
> > > 
> > > IIRC think it can be reproduced easily by using the filters without
> > > parameters,
> > > where scale_qsv works and vpp_qsv doesn't work.
> > 
> > scale_qsv only supports video memory input and output however vpp_qsv may
> > support both video memory and system memory, there is a case where scale_qsv
> > doesn't work but vpp_qsv may work. 
> > 
> > $> ffmpeg -c:v h264_qsv -i input.mp4 -vf scale_qsv -f null -
> > 
> > Impossible to convert between the formats supported by the filter 'graph 0
> > input
> > from stream 0:0' and the filter 'auto_scale_0'
> > Error reinitializing filters!
> > Failed to inject frame into filter network: Function not implemented
> > Error while processing the decoded data for stream #0:0
> > Conversion failed!
> > 
> > After applying this patchset, scale_qsv may work with both video memory and
> > system memory.
> > 
> > In addition, vpp_qsv may support crop however scale_qsv doesn't support
> > crop.
> > After applying  this patchset, scale_qsv may support crop too. 
> > 
> > > 
> > > If it doesn't reproduce, try this in combination with hwupload, e.g.
> > > 
> > > - "hwupload at f1=extra_hw_frames=32,vpp_qsv"
> > > vs.
> > > - "hwupload at f1=extra_hw_frames=32,scale_qsv"
> > > 
> > 
> > The above two combinations work for me
> 
> Hi Softworkz,
> 
> I failed to reproduce your issue, could you provide more info if possible? 

Hi Softworkz,

I didn't reproduce your issue, maybe the issue has been fixed on master. How
about to merge this patchset first? I will rebase the patchset if no objection. 
 BTW we didn't see regression in our internal testing. 

Thanks
Haihao