[FFmpeg-devel] [PATCH] lavfi/vf_vpp_qsv: fix the infinite loop while framerate lower than input

[Fu, Linjie]

> From: Zhong Li <lizhong1008 at gmail.com>
> Sent: Sunday, April 19, 2020 23:00
> To: Fu, Linjie <linjie.fu at intel.com>
> Cc: FFmpeg development discussions and patches <ffmpeg-
> devel at ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH] lavfi/vf_vpp_qsv: fix the infinite loop
> while framerate lower than input
> 
> Fu, Linjie <linjie.fu at intel.com> 于2020年2月29日周六 下午5:35写道：
> >
> > > -----Original Message-----
> > > From: Zhong Li <lizhong1008 at gmail.com>
> > > Sent: Saturday, February 29, 2020 13:14
> > > To: FFmpeg development discussions and patches <ffmpeg-
> > > devel at ffmpeg.org>
> > > Cc: Fu, Linjie <linjie.fu at intel.com>
> > > Subject: Re: [FFmpeg-devel] [PATCH] lavfi/vf_vpp_qsv: fix the infinite
> loop
> > > while framerate lower than input
> > >
> > > Linjie Fu <linjie.fu at intel.com> 于2020年2月28日周五 下午11:34写
> 道：
> > > >
> > > > There are frame droppings in frc while converting into a lower
> framerate,
> > > > and MSDK returns ERROR_MORE_DATA which should be ignored.
> > >
> > > Should be fixed in MSDK instead of working around in FFmpeg?
> >
> > MSDK made decision regarding frame rate conversion. If it's the framerate
> down case,
> > FRC would skip frame without producing an output [1], and request a new
> input frame.
> 
> I can't see the benefit to use MSDK framerate conversion function. Is
> it a good idea to drop it and use ffmpeg native fps filter instead?

The implementation of FRC inside MSDK is quite straight-forward or simple
currently since it just drops or repeats frames, hence I think using native fps
filter is a good idea for decoding + FRC or FRC + encoding.

However, for a pure hardware transcoding pipeline, there may be some
performance issues if inserting a software filter, extra memory copy would
be introduced in hwdownload/hwupload between system memory and video
memory, which would impact a lot for large resolutions.

Took a simple tests for 4K 10bit HEVC transcoding:

# inserting a fps filter in the transcoding: 20 fps
$ ffmpeg -hwaccel qsv -c:v hevc_qsv -i ../Exodus_UHD_HDR_Exodus_draft.mp4 -vf "hwdownload,format=p010le,fps=fps=60,hwupload=extra_hw_frames=40" -c:v hevc_qsv out.mp4

# using msdk framerate conversion: 33 fps
$ ffmpeg -hwaccel qsv -c:v hevc_qsv -i ../Exodus_UHD_HDR_Exodus_draft.mp4 -vf "vpp_qsv=framerate=60" -c:v hevc_qsv out.mp4

- Linjie