[FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV encoder encode VAAPI and D3D11 frames directly

[Xiang, Haihao]

On Wed, 2022-06-08 at 11:13 +0000, Soft Works wrote:
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Xiang,
> > Haihao
> > Sent: Wednesday, June 8, 2022 10:42 AM
> > To: ffmpeg-devel at ffmpeg.org
> > Cc: Wu, Tong1 <tong1.wu at intel.com>; Chen, Wenbin <wenbin.chen at intel.com>
> > Subject: Re: [FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV encoder encode
> > VAAPI and D3D11 frames directly
> > 
> > On Wed, 2022-06-08 at 05:08 +0000, Soft Works wrote:
> > > > -----Original Message-----
> > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Tong
> > > > Wu
> > > > Sent: Tuesday, June 7, 2022 11:22 AM
> > > > To: ffmpeg-devel at ffmpeg.org
> > > > Cc: Tong Wu <tong1.wu at intel.com>; Wenbin Chen <wenbin.chen at intel.com>
> > > > Subject: [FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV encoder encode
> > > > VAAPI
> > > > and D3D11 frames directly
> 
> [..]
> 
> > > > 2.35.1.windows.2
> > > 
> > > Hi,
> > > 
> > > thanks for submitting this patch. Though, I'm afraid, but this
> > > 
> > > - fundamentally contradicts the logic of ffmpeg's handling of hw
> > 
> > acceleration,
> > >   hw device and hw frames contexts
> > > - adds code to an encoder, doing things an encoder is not supposed to do-
> > 
> > qsv
> > > encoders and decoders have their own context => QSV
> > 
> > nvdec and nvenc have CUDA but nvenc can also support D3D11va, it sounds make
> > sense for me to support D3D11va/vaapi in qsvenc too as d3d11va/vaapi are
> > used
> > internally in MediaSDK.
> 
> Can you please post a command line showing nvenc working with input
> from a D3D11VA decoder and without using any hwmap/hwupload/hwdownload
> filters?
> 

According to the code below, nvenc may accept d3d11 frames directly,

https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/nvenc.c#L46-L72

so the command below should work

$> ffmpeg -y -hwaccel_output_format d3d11 -hwaccel d3d11va -i input.mp4 -c:v
hevc_nvenc out.mp4 

> 
> > 
> > > - is not safe/guaranteed to work always
> > >   there are different requirements for QSV than for other other cases
> > >   like VAAPI - for example: QSV requires a fixed-size frame pool
> > >   and encoders often need a larger frame pool than VAAPI
> > > 
> > 
> > Encoders in MediaSDK don't need a fixed pool, probably we may relax this
> > limitation in QSV.
> 
> Well - I think they do:
> 
> Common
> The QSV hw frames context implementation is using a fixed pool size and
> changing that would be anything but trivial. 
> 
> D3D11
> The decoders are using and allocating a D3D11 array textures which cannot 
> be resized. When the surface count is too low, QSV encoding will fail.
> 
> VAAPI
> See vaapi_frames_init(): there is support for dynamically sized pools, but
> not as render targets.
> 
> 
> The actual problem though is not about fixed vs dynamic pool size - it's
> about whether the number of frames will be sufficient for encoding.
> And when try to use the (d3d11va) decoder's output frames and supply them
> to the encoder, they may very likely be insufficient.
> 
> 
> QSV encoding always requires quite a number of surfaces.
> You might remember all the extra_hw_frames parameters in the sample
> command lines: 
> 
> https://trac.ffmpeg.org/wiki/Hardware/QuickSync
> 
> -------------
> 
> Did you ever test this with a D3D11VA decoder on Windows? 
> I'm not sure whether this can work at all (feeding decoder frames
> into the encoder) due to the differing bind_flags requirements.
> 
> In case you did test that - could you please post the command line?

vaapi on Linux:

$ ffmpeg -y -hwaccel vaapi -hwaccel_output_format vaapi -i input.mp4 -c:v
hevc_qsv out.mp4

d3d11va on Windows:

$ ffmpeg.exe -y -hwaccel d3d11va -hwaccel_output_format d3d11 -i input.mp4 -c:v
h264_qsv -y out.mp4


Thanks
Haihao