[FFmpeg-devel] have some major changes for nvenc support

[Andrey Turkin]

In my opinion this proliferation of various filters which do the same thing
in different way is a configuration headache. There's CPU filters: one for
scaling/format conversion, one for padding, one  for cropping, like 5
different filters for deinterlacing. And now there would be nvresize for
scaling, and we gotta add CUDA-based nvpad and nvdeint if we want to do
some transcoding. Maybe add overlaying too - so nvoverlay. Then there is
OpenCL which can do everything - so 4 more filters for that. And there is
quicksync which I think can do those things, so there would be qsvscale and
qsvdeint. And there is D3D11 video processor which can do those things too
(simultaneously in fact), so there's gotta be
d3d11vascaledeintpadcropoverlay. And then we've got a whole bunch of video
filters which can only do their job on a specific hwaccel platform. Want to
try different hwaccel? Rewrite damn filter string. Want to do something
generic that can be used over different platforms? Can't be done.
Maybe it's just my wishful thinking, but I was wondering for some time if
there can be one "smart" filter to do one specific thing? Say, single
deinterlace filter which can automatically use whichever hwaccel was used
on its input (or whichever will be used on its output)? We've already got
pixel formats which describe particular hwaccel - can't filters decide
which code path to use based on that? And it can have a configuration
option to choose which CPU-based fallback to use (in fact that option can
be used to tweak GPU-based algorithm for platforms which support it).
Same goes for encoders - can't there be "just" h264 encoder? This one's
tough though - you might have dxva decoder, cuda filters and nvenc encoder
and you probably want to keep them all on the same GPU. Or not - maybe you
do want to decode on qsv and encode on nvenc, or maybe vice versa. Probably
single-GPU case is more common so it should try to use same GPU what was
used for decoding and/or filtering, and allow to override with encoder
options (like nvenc allows to specify cuda device to use).
Interop will be a pain though (obviously we'd want to avoid device-host
frame transfers). I'm trying to share d3d11va-decoded frames (nv12 texture)
with OpenCL or CUDA right now,and I've been at it for days with no luck
whatsoever. My last resort now is to write a shader to "draw" (in fact just
copy pixels around) nv12 texture onto another texture in more "common"
format which can be used by OpenCL... There's got to be some easier way
(other than using cuvid to decode  the video), right?

Regards, Andrey

2016-01-08 11:29 GMT+03:00 Roger Pack <rogerdpack2 at gmail.com>:

> On 11/5/15, wm4 <nfxjfg at googlemail.com> wrote:
> > On Thu, 5 Nov 2015 16:23:04 +0800
> > Agatha Hu <ahu at nvidia.com> wrote:
> >
> >> 2) We use AVFrame::opaque field to store a customized ffnvinfo struture
> >> to prevent expensive CPU<->GPU transferration. Without it, the workflow
> >> will be like CPU AVFrame input-->copy to GPU-->do CUDA resizing-->copy
> >> to CPU AVFrame-->copy to GPU-->NVENC encoding. And now it becomes:
> >> CPU AVFrame input-->copy to GPU-->do CUDA resizing-->NVENC encoding.
> >> Our strategy is to check whether AVFrame::opaque is not null AND its
> >> first 128 bytes matches some particular GUID. If so, AVFrame::opaque is
> >> a valid ffnvinfo struture and we read GPU address directly from it
> >> instead of copying data from AVFrame.
> >
> > Please no, not another hack that makes the hw decoding API situation
> > worse. Do this properly and coordinate with Gwenole Beauchesne, who
> > plans improvements into this direction.
>
> Which part are you referring to (though I'll admit putting some stuff
> in libavutil seems a bit suspect).
> It would be nice to have the nvresize filter available anyway, and it
> looks like it mostly just deals with private context variables.
> Cheers!
> -roger-
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>