[FFmpeg-devel] [PATCH] libavfilter: scale_cuda filter adds dynamic command values

[Timo Rothenpieler]

On 26.11.2018 19:09, msanders wrote:
> Hi,
> 
> This patch adds command support for dynamic change the size in the “scale_cuda” resize filter. In fact, it’s the first GPU filter accepting realtime commands. Using similar changes it’s possible to port it to other hwaccelerators. The only limitation is that the values cannot exceed the initial values. It is therefore necessary to set up the graph with the higher values you may need.
> 
> One example: { -filter_complex "scale_cuda=720:576,hwdownload,format=nv12,zmq" }
> And then you can use the <c> or ZMQ messages to change the width and/or height.
> 
> Warning: This patch requires, to have sense, to apply too this other patch that fixes the hwdownload filter.
> https://patchwork.ffmpeg.org/patch/11165/
> 

I'm not sure if this is such a good idea. There's a lot of places all 
over the codebase that have a hardcoded assumption about how hardware 
frames in general, and CUDA frames in particular work.

A lot of code checks the width/height from the hwframes ctx instead of 
the frame itself, because it needs the real size(1088 for 1080p for 
example) of the underlying buffer at all times.
So those consumers would straight up ignore any scaling done to the 
frames, reading messed up data instead.

On top of that, in the specific case of CUDA, the CUDA pix_fmt is 
defined with the assumption that the entire frame is in a single 
continuous buffer with all planes right after one another. This would 
most prominently affect nvenc, as its API only takes one lone CUdevptr 
as input, and then has a fixed idea about how the data behind it looks.

So it would produce output with random data to the right and bottom of 
the scaled frame, still with the outer dimensions before the 
re-configuration.

The only way this could possibly work is if a new hw_frames_ctx is 
created on reconfiguration.
With nvenc this would actually work without any changes, as it re-reads 
the width/height out of it on every frame already, and initially only 
gets the CUDA-Context and sw_pix_fmt out of it, so those would need to 
stay the same, which isn't an issue.
But for a bunch of other hardware filters more work is needed. Specially 
as some parts overlap with other APIs.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4538 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20181126/b4f285cc/attachment.bin>