[Libav-user] Hardware decoding using DXVA2 does not work on AMD GPUs

[Nicolas Aguilera]

 Hello!

I'm trying to make a hardware decoder/renderer analogue to Moonlight's
DXVA2Renderer class
<https://github.com/moonlight-stream/moonlight-qt/blob/master/app/streaming/video/ffmpeg-renderers/dxva2.cpp>.
As the one from the Moonlight-qt project, it must provide a decoder service
to ffmpeg and then retrieve the decoded frame in order to present it to the
screen.
I have implemented it and it works with NVIDIA and Intel GPUs, but every
time I try to use an AMD GPU, it fails to decode a test frame (also fails
to decode any subsequent frame. The error message from ffmpeg says:
"avcodec send packet: Operation not permitted").

I get no errors while initializing ffmpeg nor DXVA2; the first error occurs
when trying to decode the test frame. I tested it on a laptop with an AMD
Ryzen 9 5900HX CPU with Radeon Graphics (integrated GPU), and an NVIDIA GTX
3060 dedicated GPU. Also I've made tests on a Ryzen 3000 series with
integrated graphics. The behaviour is exactly the same on both systems.

I know it's difficult to pinpoint a problem without looking at the code.
I'd like to know if anyone has already seen this behaviour before, or maybe
has a sense of what could be going on.

Any help is much appreciated!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ffmpeg.org/pipermail/libav-user/attachments/20230317/13bbdbe7/attachment.htm>