[Libav-user] vaapi decoding, av_hwframe_transfer_data

[Mark Thompson]

On 20/05/16 11:27, Nikita Orlov wrote:
> Hello!
> 
> I am decoding using last ffmpeg with vaapi support (libva for intel).
> 
> For decoding I am handling my pool of VASurfaces, and set pointers for my own get_frame, get_buffer and release_buffer for AVCodecContext.
> So, when I am try to transfer hw decoded frame to memory, using  av_hwframe_transfer_data it assumes, that src AVFrame will have pointer to hw_frames_ctx,
> but mine is null, so segfault.
> 
> During encoding, I am creating AVHWDeviceContext, AVVAAPIDeviceContext and AVHWFramesContext, so ffmpeg inits VASurfaces by its own.
> 
> Question is, how to use it for decoding (h.264)? How to set proper get_frame, get_buffer and release_buffer for AVCodecContext?

The frame has to come from a buffer pool associated with an AVHWFramesContext: as you observe, it falls over if this requirement is violated.

If you are happy for libav* to manage the surfaces for you, then you can use code like that in ffmpeg_vaapi.c to set up the buffer pool.

If you want to manage the surfaces yourself, then you will need to make your own AVBufferPool containing the surfaces and associate it with the AVHWFramesContext.  The documentation for this is in the doxygen - see <https://www.ffmpeg.org/doxygen/trunk/hwcontext_8h.html>, though it might be clearer just to read the libavutil/hwcontext.h headers directly.

- Mark