[Libav-user] Using Video Toolbox hardware decoding

[wm4]

On Thu, 5 Nov 2015 22:38:14 +1300
Jean-Yves Avenard <jyavenard at gmail.com> wrote:

> On 3 November 2015 at 21:04, Info || Non-Lethal Applications
> <info at non-lethal-applications.com> wrote:
> 
> >
> > I just grabbed the latest 1.8.1 source code and I don’t see any flag to
> > toggle async decoding.
> > I only see that the function videotoolbox_session_decode_frame waits for the
> > asynchronously decoded frames using
> > VTDecompressionSessionWaitForAsynchronousFrames.
> > Is the waiting part what you mean?  
> 
> I have no experience in using VT via ffmpeg, I've only ever accessed
> VT directly (wrote the Firefox h264 decoder on mac)
> 
> The async flag (kVTDecodeFrame_EnableAsynchronousDecompression) is
> provided to VTDecompressionSessionDecodeFrame
> 
> In my personaly experience, not turning that flag, and instead waiting
> for the frame with VTDecompressionSessionWaitForAsynchronousFrames
> immediately leads to severe performance issue.
> To the point where you typically get better performance using the
> ffmpeg sofware decoder
> 
> This is even more true on mac not supporting HW decode (in particular
> late-2013 mac pro)
> 
> >
> > Do you have any insight why this is not used for Pro Res decoding?  
> 
> most likely because the FFmpeg avcodec_decode_video2 is synchronous.
> There's no straight forward mechanism to have callbacks returning
> frames at a later point.
> Not that it can't be done.

There are diffuse plans to introduce async decoding to FFmpeg (both
API, and Videotoolbox), but nothing concrete yet.