[Libav-user] Using Video Toolbox hardware decoding

[Info || Non-Lethal Applications]

Jean-Yves,

> On 03 Nov 2015, at 07:59, Jean-Yves Avenard <jyavenard at gmail.com> wrote:
> 
> Le lundi 2 novembre 2015, Info || Non-Lethal Applications <info at non-lethal-applications.com <mailto:info at non-lethal-applications.com>> a écrit :
> 
>> On 02 Nov 2015, at 11:22, Jean-Yves Avenard <jyavenard at gmail.com <>> wrote:
>> 
>> 
>> 
>> Le vendredi 23 octobre 2015, Isaksson Jörgen <jogga at bitfield.se <>> a écrit :
>> Hi all,
>> 
>> I’m new to this list so forgive me if this has been answered before.
>> 
>> With version 2.8 of the libraries I saw the addition of Video Toolbox hardware decoding support (on Mac OS X that is).
>> 
>> But I can’t find much documentation on how to use it. There are some clues in the header files on how to set it up, but there’s no docs on how to do the actual decoding.
>> 
>> I have compiled version 2.8.1 of the libraries and I can confirm that the Video Toolbox hwaccels are there:
>> 
>> Enabled hwaccels:
>> h263_videotoolbox       mpeg1_videotoolbox      mpeg4_videotoolbox
>> h264_videotoolbox       mpeg2_videotoolbox
>> 
>> In my player app I have setup an AVVideotoolboxContext like described in the headers.
>> 
>> if (self.codecContext->codec_id == AV_CODEC_ID_H264) {
>>         _videoToolboxContext = av_videotoolbox_alloc_context();
>>         int result = av_videotoolbox_default_init2(self.codecContext, self.videoToolboxContext);
>>         if (result != 0) {
>>             NSLog(@"Failed to init video toolbox");
>>             _videoToolboxContext = NULL;
>>         }
>>  }
>> 
>> _codec = avcodec_find_decoder(self.codecContext->codec_id);
>> 
>> Everything seems to work fine and the _videoToolboxContext is properly allocated.
>> 
>> But how do I go about actually decoding using it?
>> 
>> Anyone with any experience from this?
>> 
>> VideoToolbox is for decoding only, just like VDA 
>> 
>> From 10.9; to get HW decoding the only thing that needs to be set is a constant. And this is just a hint. Vat won't use HW for vertical resolution < 300 typically.
>> 
>> Note that ffmpeg doesn't enable async decoding, so the performance are poor, nowhere near as fast as what it could be.
> 
> And what is the constant that needs to be set and where?
> 
> kVTVideoDecoderSpecification_EnableHardwareAcceleratedVideoDecoder
> 
> Looking at the 2.8 ffmpeg source code, this constant is set. The decoder will use HW acceleration if available...
> Note that ffmpeg code will crash if used on OSX < 10.9 (where that constant was introduced)

Ok, thanks! Good to know!

> Do you know if it’s still worth using it?
> Will the decoding be faster than FFmpeg’s standard software decoding?
> 
> 
> Faster I don't know. Using less CPU certainly.
> 
> However, in my experience, the bottleneck will be in not using async decode.
> 
> There's a good 100% performance increase with that mode set !
> 
> Too bad it isn't set

I just grabbed the latest 1.8.1 source code and I don’t see any flag to toggle async decoding.
I only see that the function videotoolbox_session_decode_frame waits for the asynchronously decoded frames using VTDecompressionSessionWaitForAsynchronousFrames.
Is the waiting part what you mean?

Do you have any insight why this is not used for Pro Res decoding?

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20151103/3869b906/attachment.html>