[FFmpeg-cvslog] lavc/videotoolbox: fix H.264 hwaccel init issue

[Richard Kern]

> On Jun 17, 2016, at 10:36 AM, Clément Bœsch <u at pkh.me> wrote:
> 
> On Fri, Jun 17, 2016 at 09:43:09AM -0400, Richard Kern wrote:
>> Since ca2f19b9cc (h264: switch to h2645_parse for NAL parsing) the hwaccel wouldn’t init (when the first h.264 frame is decoded). It was the same on OS X and iOS - creating the decoder would return a “decoder malfunction” error. Apple’s suggested way to create the decoder config for h.264 decoders is CMVideoFormatDescriptionCreateFromH264ParameterSets(), and switching to this function solved the issue on OS X and iOS. A user on IRC confirmed it worked for them too.
>> 
> 
> So avc_info constructed from extradata in decoder_config_create() wasn't
> enough? (Any idea why?)

I don’t know why. Unfortunately there isn’t much documentation, and the error message is vague.

> 
> "the hwaccel wouldn't init", why? Is it because the SPS/PPS NALs were
> somehow split out of the packets since ca2f19b9cc and thus VT never got
> them, or I misunderstand?

The same SPS and PPS from the H264Context are being used to create the avcC atom as are passed to CMVideoFormatDescriptionCreateFromH264ParameterSets()… I’ll need to do some digging in the commit that broke it this weekend.

> 
> -- 
> Clément B.