[Libav-user] Problem with h264_videotoolbox and AV_PIX_FMT_YUV420P

[Steve Green]

Just wanted to follow up on this issue.  It seems that aligning to 32 bytes with av_image_fill_arrays will work-around/resolve the problem.  That said, I couldn’t find any documentation in either VideoToolbox or Core Graphics that would explain this.

> On Feb 17, 2017, at 6:44 PM, Richard Kern <kernrj at gmail.com> wrote:
> 
> 
> 
> 
> On February 17, 2017 at 5:18:09 PM, Steve Green (steve.green at flosports.tv <mailto:steve.green at flosports.tv>) wrote:
> 
>> I assume you want to see screenshots?  These are produced after the long journey to a Wowza server and back out the other side in jwplayer.  I could capture the raw YUV and a key frame of H264 if that would help.
> 
> No need, I see the issue.
> 
>> 
>> 
>> Attached is the original picture, and the results of supplying 1-byte aligned and 16-byte aligned YUV to avcodec_encode_video2().
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>>> On Feb 17, 2017, at 5:02 PM, Richard Kern <kernrj at gmail.com <mailto:kernrj at gmail.com>> wrote:
>>> 
>>> On February 17, 2017 at 4:53:03 PM, Steve Green (steve.green at flosports.tv <mailto:steve.green at flosports.tv>) wrote:
>>>> This is the basic pipeline that I have:
>>>> 
>>>> BGRA frames -> sws_scale() -> YUV frames -> avcodec_encode_video2() -> av_interleaved_write_frame()
>>>> 
>>>> .. and this works perfectly with x264 and most of the time with h264_videotoolbox. The issue with h264_videotoolbox happens when I try to stream at 854x480. From looking at the output, it appears that someone along the way is not happy with width not being a multiple of 16. There is a very clear shift/wrapping of pixels.  
>>>> 
>>>> As an experiment, I changed my use of av_image_fill_arrays() to align on 16 bytes (with a sufficiently larger buffer) and the resulting image is improved but still not correct. The basic shift is gone and the image is legible but the colors are off and I can see an artifact over the image.
>>> 
>>> Can you post a frame grab of the unaligned and 16-byte aligned output?
>>> 
>>>> 
>>>> 
>>>> Given that x264 works and h264_videotoolbox doesn't, I’m tempted to point fingers at either VTCompressionSession or CVPixelBuffers + YUV, which would explain why I want to try AV_PIX_FMT_VIDEOTOOLBOX. I’ve read pretty much everything I can find on this but I cant find a single mention of 16-byte-multiple widths or strides.
>>> 
>>>> 
>>>> 
>>>> Any idea how to go about debugging this?
>>>> _______________________________________________
>>>> Libav-user mailing list
>>>> Libav-user at ffmpeg.org <mailto:Libav-user at ffmpeg.org>
>>>> http://ffmpeg.org/mailman/listinfo/libav-user <http://ffmpeg.org/mailman/listinfo/libav-user>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20170406/a0ccbd4c/attachment.html>