[Libav-user] Help with unpacking PIX_FMT_YUV4* data types

[wm4]

On Tue, 11 Mar 2014 17:55:34 -0700
Ricky Huang <rhuang.work at gmail.com> wrote:

> 
> On Mar 11, 2014, at 5:03 PM, wm4 <nfxjfg at googlemail.com> wrote:
> 
> > On Mon, 10 Mar 2014 18:32:54 -0700
> > Ricky Huang <rhuang.work at gmail.com> wrote:
> > 
> >>>>> AVFilterBufferRef *cur_pic = link->cur_buf;
> >>>>> uint8_t *data = cur_pic->data[0];
> >>> 
> >>> I think this functionality was removed long ago. You're supposed to
> >>> write/read AVFrames from the filters.
> >> 
> >> Due to internal reasons, we are running ffmpeg 0.7, so the primary arguments to draw_slice() is AVFilterLink*.  Sorry I should have specified that when asked the question.
> > 
> > Sure is ancient...
> > 
> >> Or maybe I am missing something obvious here?  Can you help me out with how to use AVFrame to extract the luminance component?
> > 
> > AVFrame works similar to AVFilterBufferRef. Assuming the data member
> > contains the plane pointers, data[0] will contain the plane with the
> > luminance data.
> 
> Thank you for responding, wm4.  Let's use this from pixfmt.h header file as an example:
> 
>     PIX_FMT_YUV420P,   ///< planar YUV 4:2:0, 12bpp, (1 Cr & Cb sample per 2x2 Y samples)
> 
> Does that mean I should read in the luminance plane 2 bytes at a time and interpret it as a number together?  The reason I ask that is because the linesize of data[0] is twice as wide as data[1] or data[2] (720 vs 368 and 368, respectively).  Also the data point is declared as "uint8_t *", how does one determine how to interpret the type of the value - int, float, etc?

This format uses 1 byte per sample on each plane, so basically you'd
access it as uint8_t. The linesizes of the two chroma planes are
smaller, because YUV420P is a sub-sampled format, and, as the comment
on the pixel format says, uses 1 chroma sample per 2x2 luminance
sample. (Ignore the the mention of "12bpp", this is an average and is
confusing at best.)

So I guess you'll have to read up on how planar YUV formats work. Or
just convert to RGB. (Using RGB might be more correct in some ways,
because you don't have to deal with different colorspaces and color
ranges etc..)