[Libav-user] Yet another noob converting rgb24 to yuv420

[Joshua Kordani]

Greetings all,

    This must be asked a million times, but I am trying to take rgb24 
and convert it to yuv420 for use with x264.  My code is failing on the 
sws_scale call with a bad access, inside ff_yuv2plane1_8_avx.loop_a, 
which suggests to me that I haven't given it the input data structures 
in the right way. I'm starting with packed data, that is, rgb pixel 
values every 24 bits, but it looks like the sws_scale call is 
generalized to assume that input and output data is planar, and that (I 
imagine) if the input (or output) is supposed to be non-planar, that it 
should be stored/can be found in the first plane of the relevant data 
structures

   Since x264 seems to provide for a data structure that fits this 
purpose (the img struct inside of a pic_t, properly initialized), I've 
used it as my dst and dstStride locations

    My code looks like this.

//the compiler complained that it couldn't locate these when I tried to 
call them directly in the getCachedContext call.  This is the result of 
a few rounds of debugging
     enum AVPixelFormat src_pix_fmt = AV_PIX_FMT_RGB24, dst_pix_fmt = 
AV_PIX_FMT_YUV420P;


     scale_context = sws_getCachedContext(scale_context,
                      frame_width,
                      frame_height,
                      src_pix_fmt,
                      frame_width,
                      frame_height,
                      dst_pix_fmt,
                      SWS_BICUBIC,
                      NULL,
                      NULL,
                      NULL
                      );

//where frame_width, frame_height is the size in pixels of my input 
frame (right?)

...snip...

       sws_scale(scale_context,
             &frame,
             &rowstride,
             0,
             x264_params->i_height,
             pic_in.img.plane,
             pic_in.img.i_stride
             );


where frame is a pointer to my packed data, so I pass in a reference to 
the pointer (so that, assuming that the input pixel format specifier 
assumes packed data, and that the packed data is in the [0] offset of 
the pointer provided to the function....) right?,
rowstride is 3 * frame_width stored in a local, same problem, the 
function call is expecting a pointer or reference to de-reference, but 
since this is packed data there is only one stride, right?

pic_in is a struct from the x264 library, with a convenient sub 
structure img, which seems to be purpose built for this kind of function 
(or vice versa), where i_stride is an array of strides, plane an array 
of planes, i_height was a convenient place for me to retrieve this value.

I have a feeling I'm abusing lots of things, I was trying to avoid doing 
this and using x264 support for encoding rgb directly, but apparently it 
was too bleeding edge and decoding it was becoming a problem.  I thought 
the plan b wouldn't be too hard to implement but I just don't seem to be 
understanding how to setup and use this code.

I'm starting from a codebase that did compile and run, attempting to 
encode the rgb directly.  I couldn't find a decoder that was able to 
decode what I was producing, it seemed to be related to the colorspace I 
was trying to decode to, so it became more apparent that I needed to do 
a conversion.

I would appreciate any pointers anyone may have.  I'm not afraid of 
reading, I just haven't been able to leverage the things I've found so far.

-- 
Joshua Kordani
LSA Autonomy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20130917/7671dbe2/attachment.html>