[Libav-user] Fill AVFrame from a Cairo surface

[Nicolas George]

Colossus (12021-08-16):
> I have this piece of code:
> static gboolean img_export_frame_to_avframe(img_window_struct *img,
> cairo_surface_t *surface)
> {
> gint width, height, stride, row, col, offset;
> uint8_t  *pix;
> 
> /* Image info and pixel data */
> width  = cairo_image_surface_get_width( surface );
> height = cairo_image_surface_get_height( surface );
> stride = cairo_image_surface_get_stride( surface );
> pix    = cairo_image_surface_get_data( surface );
> 
> g_print("Stride: %d\n",stride);
> 
> /* Avlibrary stuff */
> img->video_frame = av_frame_alloc();
> av_frame_make_writable(img->video_frame);
> img->video_frame->format = AV_PIX_FMT_RGBA;
> img->video_frame->width  = width;
> img->video_frame->height = height;
> img->video_frame->linesize[0] = stride;
> 
> av_frame_get_buffer(img->video_frame, 1);
> img->video_packet = av_packet_alloc();
> 
> for( row = 0; row < img->video_frame->height; row++ )
> {
> for( col = 0; col < img->video_frame->width; col++ )
> {
> offset = 3 * col + row * img->video_frame->linesize[0];
> img->video_frame->data[0][offset + 0] = pix[0];
> img->video_frame->data[0][offset + 1] = pix[1];
> img->video_frame->data[0][offset + 2] = pix[2];
> }
> }
> img_export_encode_av_frame(img->video_frame, img->video_format_context,
> img->codec_context, img->video_packet);
> return TRUE;
> }
> 
> But avcodec_send_frame() called from img_export_encode_av_frame() always
> return:
> 
> libx264 @ 0x5582519787c0] Input picture width (640) is greater than stride
> (0)
> 
> The stride is 5120 not 0! Where am I wrong? Also I grepped with the
> recursive option the whole ffmpeg4 source and I couldn't find the error
> message above of course without the 640 and 0...

You are calling frame-related functions haphazardly and seem to have
misunderstood how stride works. And you do not check your return values.
And you seem to be misusing the encoding API too.

Stride is not a property of the image, it is a property of the allocated
memory. That means stride is chosen by whoever allocates the image. If
you ask FFmpeg to allocate the image for you, then FFmpeg gets to decide
the stride.

You would set the stride yourself if you were trying to use an already
allocated memory buffer. For example if you want to use memory mapped
from a video device. Or if you want to use the frame data from
cairo_surface_t directly (which I am not sure is possible, and very
tricky anyway).

As for the frame-related functions:

- Calling av_frame_make_writable() on a frame you just allocated makes
  no sense.

- Calling av_frame_make_writable() before av_frame_get_buffer(). The
  point of av_frame_make_writable() is to reuse buffers that are already
  allocated if possible.

As for the encoding API, your single call to
img_export_encode_av_frame() shows you probably assume one input frame
will give you one output packet. This is not true.

Start with checking the return value of all your function calls, and you
will see where the flaw is.

Look at doc/examples/encode_video.c to see how it does it and work from
there.

Regards,

-- 
  Nicolas George
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/libav-user/attachments/20210816/a742b6ba/attachment.sig>