[FFmpeg-devel] [PATCH] avcodec_default_get_buffer should fail instead of returning nonsense for unsupported formats

Reimar Döffinger Reimar.Doeffinger
Mon Feb 23 12:46:13 CET 2009

currently avcodec_default_get_buffer returns complete nonsense when
calling it e.g. to request a VDPAU format.
An obvious solution is to check the return value of ff_fill_pointer,
if it is -1 continuing (and treating -1 as size) obviously makes no
I suspect that some stuff might need to be freed, e.g.
undone but I still have hope that there is someone who knows the code
well enough to implement it correctly so I don't have to try to
understand all of it.
This is the change I had in mind and at least fixes a crash if
get_buffer is not overridden but a hardware-accelerated pixfmt selected:
Index: libavcodec/utils.c
--- libavcodec/utils.c  (revision 17530)
+++ libavcodec/utils.c  (working copy)
@@ -271,6 +271,8 @@
         tmpsize = ff_fill_pointer(&picture, NULL, s->pix_fmt, h);
+        if (tmpsize < 0)
+            return -1;
         for (i=0; i<3 && picture.data[i+1]; i++)
             size[i] = picture.data[i+1] - picture.data[i];

Reimar D?ffinger

More information about the ffmpeg-devel mailing list