[FFmpeg-devel] [PATCH] avcodec_default_get_buffer should fail instead of returning nonsense for unsupported formats
Mon Feb 23 12:46:13 CET 2009
currently avcodec_default_get_buffer returns complete nonsense when
calling it e.g. to request a VDPAU format.
An obvious solution is to check the return value of ff_fill_pointer,
if it is -1 continuing (and treating -1 as size) obviously makes no
I suspect that some stuff might need to be freed, e.g.
undone but I still have hope that there is someone who knows the code
well enough to implement it correctly so I don't have to try to
understand all of it.
This is the change I had in mind and at least fixes a crash if
get_buffer is not overridden but a hardware-accelerated pixfmt selected:
--- libavcodec/utils.c (revision 17530)
+++ libavcodec/utils.c (working copy)
@@ -271,6 +271,8 @@
tmpsize = ff_fill_pointer(&picture, NULL, s->pix_fmt, h);
+ if (tmpsize < 0)
+ return -1;
for (i=0; i<3 && picture.data[i+1]; i++)
size[i] = picture.data[i+1] - picture.data[i];
More information about the ffmpeg-devel