[FFmpeg-cvslog] lavc/imgconvert: fix check on av_image_check_size() return code in avpicture_get_size()

[Stefano Sabatini]

ffmpeg | branch: master | Stefano Sabatini <stefasab at gmail.com> | Sat Jun 16 11:38:50 2012 +0200| [18b4404dc35eef67620066e11086048b933c2e4c] | committer: Stefano Sabatini

lavc/imgconvert: fix check on av_image_check_size() return code in avpicture_get_size()

The documentation states that av_image_check_size() will return a
negative value in case of error, while the check is done on ret != 0.

Also return a proper error code rather than -1 in case the check fails.

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=18b4404dc35eef67620066e11086048b933c2e4c
---

 libavcodec/imgconvert.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/imgconvert.c b/libavcodec/imgconvert.c
index 495a1d0..e517812 100644
--- a/libavcodec/imgconvert.c
+++ b/libavcodec/imgconvert.c
@@ -346,8 +346,8 @@ int avpicture_layout(const AVPicture* src, enum PixelFormat pix_fmt, int width,
 int avpicture_get_size(enum PixelFormat pix_fmt, int width, int height)
 {
     AVPicture dummy_pict;
-    if(av_image_check_size(width, height, 0, NULL))
-        return -1;
+    if (av_image_check_size(width, height, 0, NULL) < 0)
+        return AVERROR(EINVAL);
     if (av_pix_fmt_descriptors[pix_fmt].flags & PIX_FMT_PSEUDOPAL)
         // do not include palette for these pseudo-paletted formats
         return width * height;