[FFmpeg-devel] [PATCH] imgutils: make av_image_get_linesize() return 0 for invalid pixel formats

[Stefano Sabatini]

Make av_image_get_linesize() return 0 in case of invalid/meaningless
value for the pixel format.

This makes the behavior of av_image_get_linesize() more consistent
with that of av_image_fill_linesizes(), which sets all linesizes value
to 0 in that case.

Alternatively we may return AVERROR(EINVAL).
---
 libavutil/imgutils.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/libavutil/imgutils.c b/libavutil/imgutils.c
index 154fe87..b274d6b 100644
--- a/libavutil/imgutils.c
+++ b/libavutil/imgutils.c
@@ -50,6 +50,9 @@ int av_image_get_linesize(enum PixelFormat pix_fmt, int width, int plane)
     int max_step_comp[4];       /* the component for each plane which has the max pixel step */
     int s, ls;
 
+    if ((unsigned)pix_fmt >= PIX_FMT_NB || desc->flags & PIX_FMT_HWACCEL)
+        return 0;
+
     av_image_fill_max_pixsteps(max_step, max_step_comp, desc);
     s = (max_step_comp[plane] == 1 || max_step_comp[plane] == 2) ? desc->log2_chroma_w : 0;
     ls = max_step[plane] * (((width + (1 << s) - 1)) >> s);
-- 
1.7.2.3