[FFmpeg-devel] [PATCH v3] vaapi_encode_mjpeg: fix bad component id bug

[Eoff, Ullysses A]

> -----Original Message-----
> From: Eoff, Ullysses A
> Sent: Friday, June 07, 2019 2:42 PM
> To: ffmpeg-devel at ffmpeg.org
> Cc: Eoff, Ullysses A <ullysses.a.eoff at intel.com>
> Subject: [PATCH v3] vaapi_encode_mjpeg: fix bad component id bug
> 
> The compound literals assigned to "components"
> only exist within the scope of the if/else
> block (thanks Mike Thompson for the better

Oh crap, I got your name wrong.  It must be a Friday!  V4 coming up!

> explanation).
> 
> Thus, after this if/else block, "components"
> ends up pointing to an arbitrary/undefined
> array.  With some compilers and depending on
> optimization settings, these arbitrary values
> may end up being the same value (i.e. 0 with
> GNU GCC 9.x).  Unfortunately, the GNU GCC
> compiler, at least, never prints any warnings
> about this.
> 
> This patch fixes this issue by assigning the
> constant arrays to local variables at function
> scope and then pointing "components" to those
> as necessary.
> 
> Fixes #7915
> 
> Signed-off-by: U. Artie Eoff <ullysses.a.eoff at intel.com>
> ---
>  libavcodec/vaapi_encode_mjpeg.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/vaapi_encode_mjpeg.c b/libavcodec/vaapi_encode_mjpeg.c
> index 4dcdc3d16bb0..bd029cc90315 100644
> --- a/libavcodec/vaapi_encode_mjpeg.c
> +++ b/libavcodec/vaapi_encode_mjpeg.c
> @@ -227,6 +227,8 @@ static int vaapi_encode_mjpeg_init_picture_params(AVCodecContext *avctx,
>      JPEGRawScanHeader                 *sh = &priv->scan.header;
>      VAEncPictureParameterBufferJPEG *vpic = pic->codec_picture_params;
>      const AVPixFmtDescriptor *desc;
> +    const uint8_t components_rgb[3] = { 'R', 'G', 'B' };
> +    const uint8_t components_yuv[3] = {  1,   2,   3  };
>      const uint8_t *components;
>      int t, i, quant_scale, len;
> 
> @@ -235,9 +237,9 @@ static int vaapi_encode_mjpeg_init_picture_params(AVCodecContext *avctx,
>      desc = av_pix_fmt_desc_get(priv->common.input_frames->sw_format);
>      av_assert0(desc);
>      if (desc->flags & AV_PIX_FMT_FLAG_RGB)
> -        components = (uint8_t[3]) { 'R', 'G', 'B' };
> +        components = components_rgb;
>      else
> -        components = (uint8_t[3]) {  1,   2,   3  };
> +        components = components_yuv;
> 
>      // Frame header.
> 
> --
> 2.20.1