[FFmpeg-devel] [PATCH 4/9] avformat/rmdec: Actually return value < 0 on read_header failure

[Andreas Rheinhardt]

Andreas Rheinhardt:
> The RealMedia demuxer's read_header function initially initializes ret,
> the variable designated for the return variable to -1. Afterwards, chunks
> of the file are parsed in a loop until an error happens or until the actual
> frame data is encountered. If the first function whose return
> value is put into ret doesn't fail, then ret contains a value >= 0
> (actually == 0) and this is what will be returned if an error is
> encountered afterwards.
> 
> This is a regression since 35bbc1955a58ba74552c50d9161084644f00bbd3.
> Before that, ret had never been overwritten with a nonnegative value.
> 
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at gmail.com>
> ---
>  libavformat/rmdec.c | 8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/libavformat/rmdec.c b/libavformat/rmdec.c
> index a36e693ab2..6851b7e1f4 100644
> --- a/libavformat/rmdec.c
> +++ b/libavformat/rmdec.c
> @@ -538,7 +538,7 @@ static int rm_read_header(AVFormatContext *s)
>      unsigned int data_off = 0, indx_off = 0;
>      char buf[128], mime[128];
>      int flags = 0;
> -    int ret = -1;
> +    int ret;
>      unsigned size, v;
>      int64_t codec_pos;
>  
> @@ -554,6 +554,7 @@ static int rm_read_header(AVFormatContext *s)
>      avio_skip(pb, tag_size - 8);
>  
>      for(;;) {
> +        ret = AVERROR_INVALIDDATA;
>          if (avio_feof(pb))
>              goto fail;
>          tag = avio_rl32(pb);
> @@ -619,8 +620,9 @@ static int rm_read_header(AVFormatContext *s)
>                  avio_seek(pb, codec_pos + size, SEEK_SET);
>              } else {
>                  avio_skip(pb, -4);
> -                if (ff_rm_read_mdpr_codecdata(s, s->pb, st, st->priv_data,
> -                                              size, mime) < 0)
> +                ret = ff_rm_read_mdpr_codecdata(s, s->pb, st, st->priv_data,
> +                                                size, mime);
> +                if (ret < 0)
>                      goto fail;
>              }
>  
> 
Will apply this and the next patch soon.

- Andreas