[FFmpeg-devel] [PATCH 7/9] avformat/rmdec: Fix potential shift outside of range of int

[James Almer]

On 7/20/2020 11:12 PM, Andreas Rheinhardt wrote:
> The loop variable here that can be as high as UINT16_MAX - 1 gets
> left-shifted by 16 bits which is outside the range of int. So use
> unsigned.
> 
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at gmail.com>
> ---
>  libavformat/rmdec.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavformat/rmdec.c b/libavformat/rmdec.c
> index c88f41c121..e97b861dee 100644
> --- a/libavformat/rmdec.c
> +++ b/libavformat/rmdec.c
> @@ -500,7 +500,7 @@ static int rm_read_multi(AVFormatContext *s, AVIOContext *pb,
>      if (number_of_mdpr != 1) {
>          avpriv_request_sample(s, "MLTI with multiple (%d) MDPR", number_of_mdpr);

So most of the code below is untested?

Also, AVStream->id is an int, so maybe just ensure number_of_mdpr is
equal or less than INT16_MAX, and perhaps also that st->id is equal or
less than UINT16_MAX before doing the addition, and abort otherwise
instead of changing the type for i.

>      }
> -    for (i = 0; i < number_of_mdpr; i++) {
> +    for (unsigned i = 0; i < number_of_mdpr; i++) {
>          AVStream *st2;
>          if (i > 0) {
>              st2 = avformat_new_stream(s, NULL);
>