[FFmpeg-devel] [PATCH 3/4] avformat/rmdec: Use 64bit for intermediate for DEINT_ID_INT4

[Michael Niedermayer]

On Thu, Apr 15, 2021 at 06:22:10PM -0300, James Almer wrote:
> On 4/15/2021 5:44 PM, Michael Niedermayer wrote:
> > Fixes: runtime error: signed integer overflow: 65312 * 65535 cannot be represented in type 'int'
> > Fixes: 32832/clusterfuzz-testcase-minimized-ffmpeg_dem_RM_fuzzer-4817710040088576
> > 
> > Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> > ---
> >   libavformat/rmdec.c | 4 ++--
> >   1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/libavformat/rmdec.c b/libavformat/rmdec.c
> > index fc3bff4859..af032ed90a 100644
> > --- a/libavformat/rmdec.c
> > +++ b/libavformat/rmdec.c
> > @@ -269,9 +269,9 @@ static int rm_read_audio_stream_info(AVFormatContext *s, AVIOContext *pb,
> >           case DEINT_ID_INT4:
> >               if (ast->coded_framesize > ast->audio_framesize ||
> >                   sub_packet_h <= 1 ||
> > -                ast->coded_framesize * sub_packet_h > (2 + (sub_packet_h & 1)) * ast->audio_framesize)
> > +                ast->coded_framesize * (uint64_t)sub_packet_h > (2 + (sub_packet_h & 1)) * ast->audio_framesize)
> 
> This check seems superfluous with the one below right after it.
> ast->coded_framesize * sub_packet_h must be equal to 2 *
> ast->audio_framesize. It can be removed.
> 
> >                   return AVERROR_INVALIDDATA;
> > -            if (ast->coded_framesize * sub_packet_h != 2*ast->audio_framesize) {
> > +            if (ast->coded_framesize * (uint64_t)sub_packet_h != 2*ast->audio_framesize) {
> >                   avpriv_request_sample(s, "mismatching interleaver parameters");
> >                   return AVERROR_INVALIDDATA;
> >               }
> 
> How about something like
> 
> > diff --git a/libavformat/rmdec.c b/libavformat/rmdec.c
> > index fc3bff4859..09880ee3fe 100644
> > --- a/libavformat/rmdec.c
> > +++ b/libavformat/rmdec.c
> > @@ -269,7 +269,7 @@ static int rm_read_audio_stream_info(AVFormatContext *s, AVIOContext *pb,
> >          case DEINT_ID_INT4:
> >              if (ast->coded_framesize > ast->audio_framesize ||
> >                  sub_packet_h <= 1 ||
> > -                ast->coded_framesize * sub_packet_h > (2 + (sub_packet_h & 1)) * ast->audio_framesize)
> > +                ast->audio_framesize > INT_MAX / sub_packet_h)
> >                  return AVERROR_INVALIDDATA;
> >              if (ast->coded_framesize * sub_packet_h != 2*ast->audio_framesize) {
> >                  avpriv_request_sample(s, "mismatching interleaver parameters");
> 
> Instead?

The 2 if() execute different things, the 2nd requests a sample, the first
not. I think this suggestion would change when we request a sample

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

"Nothing to hide" only works if the folks in power share the values of
you and everyone you know entirely and always will -- Tom Scott

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20210416/470187d5/attachment.sig>