[FFmpeg-devel] [PATCH] avutil/softfloat: Assert that the exponent did not overflow the legal range in av_normalize1_sf()

[Andreas Cadhalpun]

On 08.11.2015 13:41, Michael Niedermayer wrote:
> From: Michael Niedermayer <michael at niedermayer.cc>
> 
> Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> ---
>  libavutil/softfloat.h |    1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/libavutil/softfloat.h b/libavutil/softfloat.h
> index 023ccd0..ed1aab3 100644
> --- a/libavutil/softfloat.h
> +++ b/libavutil/softfloat.h
> @@ -79,6 +79,7 @@ static inline av_const SoftFloat av_normalize1_sf(SoftFloat a){
>          a.mant>>=1;
>      }
>      av_assert2(a.mant < 0x40000000 && a.mant > -0x40000000);
> +    av_assert2(a.exp <= MAX_EXP);
>      return a;
>  #elif 1
>      int t= a.mant + 0x40000000 < 0;
> 

This assert would be triggered by more than 15% of my test samples for aac_fixed.
So unless that changes, this assert shouldn't be added.

Example backtrace from id_006808b303914475301abea20d3dd4a0c2aee5a6.aac:
(gdb) bt
#0  0x00007f9707a96107 in __GI_raise (sig=sig at entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007f9707a974e8 in __GI_abort () at abort.c:89
#2  0x00007f970815845b in av_normalize1_sf (a=...) at ./libavutil/softfloat.h:82
#3  0x00007f9708158596 in av_div_sf (a=..., b=...) at ./libavutil/softfloat.h:115
#4  0x00007f970815a3c3 in sbr_gain_calc (ac=0x25ed6e0, sbr=0x7f9709d7dba0, ch_data=0x7f9709d7dc00, e_a=0x7f9709d814e4) at libavcodec/aacsbr_fixed.c:402
#5  0x00007f9708164ddf in ff_sbr_apply_fixed (ac=0x25ed6e0, sbr=0x7f9709d7dba0, id_aac=0, L=0x25f6180, R=0x25f81e0) at libavcodec/aacsbr_template.c:1489
#6  0x00007f970810fa11 in spectral_to_sample (ac=0x25ed6e0, samples=0) at libavcodec/aacdec_template.c:2758
#7  0x00007f9708110713 in aac_decode_frame_int (avctx=0x25eafa0, data=0x25f59c0, got_frame_ptr=0x7fffe9f53f9c, gb=0x7fffe9f53bf0, avpkt=0x7fffe9f53c80)
    at libavcodec/aacdec_template.c:3053
#8  0x00007f9708110a53 in aac_decode_frame (avctx=0x25eafa0, data=0x25f59c0, got_frame_ptr=0x7fffe9f53f9c, avpkt=0x7fffe9f53c80) at libavcodec/aacdec_template.c:3153
#9  0x00007f97087de240 in avcodec_decode_audio4 (avctx=0x25eafa0, frame=0x25f59c0, got_frame_ptr=0x7fffe9f53f9c, avpkt=0x7fffe9f53f40) at libavcodec/utils.c:2203
...
(gdb) frame 2
#2  0x00007f970815845b in av_normalize1_sf (a=...) at ./libavutil/softfloat.h:82
82	    av_assert2(a.exp <= MAX_EXP);
(gdb) p a
$1 = {mant = 536870912, exp = 268435463}

Best regards,
Andreas