[FFmpeg-devel] [PATCH] aacpsy: avoid norm_fac becoming NaN

[Claudio Freire]

On Thu, Apr 16, 2015 at 1:00 PM, Andreas Cadhalpun
<andreas.cadhalpun at googlemail.com> wrote:
> If both band->active_lines and band->thr are 0.0f, the division is
> undefined, making norm_fac not a number.
>
> NaN is passed on to other variables until it finally reaches
> sce->sf_idx and is converted to an integer (-2147483648).
>
> This causes a segmentation fault when it is used as array index.
>
> Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun at googlemail.com>
> ---
>  libavcodec/aacpsy.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/libavcodec/aacpsy.c b/libavcodec/aacpsy.c
> index d1e65f6..b71933b 100644
> --- a/libavcodec/aacpsy.c
> +++ b/libavcodec/aacpsy.c
> @@ -727,7 +727,10 @@ static void psy_3gpp_analyze_channel(FFPsyContext *ctx, int channel,
>                      if (active_lines > 0.0f)
>                          band->thr = calc_reduced_thr_3gpp(band, coeffs[g].min_snr, reduction);
>                      pe += calc_pe_3gpp(band);
> -                    band->norm_fac = band->active_lines / band->thr;
> +                    if (band->active_lines != 0.0f)
> +                        band->norm_fac = band->active_lines / band->thr;
> +                    else
> +                        band->norm_fac = 0.0f;
>                      norm_fac += band->norm_fac;
>                  }
>              }


It should be if band->thr > 0.0f, all divisions by zero return
something that casts into an ~1:

Try:

#include "stdio.h"
int main() {
   printf("%d\n", int(0.0f / 0.0f));
   printf("%d\n", int(1.0f / 0.0f));
   return 0;
}