[FFmpeg-devel] SAMPLE_FMT_FLT bounds

[Måns Rullgård]

Colin McQuillan <m.niloc at googlemail.com> writes:

> I'm looking for the correct way to clip the output of a
> floating-point decoder.
>
> audioconvert.c converts floats thus:
> lrintf(*(const float*)pi * (1<<15))) to convert to int16_t
> lrintf(*(const float*)pi * (1<<7)) + 0x80) to convert to uint8_t
>
> qcelpdec.c clips to [-1.0, 32767.0/32768.0]. This is alright when
> converting to 16-bit but will overflow when converting to 8-bit.
> Should I clip to [-1, 127.0/128.0]? Or [-128.499/127, 127.499/128]?

IMO it would make more sense to combine clipping with conversion, as
this can often be done in a single instruction.  If this means
changing other decoders, so be it.

> The attached patch is just an idea.
>
> Index: libavcodec/qcelpdec.c
> ===================================================================
> --- libavcodec/qcelpdec.c	(revision 19594)
> +++ libavcodec/qcelpdec.c	(working copy)
> @@ -826,8 +826,8 @@
>  
>      formant_mem = q->formant_mem + 10;
>      for(i=0; i<160; i++)
> -        *outbuffer++ = av_clipf(*formant_mem++, QCELP_CLIP_LOWER_BOUND,
> -                                QCELP_CLIP_UPPER_BOUND);
> +        *outbuffer++ = av_clipf(*formant_mem++, SAMPLE_FMT_FLT_MIN,
> +                                                SAMPLE_FMT_FLT_MAX);

This should be simdified.

-- 
M?ns Rullg?rd
mans at mansr.com