[FFmpeg-devel] [PATCH] libavutil/softfloat: Fix normalizations.

[Michael Niedermayer]

On Wed, Jun 03, 2015 at 04:17:39PM +0200, Nedeljko Babic wrote:
> av_normalize_sf doesn't properly address case when negative numbers are out
> of defined range.
> 
> av_normalize1_sf doesn't properly address border case when mantis is
> exactly 0x40000000.
> 
> This patch solves both of these problems.
> 
> Signed-off-by: Nedeljko Babic <nedeljko.babic at imgtec.com>
> ---
>  libavutil/softfloat.h | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/libavutil/softfloat.h b/libavutil/softfloat.h
> index 8097d28..182e517 100644
> --- a/libavutil/softfloat.h
> +++ b/libavutil/softfloat.h
> @@ -47,7 +47,7 @@ static const SoftFloat FLOAT_0999999    = { 0x3FFFFBCE,   0};
>  static av_const SoftFloat av_normalize_sf(SoftFloat a){
>      if(a.mant){
>  #if 1
> -        while((a.mant + 0x20000000U)<0x40000000U){
> +        while((FFABS(a.mant) + 0x20000000U)<0x40000000U){

when exactly is this needed and how does a.mant reach that value
for which this is needed?
FFABS would significantly slow this down i suspect and it just looks
wrong


>              a.mant += a.mant;
>              a.exp  -= 1;
>          }
> @@ -68,7 +68,7 @@ static av_const SoftFloat av_normalize_sf(SoftFloat a){
>  
>  static inline av_const SoftFloat av_normalize1_sf(SoftFloat a){
>  #if 1
> -    if((int32_t)(a.mant + 0x40000000U) < 0){
> +    if((int32_t)(a.mant + 0x40000000U) <= 0){

    0x40000000 + 0x40000000U == 0x80000000 not 0 so this shouldnt
make a difference

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

I have never wished to cater to the crowd; for what I know they do not
approve, and what they approve I do not know. -- Epicurus
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20150603/dcbc54be/attachment.asc>