[FFmpeg-devel] libavcodec/exr : fix float to uint16 conversion for negative value

[Michael Niedermayer]

On Sun, Apr 23, 2017 at 06:00:06PM +0200, Martin Vignali wrote:
> Hello,
> 
> In attach a patch who fix this sample :
> https://we.tl/HWgF0SFEll
> 
> This sample contain negative value in the red channel.
> 
> Before this patch, negative red float are converted to max red
> instead of zero.
> 
> The previous hack doesn't seems to work for float to uint
> so i add an explicit sign check
> 
> but i'm not familiar with float binary manipulation,
> so maybe there is a better way to fix the problem
> 
> Comments welcome
> 
> Martin
> Jokyo Images

>  exr.c |    6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 99febb919a36ca28f2fd2500cd820f5ec75ebeb0  0002-libavcodec-exr-fix-float-to-uint16-conversion-for-ne.patch
> From 5385ed8d8d258e68474e3ff24b601fd9ae595677 Mon Sep 17 00:00:00 2001
> From: Martin Vignali <martin.vignali at gmail.com>
> Date: Sun, 23 Apr 2017 17:57:53 +0200
> Subject: [PATCH 2/2] libavcodec/exr : fix float to uint16 conversion for
>  negative value
> 
> the previous hack doesn't seems to work
> so add an explicit sign check
> ---
>  libavcodec/exr.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/libavcodec/exr.c b/libavcodec/exr.c
> index 7194640..0e69a8d 100644
> --- a/libavcodec/exr.c
> +++ b/libavcodec/exr.c
> @@ -223,9 +223,9 @@ static union av_intfloat32 exr_half2float(uint16_t hf)
>  static inline uint16_t exr_flt2uint(uint32_t v)
>  {
>      unsigned int exp = v >> 23;
> -    // "HACK": negative values result in exp<  0, so clipping them to 0
> -    // is also handled by this condition, avoids explicit check for sign bit.
> -    if (exp <= 127 + 7 - 24) // we would shift out all bits anyway
> +    if (v >> 31 == 1)/* clamp negative value to 0*/
> +        return 0;
> +    if (exp <= 127 + 7 - 24)
>          return 0;

did you try to make exp int32_t to avoid the 2nd check ?

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Those who are too smart to engage in politics are punished by being
governed by those who are dumber. -- Plato 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20170425/a61a4385/attachment.sig>