[FFmpeg-devel] swscale : hScale16To19 : limit shift for float(32bits) input

[Michael Niedermayer]

On Sun, Aug 19, 2018 at 09:00:05PM +0200, Martin Vignali wrote:
> Hello,
> 
> Patch in attach fix for me, Gray Float to UINT16 conversion
> (see discussion : avcodec/psd : add support for gray float (WIP))
> 
> After this patch and psd patch,
> 
> this command line works :
> ./ffmpeg -i lena-gray_float.psd res16.png
> 
> Maybe not the best way to fix that.
> 
> Comments welcome

please add more details to the commit message of this change.


> 
> Martin

>  swscale.c |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 100b144ce6d93e9d40a4f0d6308da2534b346dc6  0002-swscale-hScale16To19-limit-shift-for-float-32bits-in.patch
> From 738a5fc6d5b00bcf6764ca633c3112476149f517 Mon Sep 17 00:00:00 2001
> From: Martin Vignali <martin.vignali at gmail.com>
> Date: Sun, 19 Aug 2018 20:55:02 +0200
> Subject: [PATCH 2/2] swscale : hScale16To19 : limit shift for float(32bits)
>  input
> 
> ---
>  libswscale/swscale.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libswscale/swscale.c b/libswscale/swscale.c
> index 7f3e22355f..7163eb8c5c 100644
> --- a/libswscale/swscale.c
> +++ b/libswscale/swscale.c
> @@ -71,7 +71,7 @@ static void hScale16To19_c(SwsContext *c, int16_t *_dst, int dstW,
>      int i;
>      int32_t *dst        = (int32_t *) _dst;
>      const uint16_t *src = (const uint16_t *) _src;
> -    int bits            = desc->comp[0].depth - 1;
> +    int bits            = FFMIN(desc->comp[0].depth, 16) - 1;

I dont like this, because it is not really capturing the actual intend here
its not that depth beyond 16 are limit to 16.
Its rather that float formats are treated like 16
if there was a float format with 12bit it would likely also be treated like 16

2 lines below is already a if() that override the bits, the float case could be
added below that

i think that would result in easier to understand code
that is unless iam missing something

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

I am the wisest man alive, for I know one thing, and that is that I know
nothing. -- Socrates
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20180819/723144df/attachment.sig>