[FFmpeg-devel] Is this the expected behavior in libswscale/input.c
dan.parrot at mail.com
Thu Jun 9 22:02:00 CEST 2016
Line 72 of libswscale/input.c is:
dstU[i] = (ru*r + gu*g + bu*b + (0x10001<<(RGB2YUV_SHIFT-1))) >>
The definition of macro RGB2YUV_SHIFT in libswscale/swscale_internal.h
is on line 417:
#define RGB2YUV_SHIFT 15
By examining the result of executing line 72 in input.c it appears that
the radix used for macro RGB2YUV_SHIFT is hexadecimal. So that 0x10001
is shifted left by 20 in subexpression 0x10001<<(RGB2YUV_SHIFT-1) with
the result being 0x100000.
I was expecting a left-shift of 14, given the macro definition.
Should the macro be interpreted as decimal 15 or hexadecimal 15?
More information about the ffmpeg-devel