[FFmpeg-devel] [PATCH] swscale: fix chroma width rounding

[Michael Niedermayer]

On Wed, Jun 02, 2021 at 11:02:44PM +0800, lance.lmwang at gmail.com wrote:
> From: Limin Wang <lance.lmwang at gmail.com>
> 
> fixes vertical line at the right side of video for rgb24toyv12 conversion.
> Please reproduce it with below command:
> ./ffplay -f lavfi -i "testsrc=size=529x656,ocv=filter_name=smooth,format=yuv420p"
> 
> Signed-off-by: Limin Wang <lance.lmwang at gmail.com>
> ---
>  libswscale/rgb2rgb_template.c     | 2 +-
>  libswscale/x86/rgb2rgb_template.c | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/libswscale/rgb2rgb_template.c b/libswscale/rgb2rgb_template.c
> index 42c6980..e716ba5 100644
> --- a/libswscale/rgb2rgb_template.c
> +++ b/libswscale/rgb2rgb_template.c
> @@ -654,7 +654,7 @@ void ff_rgb24toyv12_c(const uint8_t *src, uint8_t *ydst, uint8_t *udst,
>      int32_t ru = rgb2yuv[RU_IDX], gu = rgb2yuv[GU_IDX], bu = rgb2yuv[BU_IDX];
>      int32_t rv = rgb2yuv[RV_IDX], gv = rgb2yuv[GV_IDX], bv = rgb2yuv[BV_IDX];
>      int y;
> -    const int chromWidth = width >> 1;
> +    const int chromWidth = AV_CEIL_RSHIFT(width, 1);
>  
>      for (y = 0; y < height; y += 2) {
>          int i;
> diff --git a/libswscale/x86/rgb2rgb_template.c b/libswscale/x86/rgb2rgb_template.c
> index ae2469e..704b0fa 100644
> --- a/libswscale/x86/rgb2rgb_template.c
> +++ b/libswscale/x86/rgb2rgb_template.c
> @@ -1578,7 +1578,7 @@ static inline void RENAME(rgb24toyv12)(const uint8_t *src, uint8_t *ydst, uint8_
>  #define BGR2U_IDX "16*4+16*33"
>  #define BGR2V_IDX "16*4+16*34"
>      int y;
> -    const x86_reg chromWidth= width>>1;
> +    const x86_reg chromWidth = AV_CEIL_RSHIFT(width, 1);
>  
>      if (height > 2) {
>          ff_rgb24toyv12_c(src, ydst, udst, vdst, width, 2, lumStride, chromStride, srcStride, rgb2yuv);

the comments above the functions are incorrect if you add support for odd width
also the code writes outside the luma array i think with this change

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Those who are too smart to engage in politics are punished by being
governed by those who are dumber. -- Plato 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20210603/58a9dc0e/attachment.sig>