[FFmpeg-devel] [PATCH 3/3] swscale/swscale_unscaled: fix packed16togbra16() for formats with bpc between 9-14 bits

[Ramiro Polla]

On Mon, May 19, 2025 at 12:02 AM Ramiro Polla <ramiro.polla at gmail.com> wrote:
> On Sun, May 18, 2025 at 11:17 PM James Almer <jamrial at gmail.com> wrote:
> > On 5/18/2025 6:14 PM, James Almer wrote:
> > > On 5/18/2025 5:52 PM, Ramiro Polla wrote:
> > >> Currently, packed16togbra16() always sets the alpha value to 0xFFFF,
> > >> without taking the bit depth into consideration.
> > >>
> > >> This commit restricts the alpha value to the bit depth.
> > >
> > > packed16togbra16() seems to only be called for BGR48 and BGRA64, both of
> > > which are 16bits, so this change is superfluous.
> >
> > Ah, nevermind, i was looking at the src formats, not dst.
> >
> > Are there no tests that cover these paths? I added a bunch a couple
> > months ago, so maybe it could be extended.
>
> It can be reproduced with:
> ./libswscale/tests/swscale -unscaled 1 -src xyz12le -dst gbrap12be
>
> A little bit more information: this bug only happens on x86. The
> problem arises from the optimized conversion that comes afterwards,
> from gbrap12be to yuva444p, in ff_hscale14to15_4_ssse3(). It has
> something to do with pmaddwd not working on unsigned values IIRC. We
> could fix ff_hscale14to15_4_ssse3() to also work correctly with 0xFFFF
> on bit depths < 16, or we could just not write 0xFFFF there in the
> first place, which is what this patch does.
>
> I thought about adding libswscale/tests/swscale to FATE, but the tests
> take way too long.

I'll apply this patchset and the other similar patch about
planarRgbToplanarRgbWrapper() tomorrow if there are no more comments.

Ramiro