[FFmpeg-devel] [FFmpeg-cvslog] swscale: use 16-bit intermediate precision for RGB/XYZ conversion

[Michael Niedermayer]

On Thu, Dec 26, 2024 at 07:33:23PM +0000, Niklas Haas wrote:
> ffmpeg | branch: master | Niklas Haas <git at haasn.dev> | Mon Dec 16 14:49:39 2024 +0100| [af6d52eec66961f6a502b0f2f390c12226d087cd] | committer: Niklas Haas
> 
> swscale: use 16-bit intermediate precision for RGB/XYZ conversion
> 
> The current logic uses 12-bit linear light math, which is woefully insufficient
> and leads to nasty postarization artifacts. This patch simply switches the
> internal logic to 16-bit precision.
> 
> This raises the memory requirement of these tables from 32 kB to 272 kB.
> 
> All relevant FATE tests updated for improved accuracy.
> 
> Fixes: #4829
> Signed-off-by: Niklas Haas <git at haasn.dev>
> Sponsored-by: Sovereign Tech Fund
> 
> > http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=af6d52eec66961f6a502b0f2f390c12226d087cd
> ---

this breaks on x86-32

grep Err fatetest
make: *** [src/tests/Makefile:310: fate-filter-pixdesc-xyz12le] Error 1
make: *** [src/tests/Makefile:311: fate-filter-pixdesc-xyz12be] Error 1
make: *** [src/tests/Makefile:310: fate-pixfmt-xyz12le] Error 1
make: *** [src/tests/Makefile:311: fate-filter-pixfmts-fieldorder] Error 1
make: *** [src/tests/Makefile:310: fate-filter-pixfmts-crop] Error 1
make: *** [src/tests/Makefile:311: fate-filter-pixfmts-transpose] Error 1
make: *** [src/tests/Makefile:311: fate-filter-pixfmts-field] Error 1
make: *** [src/tests/Makefile:311: fate-filter-pixfmts-copy] Error 1
make: *** [src/tests/Makefile:310: fate-filter-pixfmts-hflip] Error 1
make: *** [src/tests/Makefile:311: fate-filter-pixfmts-il] Error 1
make: *** [src/tests/Makefile:311: fate-filter-pixfmts-null] Error 1
make: *** [src/tests/Makefile:311: fate-filter-pixfmts-vflip] Error 1
make: *** [src/tests/Makefile:311: fate-filter-pixfmts-scale] Error 1

examples:
--- src/tests/ref/fate/filter-pixfmts-copy	2025-01-26 19:13:18.982388050 +0100
+++ tests/data/fate/filter-pixfmts-copy	2025-01-26 19:14:39.267034804 +0100
@@ -111,8 +111,8 @@
 xv36le              e08dcbde02f1c28a3554f372ad1278e2
 xv48be              ce34993b4b4411bba1d852b9b86aa39e
 xv48le              df913a7e61b162aa98303e5393e60c63
-xyz12be             f257f86373207af8aed0a1a05171df3b
-xyz12le             7922f99edc44a2c26a25becbea9914cc
+xyz12be             d4562ab725e3ab7e5573dd03218bb03c
+xyz12le             ce17d867c3fea4c42324e40353b72de4
 y210le              04e9487b6cce38e7531437e946cdd586
 y212le              825768be8fe92708ae80be84855066ed
 y216le              0e99aeddfee304e72d525d72998d9e9b
--- src/tests/ref/fate/filter-pixfmts-hflip	2025-01-26 19:13:18.982388050 +0100
+++ tests/data/fate/filter-pixfmts-hflip	2025-01-26 19:14:39.275034869 +0100
@@ -108,8 +108,8 @@
 xv36le              e478b4b54698beb3ce1b9a2dd691d544
 xv48be              e030a2c7b1b600cfacb691b6e90c2e3d
 xv48le              fbd7f8c65cd6fc9f9108dc9a1f977dc3
-xyz12be             3c50a51a3c486a0c6853e4bbbcf3f244
-xyz12le             e020897d826ea20ded16f30ea1eb018d
+xyz12be             c678c7f19c610063f352a0253121441f
+xyz12le             2320702cb1c8ccaad27d7004d4f3a195
 ya16be              70fa41c32ecaf3370edc38add6096db2
 ya16le              3b2c20f9e80717628ced6c6468507f63
 ya8                 4ad5920716de3d2fbbc49f95adb60345
Test filter-pixfmts-copy failed. Look at tests/data/fate/filter-pixfmts-copy.err for details.
make: *** [src/tests/Makefile:311: fate-filter-pixfmts-copy] Error 1
Test filter-pixfmts-hflip failed. Look at tests/data/fate/filter-pixfmts-hflip.err for details.
make: *** [src/tests/Makefile:310: fate-filter-pixfmts-hflip] Error 1
TEST    vsynth2-mov-bgr24
TEST    vsynth2-mov-bpp15
--- src/tests/ref/fate/filter-pixfmts-il	2025-01-26 19:13:18.982388050 +0100
+++ tests/data/fate/filter-pixfmts-il	2025-01-26 19:14:39.315035190 +0100
@@ -110,8 +110,8 @@
 xv36le              102c0e817d375ddd6b2cfbb4262dec95
 xv48be              4d7376651fb7b3e84d00abad6c785aad
 xv48le              a1a8ff16d9a864568e5e557734bf3d6d
-xyz12be             b7d50e283360bf69fd661369110b26ef
-xyz12le             d5b1d45c3a136bb3d04f70a619c86c8d
+xyz12be             3d9ae42c65eb645bb8e99dc708716f2d
+xyz12le             b27ec91dcff02d2b94b17c136be55001
 y210le              d4cf9b53cd7ff22f087743d483e88480
 y212le              d5a2b4677ddb4a3bc3e5cd5cbb20f426
 y216le              9e44c6d76b09bcbe71738423b4b3d67a
Test filter-pixfmts-il failed. Look at tests/data/fate/filter-pixfmts-il.err for details.
make: *** [src/tests/Makefile:311: fate-filter-pixfmts-il] Error 1
TEST    vsynth2-mov-bpp16
TEST    vsynth2-roqvideo
TEST    vsynth2-rpza
TEST    vsynth2-smc
--- src/tests/ref/fate/filter-pixfmts-null	2025-01-26 19:13:18.982388050 +0100
+++ tests/data/fate/filter-pixfmts-null	2025-01-26 19:14:39.451036285 +0100
@@ -111,8 +111,8 @@
 xv36le              e08dcbde02f1c28a3554f372ad1278e2
 xv48be              ce34993b4b4411bba1d852b9b86aa39e
 xv48le              df913a7e61b162aa98303e5393e60c63
-xyz12be             f257f86373207af8aed0a1a05171df3b
-xyz12le             7922f99edc44a2c26a25becbea9914cc
+xyz12be             d4562ab725e3ab7e5573dd03218bb03c
+xyz12le             ce17d867c3fea4c42324e40353b72de4
 y210le              04e9487b6cce38e7531437e946cdd586
 y212le              825768be8fe92708ae80be84855066ed
 y216le              0e99aeddfee304e72d525d72998d9e9b
TEST    vsynth2-snow
TEST    vsynth2-snow-hpel
Test filter-pixfmts-null failed. Look at tests/data/fate/filter-pixfmts-null.err for details.
make: *** [src/tests/Makefile:311: fate-filter-pixfmts-null] Error 1


[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The difference between a dictatorship and a democracy is that every 4 years
the population together is allowed to provide 1 bit of input to the government.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20250126/93103320/attachment.sig>