[FFmpeg-devel] [PATCH] swresample/rematrix: fix mixing of center channels

[Michael Niedermayer]

Hi

On Mon, Jan 13, 2025 at 05:38:14PM -0300, James Almer wrote:
> On 1/13/2025 5:30 PM, Michael Niedermayer wrote:
> > Hi
> > 
> > On Mon, Jan 13, 2025 at 03:51:07PM -0300, James Almer wrote:
> > > On 1/13/2025 12:49 AM, Michael Niedermayer wrote:
> > > > Hi James
> > > > 
> > > > On Sun, Jan 12, 2025 at 06:52:55PM -0300, James Almer wrote:
> > > > > On 1/12/2025 6:25 PM, Michael Niedermayer wrote:
> > > > > > Hi
> > > > > > 
> > > > > > On Sat, Jan 11, 2025 at 11:13:30AM -0300, James Almer wrote:
> > > > > > > With this, the output from
> > > > > > > 
> > > > > > > ffmpeg -i $fate-samples/ac3/monsters_inc_5.1_448_small.ac3 -af aresample,aformat=channel_layouts=mono -f s16le -
> > > > > > > 
> > > > > > > matches the one from
> > > > > > > 
> > > > > > > ffmpeg -downmix mono -i $fate-samples/ac3/monsters_inc_5.1_448_small.ac3 -f s16le -
> > > > > > > 
> > > > > > > When testing with tiny_psnr.
> > > > > > > 
> > > > > > > Signed-off-by: James Almer <jamrial at gmail.com>
> > > > > > > ---
> > > > > > >     libswresample/rematrix.c | 4 ++--
> > > > > > >     1 file changed, 2 insertions(+), 2 deletions(-)
> > > > > > 
> > > > > > This worsense test results:
> > > > > 
> > > > > What test is this, and how is it read?
> > > > 
> > > > its just tests/swresample
> > > > 
> > > > and a dumb script that tries to compare its output to spot cases that worsen
> > > Is this any better? (Also makes the above command line match)
> > 
> > Can we take a step back here.
> > 
> > please explain what you are fixing exactly
> > 
> > the matrix should be 1.0 on channels that exist before and afterwards
> > so a speaker thats there before the mix and after the mix produces the
> > same sound. That should naively be closest to correct,
> > Speakers which disappear need to have their sound mixed into existing
> > speakers. But existing speakers that dont change should not have their
> > output change if the goal is to replicate the input
> > 
> > Of course maybe something else is better in some case (i dont know)
> > but id like to read an explanation what and why. Not just that it
> > matches something else
> 
> The line in question is the coeff to mix front center from input to front
> center from output in case front L/R channels from input are also mixed into
> front center (so surround/3.1 -> mono, 5.1 -> mono, etc).
> 

> The current coeff doesn't match the behavior of the downmix code in the ac3
> decoder.

I dont like this argument, we are not trying to replicate the downmix code
from our ac3 decoder.


> Seeing it's the only coeff here where the value is outright
> overwritten rather than being subtracted/added, it's probably bogus, and I'm
> trying to find the (most) correct one that gives equal results.

there are 2 different cases
matrix[a][b] where a==b and where they are not equal

one uses add the other uses overwritte
iam not sure it makes a difference for teh != case but for == it makes a difference
because the value is not 0 before.

also theres a user parameter (center mix level) to adjust this.

What id like to see is one of these:
A. A double blind listeing test where someone confirms it sounds more correct
B. A reference to some mixing related paper that supports this
C. Some argumentation related to physics and audio that supports
   the channge of value

Also this patch affects many cases. Iam not saying its bad, just that
i cannot judge it from just knowing its different from our ac3 decoder

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Good people do not need laws to tell them to act responsibly, while bad
people will find a way around the laws. -- Plato
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20250114/2470fdb9/attachment.sig>