[FFmpeg-devel] [PATCH 000/279] New channel layout API

[Anton Khirnov]

Quoting Marton Balint (2021-12-10 01:04:57)
> 
> 
> On Thu, 9 Dec 2021, Anton Khirnov wrote:
> 
> > Quoting Nicolas George (2021-12-09 11:31:54)
> >> Anton Khirnov (12021-12-09):
> >>> I disagree. Technical limitations that were overcome 10 years ago should
> >>> not guide new API design.
> >>
> >> In the case of amerge, it was not a technical limitation, merging
> >> several streams into one so that they can be handled by single-stream
> >> filters is 100% part of the design.
> >
> > I fail to see how that is an advantage. You can just as well create
> > multiple instances of those single-stream filters instead of adding
> > hacks into core APIs.
> >
> >> I suspect devices that capture several independent channels are
> >> designed that way intentionally too, possibly to reduce the risk of
> >> desynchronization.
> >
> > "possibly" is not a strong enough argument. I'd like to hear at least
> > one clearly-defined use case that cannot just as well be handled by
> > using multiple streams.
> 
> I recently worked on the MXF demuxer to recognize channel designations in 
> MXF files, and in MXF the designation and grouping of the channels is 
> completely separate from the track those channels are muxed in.
> 
> So if you have e.g. english stereo sound, and german stereo sound you 
> can mux it
> - as a single 4 channel track
> - as two 2 channel tracks
> - as four 1 channel tracks.
> Some MXF flavors use the multichannel single track approach, others the 
> mono-track-only approach. So the user may not be able to choose the 
> optimal muxed track assignment...
> 
> So ultimately, if you demux and decode a packet from a track, you will 
> have an AVFrame, which might contain a single sound group on its own 
> (optimal case), part of a sound group, or multiple sound groups.
> 
> To summerize, muxed tracks are not necessarily 1:1 mapping between sound 
> groups. And when processing/filtering audio, you typically want sound 
> groups, not tracks. And yes, it is very rare to have a soundgroup which 
> has channels with the same designation, but for a muxed track, it depends 
> on the format.
> 
> The goal of the end user is probably to be able to specify sound groups, 
> not select muxed tracks. Preferably a demuxer should provide which channel 
> is part of which sound group, and you should be able to use a filter or a 
> combination of filters select a specific sound group. E.g. amerge all 
> tracks, then keep only the channels from all the merged channels which are 
> part of a specific sound group.

So what are you proposing? In my view, such higher level information
should live at a higher level - e.g. in the side data. You can then
have a filter that reads this side data and gets you the group you want.

-- 
Anton Khirnov