[FFmpeg-devel] [PATCH 1/3] lavu/pixfmt: Add Y216, Y410, and Y416 formats

[Philip Langdale]

On Mon, 15 Aug 2022 10:42:11 -0700
Philip Langdale <philipl at overt.org> wrote:

> On Mon, 15 Aug 2022 06:12:20 +0000
> "Xiang, Haihao" <haihao.xiang-at-intel.com at ffmpeg.org> wrote:
> 
> > 
> > Hi Philip,
> > 
> > May we add new formats P012, Y212 and Y412 for 12bit contents ? I
> > agree with Mark's comment in 
> > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200619015248.21873-1-fei.w.wang@intel.com/
> >  
> > 
> > "
> > Tracking it separately does not seem fun - it looks to me like it
> > would require adding a new bit depth field to AVFrame.
> > 
> > FFmpeg has always used pixfmt as defining both the memory layout and
> > which bits are used in that (so, for example, ARGB and 0RGB are not
> > the same thing), unlike most of the graphics APIs which tend to 
> > define those two separately.
> > "  
> 
> I went through this same conversation a few years ago around the
> behaviour of nvdec, which uses p016 and yuv444p16 for 12bit content.
> All the same arguments were made about not introducing new pixel
> formats to just indicate stream depth and how we could add a new
> attribute but it would be a bunch of work, and to avoid bogging down
> and doing nothing, I just had nvdec/nvenc work with P016 (and
> honestly, nvidia don't even use P010 internally, it's P016 for >=
> 10bit) and it was fine. On the decoder side, there's no real problem,
> and on the encoder side, I think the key insight is that the encoder
> doesn't encoder to the frame's pixel format depth, it encodes to the
> depth _specified by the profile_. So even if you give it real 16bit
> frames in P016, if you are encoding Main12, you are going to get
> 12bit. Same for any future Main14. I'd argue this is the most correct
> way to handle it anyway. If the default profile we guess based on
> format isn't right, specify the one you actually want.

Hi Hai Hao,

So, Mark has pointed out to me that this isn't the same situation as
nvidia, because the driver is actually exposing support for specific
P012, Y212, Y412 formats. I didn't realise this, which is why I took
the approach I did.

With this new information, I will redo it to add those formats instead
of the 16bit ones (which don't have a reason to exist in ffmpeg for
now).

Thanks!

--phil