[FFmpeg-devel] [PATCH 1/5] avutil: Add YUV444P10_LSB and YUV444P12_LSB pixel formats

[Marton Balint]

On Tue, 9 Oct 2018, Timo Rothenpieler wrote:

> On 09/10/2018 02:32, Carl Eugen Hoyos wrote:
>> 2018-10-08 21:59 GMT+02:00, Timo Rothenpieler <timo at rothenpieler.org>:
>>>>> So, to be able to use these hardware compatible formats, we need
>>>>> definitions for them in ffmpeg. Right now, I need this for nvdec,
>>>>> but Vulkan also uses the same format definitions.
>>>> 
>>>> Sorry if this was already done and I forgot but please explain why
>>>> you cannot use YUV444P16 for this use-case.
>>>> 
>>>> If the only thing missing is libavfilter understanding
>>>> bits_per_raw_sample
>>>> we should add it there: It is needed in any case and would save us a few
>>>> pix_fmts and could speed up many use cases compared to your solution.
>>> 
>>> It's pretty much all of ffmpeg and all client applications understanding
>>> it.
>>> First of all, bits_per_raw_sample, or an equivalent bit depth field, is
>>> missing from AVFrame.
>>> "In" avfilter it's primarily swscale I guess, but it also affects each
>>> and every filter supporting YUV444P16.
>>> Also, for encoders, it would mean they need a way to indicate what bit
>>> depths they can take, and that also has to be per pix_fmt they support.
>>> Then all of avcodec, ffmpeg.c, ... needs support for that.
>>> It's quite a deep rabit hole of changes with quite a good chance of
>>> something going horribly wrong in the process. Probably even in more
>>> places than just the ones mentioned here.
>> 
>> This may or may not be true, unfortunately my question was not
>> how difficult it would be to fix this mess (I don't think it is), but
>> why you want to add more to the mess.
>> 
>> What would happen if you just use YUV444P16?
>
> For example nvenc only supports 10 bit encoding, but takes it in the 
> YUV444P16 format. The rest of ffmpeg has no idea it only supports 10 bit, and 
> at times even up-converts 10 bit content to 16 bit via swscale due to that.

Isn't the same thing happening for audio with s24le codec?

What is the downside of extending the precision of the filtering pipeline 
to full 16 bit in cases like this? I don't think it affects most filters 
speedwise.

> Also, it loses quality without the user noticing, as there is no outside 
> indication it does in fact not support 16 bit.

It's up to the tool to be user friendly, not the framework.

I still think proper implementation is hard (even the YUVJ removal patch 
got abandoned), so I don't know how to move forward. Maybe we should come 
up with some issue and requirement list first, and decide after that.

Regards,
Marton