[Libav-user] Handling PIX_FMT_YUV422P10 in my code

[Bruce Wheaton]

> I noticed that a few codecs (DNxHD, ProRes and v210 support this pixel format now). That's great, but I can't find much information about it. I tried to trace where it's being handled in swsscale, and it seems that it's actually a 16-bit format, with unused bits, much like the rarely seen 16-bit planar format. That seems to be borne out by looking at the raw pixels coming out of the decode function?
> 
> Is that right? An array of 16-bit integers, of which only 10 bits are used? So I should just need to scale it to make peak white 1023 instead of 65535?

For anyone else encountering this pixel format the first time:

PIX_FMT_YUV422P10 and PIX_FMT_YUV444P10 are 10-bit planar formats, delivered in 3 (or 4, with alpha) planes of 16-bit integers, in the endianness of the host platform (the actual pixel format has an endian postifix). The 10 least significant bits of each integer represent the pixel component's value.

In the context of OpenGL (on a little endian computer), push the planes onto the GPU as 16-bit luminance integer textures (floating point may work as well, with some different scaling). After sampling a single value, multiply by 1023 to get to a 0.0 - 1.0 range. It's safe to assume that codecs using this pixel format will be providing video level  values, so you would need to scale and offset as well to match openGL ranges.

Bruce