[FFmpeg-devel] [PATCH 2/5] lavu/hwcontext_vaapi: add vaapi_format_map for Y210/AYUV/Y410

[Fu, Linjie]

> -----Original Message-----
> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On Behalf
> Of Mark Thompson
> Sent: Sunday, July 7, 2019 20:13
> To: ffmpeg-devel at ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH 2/5] lavu/hwcontext_vaapi: add
> vaapi_format_map for Y210/AYUV/Y410
> 
> On 28/06/2019 03:27, Linjie Fu wrote:
> > Signed-off-by: Linjie Fu <linjie.fu at intel.com>
> > ---
> >  libavutil/hwcontext_vaapi.c | 8 ++++++++
> >  1 file changed, 8 insertions(+)
> >
> > diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c
> > index 4227c3c..6378d0e 100644
> > --- a/libavutil/hwcontext_vaapi.c
> > +++ b/libavutil/hwcontext_vaapi.c
> > @@ -116,6 +116,14 @@ static const VAAPIFormatDescriptor
> vaapi_format_map[] = {
> >  #endif
> >      MAP(UYVY, YUV422,  UYVY422, 0),
> >      MAP(YUY2, YUV422,  YUYV422, 0),
> > +#ifdef VA_FOURCC_Y210
> > +    MAP(Y210, YUV422_10,Y210LE, 0),
> > +#endif
> > +#define VA_RT_FORMAT_AYUV VA_FOURCC_AYUV
> 
> VA_RT_FORMAT_* is a mask, VA_FOURCC_* is effectively a set of enum
> values.  They aren't usable as each other like this.
> 
> > +    MAP(AYUV,   AYUV,     AYUV, 0),
> 
> Is AYUV actually intended to support decoding alpha, or is it just a packed
> representation for YUV 4:4:4 data (like RGB0)?  If the former, you should
> probably add a new render target format to libva.  If the latter, you can use
> VA_RT_FORMAT_YUV444 with it.

There is no VA_RT_FORMAT_AYUV in defined in libva, and currently in media-driver,
VA_FOURCC_AYUV is used to represent VA_RT_FORMAT_AYUV in a hacky way.

Agreed there may be some hidden work in libva/media-driver to avoid hack.
Thanks for review.