[FFmpeg-devel] [PATCH v5 1/5] libavcodec: VAAPI support infrastructure
Mark Thompson
sw at jkqxz.net
Tue Feb 2 20:51:53 CET 2016
On 01/02/16 15:58, wm4 wrote:
> On Sat, 30 Jan 2016 22:11:52 +0000
> Mark Thompson <sw at jkqxz.net> wrote:
>
>> ---
>> configure | 5 +
>> libavcodec/Makefile | 1 +
>> libavcodec/vaapi_support.c | 710 +++++++++++++++++++++++++++++++++++++++++++++
>> libavcodec/vaapi_support.h | 122 ++++++++
>> 4 files changed, 838 insertions(+)
>> create mode 100644 libavcodec/vaapi_support.c
>> create mode 100644 libavcodec/vaapi_support.h
>>
>> diff --git a/configure b/configure
>> index dba8180..e7f53af 100755
>> --- a/configure
>> +++ b/configure
>> @@ -2037,6 +2037,7 @@ CONFIG_EXTRA="
>> texturedsp
>> texturedspenc
>> tpeldsp
>> + vaapi_recent
>> videodsp
>> vp3dsp
>> vp56dsp
>> @@ -5768,6 +5769,10 @@ enabled vaapi &&
>> check_lib va/va.h vaInitialize -lva ||
>> disable vaapi
>>
>> +enabled vaapi &&
>> + check_code cc va/va.h "vaCreateSurfaces(0, 0, 0, 0, 0, 0, 0, 0)" &&
>> + enable vaapi_recent
>> +
>> enabled vaapi && enabled xlib &&
>> check_lib2 "va/va.h va/va_x11.h" vaGetDisplay -lva -lva-x11 &&
>> enable vaapi_x11
>> diff --git a/libavcodec/Makefile b/libavcodec/Makefile
>> index de957d8..045d118 100644
>> --- a/libavcodec/Makefile
>> +++ b/libavcodec/Makefile
>> @@ -719,6 +719,7 @@ OBJS-$(CONFIG_ADPCM_YAMAHA_ENCODER) += adpcmenc.o adpcm_data.o
>> OBJS-$(CONFIG_D3D11VA) += dxva2.o
>> OBJS-$(CONFIG_DXVA2) += dxva2.o
>> OBJS-$(CONFIG_VAAPI) += vaapi.o
>> +OBJS-$(CONFIG_VAAPI_RECENT) += vaapi_support.o
>> OBJS-$(CONFIG_VDA) += vda.o videotoolbox.o
>> OBJS-$(CONFIG_VIDEOTOOLBOX) += videotoolbox.o
>> OBJS-$(CONFIG_VDPAU) += vdpau.o
>> diff --git a/libavcodec/vaapi_support.c b/libavcodec/vaapi_support.c
>> new file mode 100644
>> index 0000000..2e22f98
>> --- /dev/null
>> +++ b/libavcodec/vaapi_support.c
>> @@ -0,0 +1,710 @@
>> +/*
>> + * VAAPI helper functions.
>> + *
>> + * Copyright (C) 2016 Mark Thompson <mrt at jkqxz.net>
>> + *
>> + * This file is part of FFmpeg.
>> + *
>> + * FFmpeg is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU Lesser General Public
>> + * License as published by the Free Software Foundation; either
>> + * version 2.1 of the License, or (at your option) any later version.
>> + *
>> + * FFmpeg is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
>> + * Lesser General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU Lesser General Public
>> + * License along with FFmpeg; if not, write to the Free Software
>> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
>> + */
>> +
>> +#include "vaapi_support.h"
>> +
>> +#include "libavutil/avassert.h"
>> +#include "libavutil/imgutils.h"
>> +#include "libavutil/pixfmt.h"
>> +
>> +
>> +static AVClass vaapi_class = {
>> + .class_name = "vaapi",
>> + .item_name = av_default_item_name,
>> + .version = LIBAVUTIL_VERSION_INT,
>> +};
>> +static AVClass *vaapi_log = &vaapi_class;
>
> We've talked about it on IRC. So the idea was matching
> AVVAAPIHardwareContext with vaapi_context, which is why the first
> member of AVVAAPIHardwareContext can't be AVClass. I think trying to
> somehow make these structs compatible should be given up, and instead
> the struct should have a vaapi_context pointer field instead.
Yes.
> Also, I'm very worried about how this patch in general with combine
> with the API Libav is planning to add:
>
> https://lists.libav.org/pipermail/libav-devel/2016-January/074490.html
>
> To make this clear, above patch series:
> - extends the buffer pool to make it useful for the hwdec case (where
> you need to free the hw context when the final AVFrame is unreffed)
> - create a common API for hwaccel contexts, which overlaps with this
> patch to some degree, but not fully
> - adds a hwaccel context field to AVFrame
> - the hwaccel context is refcounted
> - adds helper for reading back data from hwaccel AVFrames in an
> API-independent way
>
> I'm not sure what to make out of this mess. Any ideas? I don't really
> want to hold this patch series back either. There is not much reason to
> refactor the whole API 1 week after it has been added either.
I'm not sure how to respond usefully to all of that, it adds a lot of new things all over the place.
It looks like it has most of the elements needed here, though I would need to examine it more carefully to be sure. (I don't see a way to do the "enumerate all of a set of frames for something which requires all render targets in advance" operation, but maybe I haven't looked hard enough.)
>> +
>> +
>> +AVVAAPIHardwareContext *av_vaapi_alloc_hardware_context(void)
>> +{
>> + return av_mallocz(sizeof(AVVAAPIHardwareContext));
>> +}
>> +
>> +void av_vaapi_lock_hardware_context(AVVAAPIHardwareContext *ctx)
>> +{
>> + if(ctx->lock)
>> + ctx->lock(ctx->lock_user_context);
>> +}
>> +
>> +void av_vaapi_unlock_hardware_context(AVVAAPIHardwareContext *ctx)
>> +{
>> + if(ctx->unlock)
>> + ctx->unlock(ctx->lock_user_context);
>> +}
>> +
>> +
>> +typedef struct AVVAAPISurface {
>> + VASurfaceID id;
>> + AVVAAPIHardwareContext *hardware_context;
>> +
>> + VAImage image;
>> + void *mapped_address;
>> +} AVVAAPISurface;
>> +
>> +static AVVAAPISurface *vaapi_get_surface(const AVFrame *frame)
>> +{
>> + av_assert0(frame);
>> + av_assert0(frame->buf[0]);
>> + av_assert0(frame->buf[0]->data);
>> + return (AVVAAPISurface*)frame->buf[0]->data;
>> +}
>> +
>> +static AVVAAPISurfaceConfig *vaapi_get_surface_config(const AVFrame *frame)
>> +{
>> + av_assert0(frame);
>> + av_assert0(frame->buf[1]);
>> + av_assert0(frame->buf[1]->data);
>> + return (AVVAAPISurfaceConfig*)frame->buf[1]->data;
>> +}
>> +
>> +static void vaapi_surface_free(void *opaque, uint8_t *data)
>> +{
>> + AVVAAPISurface *surface = (AVVAAPISurface*)data;
>> + AVVAAPIHardwareContext *hw_ctx = surface->hardware_context;
>> + VAStatus vas;
>> +
>> + av_vaapi_lock_hardware_context(hw_ctx);
>> +
>> + vas = vaDestroySurfaces(surface->hardware_context->display,
>> + &surface->id, 1);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(&vaapi_log, AV_LOG_ERROR, "Failed to destroy surface: "
>> + "%d (%s).\n", vas, vaErrorStr(vas));
>> + }
>> +
>> + av_free(surface);
>> +
>> + av_vaapi_unlock_hardware_context(hw_ctx);
>> +}
>> +
>> +int av_vaapi_surface_pool_init(AVVAAPISurfacePool *pool,
>> + AVVAAPIHardwareContext *hw_ctx,
>> + AVVAAPISurfaceConfig *config,
>> + int frame_count)
>> +{
>> + AVBufferRef *config_buffer;
>> + AVVAAPISurface *surface;
>> + AVFrame *frame;
>> + VASurfaceID surface_id;
>> + VAStatus vas;
>> + int i, err;
>> +
>> + memset(pool, 0, sizeof(*pool));
>> +
>> + pool->hardware_context = hw_ctx;
>> + pool->frame_count = frame_count;
>> +
>> + config_buffer = av_buffer_alloc(sizeof(*config));
>> + if(!config_buffer)
>> + return AVERROR(ENOMEM);
>> + memcpy(config_buffer->data, config, sizeof(*config));
>> + config = (AVVAAPISurfaceConfig*)config_buffer->data;
>> +
>> + av_vaapi_lock_hardware_context(hw_ctx);
>> +
>> + for(i = 0; i < frame_count; i++) {
>> + frame = av_frame_alloc();
>> + if(!frame) {
>> + err = AVERROR(ENOMEM);
>> + goto fail;
>> + }
>> + surface = av_mallocz(sizeof(*surface));
>> + if(!surface) {
>> + err = AVERROR(ENOMEM);
>> + goto fail;
>> + }
>> +
>> + surface->hardware_context = hw_ctx;
>> +
>> + vas = vaCreateSurfaces(hw_ctx->display, config->rt_format,
>> + config->width, config->height,
>> + &surface_id, 1,
>> + config->attributes, config->attribute_count);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(&vaapi_log, AV_LOG_ERROR, "Failed to create surface: "
>> + "%d (%s).\n", vas, vaErrorStr(vas));
>> + err = AVERROR_EXTERNAL;
>> + goto fail;
>> + }
>> +
>> + surface->id = surface_id;
>> + frame->buf[0] = av_buffer_create((uint8_t*)surface, sizeof(*surface),
>> + &vaapi_surface_free,
>> + 0, AV_BUFFER_FLAG_READONLY);
>> + if(!frame->buf[0]) {
>> + err = AVERROR(ENOMEM);
>> + goto fail;
>> + }
>> + surface = 0;
>> +
>> + frame->buf[1] = av_buffer_ref(config_buffer);
>> + if(!frame->buf[1]) {
>> + err = AVERROR(ENOMEM);
>> + goto fail;
>> + }
>> +
>> + frame->data[3] = (uint8_t*)(uintptr_t)surface_id;
>> +
>> + frame->format = AV_PIX_FMT_VAAPI;
>> + frame->width = config->width;
>> + frame->height = config->height;
>> +
>> + pool->frames[i] = frame;
>> + frame = 0;
>> + }
>> +
>> + for(; i < FF_ARRAY_ELEMS(pool->frames); i++)
>> + pool->frames[i] = 0;
>> +
>> + av_buffer_unref(&config_buffer);
>> +
>> + av_log(&vaapi_log, AV_LOG_DEBUG, "Surface pool initialised: %u surfaces "
>> + "of %ux%u.\n", pool->frame_count, config->width, config->height);
>> +
>> + err = 0;
>> + if(0) {
>> + fail:
>
> This looks funny. My personal preference is avoiding such tricks, but
> maybe it's not important.
I think this unwinding was more complex at some previous iteration. The failure case can be moved below the return now, though I still think it's nice to have exactly one lock/unlock pair to be sure that that is correct.
>> + if(surface) {
>> + if(surface->id)
>> + vaDestroySurfaces(hw_ctx->display, &surface->id, 1);
>> + av_free(surface);
>> + }
>> + if(frame)
>> + av_frame_free(&frame);
>> + for(--i; i >= 0; i--)
>> + av_frame_free(&pool->frames[i]);
>> + }
>> +
>> + av_vaapi_unlock_hardware_context(hw_ctx);
>> + return err;
>> +}
>> +
>> +int av_vaapi_surface_pool_uninit(AVVAAPISurfacePool *pool)
>> +
>> +{
>> + int i;
>> +
>> + av_vaapi_lock_hardware_context(pool->hardware_context);
>> +
>> + for(i = 0; i < FF_ARRAY_ELEMS(pool->frames); i++) {
>> + if(pool->frames[i])
>> + av_frame_free(&pool->frames[i]);
>> + }
>> +
>> + av_vaapi_unlock_hardware_context(pool->hardware_context);
>> +
>> + return 0;
>> +}
>> +
>> +int av_vaapi_surface_pool_get(AVVAAPISurfacePool *pool, AVFrame *target)
>> +{
>> + AVFrame *frame = 0;
>> + int i, err;
>> +
>> + av_vaapi_lock_hardware_context(pool->hardware_context);
>> +
>> + for(i = 0; i < FF_ARRAY_ELEMS(pool->frames); i++) {
>> + if(!pool->frames[i])
>> + break;
>> +
>> + if(av_buffer_get_ref_count(pool->frames[i]->buf[0]) == 1) {
>> + frame = pool->frames[i];
>> + break;
>> + }
>> + }
>> +
>> + if(frame) {
>> + target->data[3] = frame->data[3];
>> + target->buf[0] = av_buffer_ref(frame->buf[0]);
>> + target->buf[1] = av_buffer_ref(frame->buf[1]);
>> + if(!target->buf[0] || !target->buf[1])
>> + err = AVERROR(ENOMEM);
>> + else
>> + err = 0;
>> + } else {
>> + err = AVERROR(ENOMEM);
>> + }
>> +
>> + av_vaapi_unlock_hardware_context(pool->hardware_context);
>> +
>> + return err;
>> +}
>> +
>> +int av_vaapi_map_frame(AVFrame *frame, int get)
>> +{
>> + AVVAAPISurface *surface = vaapi_get_surface(frame);
>> + AVVAAPISurfaceConfig *config = vaapi_get_surface_config(frame);
>> + AVVAAPIHardwareContext *hw_ctx = surface->hardware_context;
>> + VAImage *image = &surface->image;
>> + VAStatus vas;
>> + int i, err;
>> + void *address;
>> + // On current Intel drivers, derive gives you memory which is very slow
>> + // to read (uncached?). It can be better for write-only cases, but for
>> + // now play it safe and never use derive.
>> + int derive = 0;
>
> Before the patch gets applied, we also should check if we can get in a
> "GPU memcpy", and whether this codepath can be enabled. If not, there
> wouldn't be much of a reason to keep it.
Even without it, it has use for the "drawing on frames" case (subtitles?). I haven't studied how that works in libavfilter at all, but I imagine we want the derive case there to avoid a pointless copy there and back to speed up changing only a few pixel values.
I guess it could go for now and be added back if/when someone implements that, though.
>> +
>> + if(surface->mapped_address) {
>> + av_log(&vaapi_log, AV_LOG_ERROR, "Surface %#x already mapped.\n",
>> + surface->id);
>> + return AVERROR(EINVAL);
>> + }
>> +
>> + av_vaapi_lock_hardware_context(hw_ctx);
>> +
>> + vas = vaSyncSurface(hw_ctx->display, surface->id);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(&vaapi_log, AV_LOG_ERROR, "Failed to sync surface "
>> + "%#x: %d (%s).\n", surface->id, vas, vaErrorStr(vas));
>> + err = AVERROR_EXTERNAL;
>> + goto fail;
>> + }
>> +
>> + if(derive) {
>> + vas = vaDeriveImage(hw_ctx->display,
>> + surface->id, image);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(&vaapi_log, AV_LOG_VERBOSE, "Failed to derive image from "
>> + "surface %#x: %d (%s).\n",
>> + surface->id, vas, vaErrorStr(vas));
>> + derive = 0;
>> + }
>> + }
>> + if(!derive) {
>> + vas = vaCreateImage(hw_ctx->display, &config->image_format,
>> + config->width, config->height, image);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(&vaapi_log, AV_LOG_ERROR, "Failed to create image for "
>> + "surface %#x: %d (%s).\n",
>> + surface->id, vas, vaErrorStr(vas));
>> + err = AVERROR_EXTERNAL;
>> + goto fail;
>> + }
>> +
>> + if(get) {
>> + vas = vaGetImage(hw_ctx->display, surface->id, 0, 0,
>> + config->width, config->height, image->image_id);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(&vaapi_log, AV_LOG_ERROR, "Failed to get image for "
>> + "surface %#x: %d (%s).\n",
>> + surface->id, vas, vaErrorStr(vas));
>> + err = AVERROR_EXTERNAL;
>> + goto fail_image;
>> + }
>> + }
>> + }
>> +
>> + if(image->format.fourcc != config->image_format.fourcc) {
>> + av_log(&vaapi_log, AV_LOG_VERBOSE, "Returned image in unexpected "
>> + "format: asked for %#x, got %#x.\n",
>> + config->image_format.fourcc, image->format.fourcc);
>> + }
>> +
>> + vas = vaMapBuffer(hw_ctx->display, image->buf, &address);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(&vaapi_log, AV_LOG_ERROR, "Failed to map image from surface "
>> + "%#x: %d (%s).\n", surface->id, vas, vaErrorStr(vas));
>> + err = AVERROR_EXTERNAL;
>> + goto fail_image;
>> + }
>> +
>> + surface->mapped_address = address;
>> +
>> + for(i = 0; i < image->num_planes; i++) {
>> + frame->data[i] = (uint8_t*)address + image->offsets[i];
>> + frame->linesize[i] = image->pitches[i];
>> + }
>> +
>> + if(image->format.fourcc == VA_FOURCC_YV12) {
>> + uint8_t *tmp;
>> + // Chroma planes are YVU rather than YUV, so swap them.
>> + tmp = frame->data[1];
>> + frame->data[1] = frame->data[2];
>> + frame->data[2] = tmp;
>> + }
>> +
>> + av_vaapi_unlock_hardware_context(hw_ctx);
>> + return 0;
>> +
>> + fail_image:
>> + vas = vaDestroyImage(hw_ctx->display, surface->image.image_id);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(&vaapi_log, AV_LOG_ERROR, "Failed to destroy image for surface "
>> + "%#x: %d (%s).\n", surface->id, vas, vaErrorStr(vas));
>> + }
>> + fail:
>> + av_vaapi_unlock_hardware_context(hw_ctx);
>> + return err;
>> +}
>> +
>> +int av_vaapi_unmap_frame(AVFrame *frame, int put)
>> +{
>> + AVVAAPISurface *surface = vaapi_get_surface(frame);
>> + AVVAAPISurfaceConfig *config = vaapi_get_surface_config(frame);
>> + AVVAAPIHardwareContext *hw_ctx = surface->hardware_context;
>> + VAImage *image = &surface->image;
>> + VAStatus vas;
>> + int i;
>> + int derive = 0;
>> +
>> + surface->mapped_address = 0;
>> +
>> + for(i = 0; i < image->num_planes; i++) {
>> + frame->data[i] = 0;
>> + frame->linesize[i] = 0;
>> + }
>> +
>> + vas = vaUnmapBuffer(hw_ctx->display, image->buf);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(&vaapi_log, AV_LOG_ERROR, "Failed to unmap image from surface "
>> + "%#x: %d (%s).\n", surface->id, vas, vaErrorStr(vas));
>> + }
>> +
>> + if(!derive && put) {
>> + vas = vaPutImage(hw_ctx->display, surface->id, image->image_id,
>> + 0, 0, config->width, config->height,
>> + 0, 0, config->width, config->height);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(&vaapi_log, AV_LOG_ERROR, "Failed to put image for surface "
>> + "%#x: %d (%s).\n", surface->id, vas, vaErrorStr(vas));
>> + }
>> + }
>> +
>> + vas = vaDestroyImage(hw_ctx->display,
>> + surface->image.image_id);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(&vaapi_log, AV_LOG_ERROR, "Failed to destroy image for surface "
>> + "%#x: %d (%s).\n", surface->id, vas, vaErrorStr(vas));
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static AVFrame *vaapi_make_proxy_frame(const AVFrame *src)
>> +{
>> + AVVAAPISurface *surface = vaapi_get_surface(src);
>> + VAImage *image = &surface->image;
>> + AVFrame *dst;
>> + int i;
>> +
>> + if(!surface->mapped_address) {
>> + av_log(&vaapi_log, AV_LOG_ERROR, "Surface %#x is not mapped.",
>> + surface->id);
>> + return 0;
>> + }
>> +
>> + dst = av_frame_alloc();
>> + if(!dst)
>> + return 0;
>> +
>> + for(i = 0; i < image->num_planes; i++) {
>> + dst->data[i] = src->data[i];
>> + dst->linesize[i] = src->linesize[i];
>> + }
>> +
>> + dst->format = av_vaapi_pix_fmt(image->format.fourcc);
>> + dst->width = src->width;
>> + dst->height = src->height;
>> +
>> + av_frame_copy_props(dst, src);
>> +
>> + return dst;
>> +}
>> +
>> +int av_vaapi_copy_to_surface(AVFrame *dst, const AVFrame *src)
>> +{
>> + AVFrame *proxy;
>> + int err;
>> +
>> + if(dst->format != AV_PIX_FMT_VAAPI)
>> + return AVERROR(EINVAL);
>> +
>> + err = av_vaapi_map_frame(dst, 0);
>> + if(err < 0)
>> + return err;
>> +
>> + proxy = vaapi_make_proxy_frame(dst);
>> + if(proxy)
>> + err = av_frame_copy(proxy, src);
>> + else
>> + err = AVERROR(ENOMEM);
>> +
>> + av_vaapi_unmap_frame(dst, 1);
>> +
>> + return 0;
>> +}
>> +
>> +int av_vaapi_copy_from_surface(AVFrame *dst, AVFrame *src)
>> +{
>> + AVFrame *proxy;
>> + int err;
>> +
>> + if(src->format != AV_PIX_FMT_VAAPI)
>> + return AVERROR(EINVAL);
>> +
>> + err = av_vaapi_map_frame(src, 1);
>> + if(err < 0)
>> + return err;
>> +
>> + proxy = vaapi_make_proxy_frame(src);
>> + if(proxy)
>> + err = av_frame_copy(dst, proxy);
>> + else
>> + err = AVERROR(ENOMEM);
>> +
>> + av_vaapi_unmap_frame(src, 0);
>> +
>> + av_frame_free(&proxy);
>> +
>> + return err;
>> +}
>> +
>> +int av_vaapi_pipeline_init(AVVAAPIPipelineContext *ctx,
>> + AVVAAPIHardwareContext *hw_ctx,
>> + AVVAAPIPipelineConfig *config,
>> + AVVAAPISurfacePool *pool)
>> +{
>> + VASurfaceID output_surface_ids[AV_VAAPI_MAX_SURFACES];
>> + int output_surface_count;
>> + VAStatus vas;
>> + int i, err;
>> + int attr_count;
>> + VASurfaceAttrib *attr_list;
>> + int min_width, max_width, min_height, max_height;
>> +
>> + av_vaapi_lock_hardware_context(hw_ctx);
>> +
>> + memset(ctx, 0, sizeof(*ctx));
>> + ctx->class = &vaapi_class;
>> +
>> + ctx->hardware_context = hw_ctx;
>> +
>> + if(pool) {
>> + output_surface_count = pool->frame_count;
>> + for(i = 0; i < output_surface_count; i++)
>> + output_surface_ids[i] = vaapi_get_surface(pool->frames[i])->id;
>> + } else {
>> + // An output surface pool need not be supplied if the pipeline
>> + // produces no image output (an intra-only codec like JPEG, say).
>> +
>> + output_surface_count = 0;
>> + }
>> +
>> + vas = vaCreateConfig(hw_ctx->display, config->profile,
>> + config->entrypoint, config->attributes,
>> + config->attribute_count, &ctx->config_id);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(ctx, AV_LOG_ERROR, "Failed to create pipeline configuration: "
>> + "%d (%s).\n", vas, vaErrorStr(vas));
>> + err = AVERROR_EXTERNAL;
>> + goto fail;
>> + }
>> +
>> + attr_count = 0;
>> + vas = vaQuerySurfaceAttributes(hw_ctx->display, ctx->config_id,
>> + 0, &attr_count);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(ctx, AV_LOG_ERROR, "Failed to get surface attribute count: "
>> + "%d (%s).\n", vas, vaErrorStr(vas));
>> + err = AVERROR_EXTERNAL;
>> + goto fail_config;
>> + }
>> +
>> + attr_list = av_calloc(attr_count, sizeof(VASurfaceAttrib));
>> + if(!attr_list) {
>> + err = AVERROR(ENOMEM);
>> + goto fail_config;
>> + }
>> + min_width = min_height = 0; // Min sizes need not be returned.
>> + max_width = max_height = INT_MIN; // Max sizes should be.
>> +
>> + vas = vaQuerySurfaceAttributes(hw_ctx->display, ctx->config_id,
>> + attr_list, &attr_count);
>> + if(vas != VA_STATUS_SUCCESS)
>> + attr_count = 0;
>> + for(i = 0; i < attr_count; i++) {
>> + switch(attr_list[i].type) {
>> + case VASurfaceAttribMinWidth:
>> + min_width = attr_list[i].value.value.i;
>> + break;
>> + case VASurfaceAttribMaxWidth:
>> + max_width = attr_list[i].value.value.i;
>> + break;
>> + case VASurfaceAttribMinHeight:
>> + min_height = attr_list[i].value.value.i;
>> + break;
>> + case VASurfaceAttribMaxHeight:
>> + max_height = attr_list[i].value.value.i;
>> + break;
>> + }
>> + }
>> + av_free(attr_list);
>> + if(attr_count == 0) {
>> + av_log(ctx, AV_LOG_ERROR, "Failed to get surface attributes: "
>> + "%d (%s).\n", vas, vaErrorStr(vas));
>> + err = AVERROR_EXTERNAL;
>> + goto fail_config;
>> + }
>> + if(min_width > config->width || max_width < config->width ||
>> + min_height > config->height || max_height < config->height) {
>> + av_log(ctx, AV_LOG_ERROR, "Pipeline does not support "
>> + "image size %dx%d (width %d-%d height %d-%d).\n",
>> + config->width, config->height,
>> + min_width, max_width, min_height, max_height);
>> + err = AVERROR(EINVAL);
>> + goto fail_config;
>> + }
>> +
>> + vas = vaCreateContext(hw_ctx->display, ctx->config_id,
>> + config->width, config->height,
>> + VA_PROGRESSIVE,
>> + output_surface_ids, output_surface_count,
>> + &ctx->context_id);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(ctx, AV_LOG_ERROR, "Failed to create pipeline context: "
>> + "%d (%s).\n", vas, vaErrorStr(vas));
>> + err = AVERROR_EXTERNAL;
>> + goto fail_config;
>> + }
>> +
>> + av_log(ctx, AV_LOG_DEBUG, "VAAPI pipeline initialised: config %#x "
>> + "context %#x.\n", ctx->config_id, ctx->context_id);
>> +
>> + err = 0;
>> + if(0) {
>> + fail_config:
>> + vaDestroyConfig(hw_ctx->display, ctx->config_id);
>> + }
>> + fail:
>> + av_vaapi_unlock_hardware_context(hw_ctx);
>> + return err;
>> +}
>> +
>> +int av_vaapi_pipeline_uninit(AVVAAPIPipelineContext *ctx)
>> +{
>> + VAStatus vas;
>> +
>> + av_vaapi_lock_hardware_context(ctx->hardware_context);
>> +
>> + av_assert0(ctx->hardware_context);
>> +
>> + vas = vaDestroyContext(ctx->hardware_context->display, ctx->context_id);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(ctx, AV_LOG_ERROR, "Failed to destroy pipeline context: "
>> + "%d (%s).\n", vas, vaErrorStr(vas));
>> + }
>> +
>> + vaDestroyConfig(ctx->hardware_context->display, ctx->config_id);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(ctx, AV_LOG_ERROR, "Failed to destroy pipeline configuration: "
>> + "%d (%s).\n", vas, vaErrorStr(vas));
>> + }
>> +
>> + av_vaapi_unlock_hardware_context(ctx->hardware_context);
>> +
>> + return 0;
>> +}
>> +
>> +
>> +static struct {
>> + unsigned int fourcc;
>> + enum AVPixelFormat pix_fmt;
>> +} vaapi_formats[] = {
>> +#define MAP(va, av) { VA_FOURCC_ ## va, AV_PIX_FMT_ ## av }
>> + MAP(NV12, NV12),
>> + MAP(IYUV, YUV420P),
>> + MAP(YV12, YUV420P), // With U/V planes swapped.
>> + MAP(YV16, YUV422P),
>> + MAP(UYVY, UYVY422),
>> + MAP(P010, P010),
>
> Heh, nice addition.
>
>> + MAP(Y800, GRAY8),
>> + MAP(BGRA, BGRA),
>> + MAP(BGRX, BGR0),
>> + MAP(RGBA, RGBA),
>> + MAP(RGBX, RGB0),
>> +#undef MAP
>> +};
>> +
>> +enum AVPixelFormat av_vaapi_pix_fmt(unsigned int fourcc)
>> +{
>> + int i;
>> + for(i = 0; i < FF_ARRAY_ELEMS(vaapi_formats); i++)
>> + if(vaapi_formats[i].fourcc == fourcc)
>> + return vaapi_formats[i].pix_fmt;
>> + return AV_PIX_FMT_NONE;
>> +}
>> +
>> +unsigned int av_vaapi_fourcc(enum AVPixelFormat pix_fmt)
>> +{
>> + int i;
>> + for(i = 0; i < FF_ARRAY_ELEMS(vaapi_formats); i++)
>> + if(vaapi_formats[i].pix_fmt == pix_fmt)
>> + return vaapi_formats[i].fourcc;
>> + return 0;
>> +}
>> +
>> +int av_vaapi_get_image_format(AVVAAPIHardwareContext *hw_ctx,
>> + enum AVPixelFormat pix_fmt,
>> + VAImageFormat *image_format)
>> +{
>> + VAStatus vas;
>> + VAImageFormat *format_list;
>> + int format_count, i, err;
>> +
>> + av_vaapi_lock_hardware_context(hw_ctx);
>> +
>> + format_count = vaMaxNumImageFormats(hw_ctx->display);
>> + if(format_count <= 0) {
>> + err = AVERROR_EXTERNAL;
>> + goto fail;
>> + }
>> +
>> + format_list = av_calloc(format_count, sizeof(VAImageFormat));
>> + if(!format_list) {
>> + err = AVERROR(ENOMEM);
>> + goto fail;
>> + }
>> +
>> + vas = vaQueryImageFormats(hw_ctx->display, format_list, &format_count);
>> + if(vas != VA_STATUS_SUCCESS) {
>> + av_log(&vaapi_log, AV_LOG_ERROR, "Failed to enumerate VAAPI image "
>> + "formats: %d (%s).\n", vas, vaErrorStr(vas));
>> + err = AVERROR_EXTERNAL;
>> + goto fail;
>> + }
>> +
>> + for(i = 0; i < format_count; i++) {
>> + if(av_vaapi_pix_fmt(format_list[i].fourcc) == pix_fmt) {
>> + memcpy(image_format, &format_list[i], sizeof(VAImageFormat));
>> + break;
>> + }
>> + }
>> +
>> + if(i < format_count)
>> + err = 0;
>> + else
>> + err = AVERROR(EINVAL);
>> + fail:
>> + av_vaapi_unlock_hardware_context(hw_ctx);
>> + return err;
>> +}
>> diff --git a/libavcodec/vaapi_support.h b/libavcodec/vaapi_support.h
>> new file mode 100644
>> index 0000000..fb1c1b3
>> --- /dev/null
>> +++ b/libavcodec/vaapi_support.h
>> @@ -0,0 +1,122 @@
>> +/*
>> + * VAAPI helper functions.
>> + *
>> + * Copyright (C) 2016 Mark Thompson <mrt at jkqxz.net>
>> + *
>> + * This file is part of FFmpeg.
>> + *
>> + * FFmpeg is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU Lesser General Public
>> + * License as published by the Free Software Foundation; either
>> + * version 2.1 of the License, or (at your option) any later version.
>> + *
>> + * FFmpeg is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
>> + * Lesser General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU Lesser General Public
>> + * License along with FFmpeg; if not, write to the Free Software
>> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
>> + */
>> +
>> +#ifndef LIBAVUTIL_VAAPI_H_
>> +#define LIBAVUTIL_VAAPI_H_
>> +
>> +#include <va/va.h>
>> +
>> +#include "libavutil/pixfmt.h"
>> +#include "libavutil/frame.h"
>> +
>> +
>> +typedef struct AVVAAPIHardwareContext {
>> + VADisplay display;
>> +
>> + VAConfigID decoder_pipeline_config_id;
>> + VAContextID decoder_pipeline_context_id;
>> +
>> + void (*lock)(void *user_context);
>> + void (*unlock)(void *user_context);
>> + void *lock_user_context;
>> +} AVVAAPIHardwareContext;
>> +
>> +AVVAAPIHardwareContext *av_vaapi_alloc_hardware_context(void);
>> +
>> +void av_vaapi_lock_hardware_context(AVVAAPIHardwareContext *ctx);
>> +void av_vaapi_unlock_hardware_context(AVVAAPIHardwareContext *ctx);
>> +
>> +
>> +#define AV_VAAPI_MAX_SURFACES 64
>> +
>> +typedef struct AVVAAPISurfaceConfig {
>> + enum AVPixelFormat av_format;
>> + unsigned int rt_format;
>> + VAImageFormat image_format;
>> +
>> + unsigned int width;
>> + unsigned int height;
>> +
>> + unsigned int attribute_count;
>> + VASurfaceAttrib *attributes;
>> +} AVVAAPISurfaceConfig;
>> +
>> +typedef struct AVVAAPISurfacePool {
>> + AVVAAPIHardwareContext *hardware_context;
>> +
>> + int frame_count;
>> + AVFrame *frames[AV_VAAPI_MAX_SURFACES];
>> +} AVVAAPISurfacePool;
>> +
>> +int av_vaapi_surface_pool_init(AVVAAPISurfacePool *pool,
>> + AVVAAPIHardwareContext *hw_ctx,
>> + AVVAAPISurfaceConfig *config,
>> + int frame_count);
>> +
>> +int av_vaapi_surface_pool_uninit(AVVAAPISurfacePool *pool);
>> +
>> +int av_vaapi_surface_pool_get(AVVAAPISurfacePool *pool, AVFrame *target);
>> +
>> +
>> +typedef struct AVVAAPIPipelineConfig {
>> + VAProfile profile;
>> + VAEntrypoint entrypoint;
>> +
>> + unsigned int width;
>> + unsigned int height;
>> +
>> + unsigned int attribute_count;
>> + VAConfigAttrib *attributes;
>> +} AVVAAPIPipelineConfig;
>> +
>> +typedef struct AVVAAPIPipelineContext {
>> + const AVClass *class;
>> +
>> + AVVAAPIHardwareContext *hardware_context;
>> +
>> + VAConfigID config_id;
>> + VAContextID context_id;
>> +} AVVAAPIPipelineContext;
>> +
>> +int av_vaapi_pipeline_init(AVVAAPIPipelineContext *ctx,
>> + AVVAAPIHardwareContext *hw_ctx,
>> + AVVAAPIPipelineConfig *config,
>> + AVVAAPISurfacePool *pool);
>> +
>> +int av_vaapi_pipeline_uninit(AVVAAPIPipelineContext *ctx);
>> +
>> +
>> +int av_vaapi_map_frame(AVFrame *frame, int get);
>> +int av_vaapi_unmap_frame(AVFrame *frame, int put);
>> +
>> +int av_vaapi_copy_to_surface(AVFrame *dst, const AVFrame *src);
>> +int av_vaapi_copy_from_surface(AVFrame *dst, AVFrame *src);
>> +
>> +
>> +enum AVPixelFormat av_vaapi_pix_fmt(unsigned int fourcc);
>> +unsigned int av_vaapi_fourcc(enum AVPixelFormat pix_fmt);
>> +
>> +int av_vaapi_get_image_format(AVVAAPIHardwareContext *hw_ctx,
>> + enum AVPixelFormat pix_fmt,
>> + VAImageFormat *image_format);
>> +
>> +#endif /* LIBAVUTIL_VAAPI_H_ */
>
> Looks in general pretty ok to me.
>
> There should be allocation functions for structs which don't have one
> yet, and a clear warning should be added to the doxygen. (Doxygen
> should also be added.)
If you're happy that this is now plausible, then I will write documentation for the relevant parts.
> One nit: I _think_ we usually add a whitespace between statement and
> "(", e.g. "if (". (Not for function calls or declarations.)
I can sed the spaces in, I guess.
Thanks for the review!
- Mark
More information about the ffmpeg-devel
mailing list