[FFmpeg-devel] [PATCH v2 2/2] lavfi/vaapi_vpp: Use dynamic frame pool in outlink if possible

[Xiang, Haihao]

From: Haihao Xiang <haihao.xiang at intel.com>

This can avoid to exhaust the buffers within outlink when libva2 is
available.

For example:
$ ffmpeg -hwaccel_output_format vaapi -hwaccel vaapi -i input.mp4 \
-vf 'scale_vaapi=w=720:h=480' -c:v hevc_vaapi -f null -
...
[vf#0:0 @ 0x55acad91f400] Error while filtering: Cannot allocate memory
[vf#0:0 @ 0x55acad91f400] Task finished with error code: -12 (Cannot
allocate memory)
[vf#0:0 @ 0x55acad91f400] Terminating thread with return code -12
(Cannot allocate memory)

Signed-off-by: Haihao Xiang <haihao.xiang at intel.com>
---
 libavfilter/vaapi_vpp.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/libavfilter/vaapi_vpp.c b/libavfilter/vaapi_vpp.c
index ace1153a23..9ef7a289fb 100644
--- a/libavfilter/vaapi_vpp.c
+++ b/libavfilter/vaapi_vpp.c
@@ -204,7 +204,10 @@ int ff_vaapi_vpp_config_output(AVFilterLink *outlink)
     output_frames->width     = ctx->output_width;
     output_frames->height    = ctx->output_height;
 
-    output_frames->initial_pool_size = 4;
+    if (CONFIG_VAAPI_1)
+        output_frames->initial_pool_size = 0;
+    else
+        output_frames->initial_pool_size = 4;
 
     err = ff_filter_init_hw_frames(avctx, outlink, 10);
     if (err < 0)
@@ -220,6 +223,8 @@ int ff_vaapi_vpp_config_output(AVFilterLink *outlink)
     va_frames = output_frames->hwctx;
 
     av_assert0(ctx->va_context == VA_INVALID_ID);
+    av_assert0(output_frames->initial_pool_size ||
+               (va_frames->surface_ids == NULL && va_frames->nb_surfaces == 0));
     vas = vaCreateContext(ctx->hwctx->display, ctx->va_config,
                           ctx->output_width, ctx->output_height,
                           VA_PROGRESSIVE,
-- 
2.34.1