[FFmpeg-devel] [PATCH v2] fftools/ffmpeg_dec: Don't keep sending frame to filters in flushing

[Fei Wang]

Filter may has a limited frame pool size. Do not always send frame to
filters without reaping.

Fix the regression of commit 5fa00b38e6.

Example cmd:
$ ffmpeg -threads 20 -init_hw_device vaapi=hw:/dev/dri/renderD128   \
-hwaccel_output_format vaapi -hwaccel vaapi -i avc_1080P.mp4        \
-vf 'scale_vaapi=w=720:h=480' -f null -

Signed-off-by: Fei Wang <fei.w.wang at intel.com>
---
Change:
1. rebase to master.

 fftools/ffmpeg_dec.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c
index 85bf8dc536..a7ae51fed2 100644
--- a/fftools/ffmpeg_dec.c
+++ b/fftools/ffmpeg_dec.c
@@ -803,6 +803,12 @@ int dec_packet(InputStream *ist, const AVPacket *pkt, int no_eof)
         av_frame_unref(d->frame);
         if (ret < 0)
             goto finish;
+
+        // During flushing, break out to reap filter once send a frame to filters to
+        // avoid drain out filter's output frame pool. Especially for HW filters which
+        // always have a limited frame pool size.
+        if (!pkt)
+            return 0;
     }
 
 finish:
-- 
2.25.1