[Libav-user] Possible memory leak in QSV encoders

[Mark Samuelson]

There was indeed a memory leak, which has been fixed.
https://github.com/intel-media-ci/ffmpeg/pull/497

Thanks everyone

From: Libav-user <libav-user-bounces at ffmpeg.org> On Behalf Of Mark Samuelson
Sent: Tuesday, January 18, 2022 9:33 AM
To: libav-user at ffmpeg.org
Subject: [Libav-user] Possible memory leak in QSV encoders

Hello,

I am using the qsv_h264 encoder, and on certain resolutions I get memory leaks.  This issue seems to be the same as https://trac.ffmpeg.org/ticket/8445, but that was closed with "I figured it out".

I create an AVFrame and use av_buffer_create() to wrap an existing buffer.  I provide a custom callback for freeing the data, but this callback is never called, which is were the memory leak is happening.

The leak only happens on certain resolutions, which makes sense stepping through the code.  In libavcodec/qsvenc.c submit_frame(), if ffmpeg makes a copy by calling av_frame_copy() on line 1518, everything is fine.  If ffmpeg doesn't need to copy, and calls av_frame_ref() on line 1524, then we run into the memory leak.

What I can't find is where ffmpeg is calling av_frame_unref to remove the reference it added on line 1524.  It seems like this should be happening in clear_unused_frames(), were av_frame_unref() is indeed called, but only for frames with format AV_PIX_FMT_QSV.  Removing that check on line 1430 does fix the memory leak.

Where is the AVBufferRef that is created in libavcodec/qsvenc.c submit_frame() on line 1524 freed?

Mark Samuelson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ffmpeg.org/pipermail/libav-user/attachments/20220120/2060e62a/attachment.htm>