[FFmpeg-cvslog] avcodec/extract_extradata: return an error when buffer allocation fails

[James Almer]

ffmpeg | branch: release/3.3 | James Almer <jamrial at gmail.com> | Wed Sep 13 17:03:56 2017 -0300| [9f7042f9cd0f4911bc5cebfcb4489bbd77826841] | committer: James Almer

avcodec/extract_extradata: return an error when buffer allocation fails

ret is 0 by default.

Reviewed-by: Mark Thompson <sw at jkqxz.net>
Signed-off-by: James Almer <jamrial at gmail.com>
(cherry picked from commit 7bae17e37ab63d1cfcea22c68c455f859db3663c)

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=9f7042f9cd0f4911bc5cebfcb4489bbd77826841
---

 libavcodec/extract_extradata_bsf.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/libavcodec/extract_extradata_bsf.c b/libavcodec/extract_extradata_bsf.c
index 4cd0ca1137..ed6509c681 100644
--- a/libavcodec/extract_extradata_bsf.c
+++ b/libavcodec/extract_extradata_bsf.c
@@ -101,14 +101,17 @@ static int extract_extradata_h2645(AVBSFContext *ctx, AVPacket *pkt,
 
         if (s->remove) {
             filtered_buf = av_buffer_alloc(pkt->size + AV_INPUT_BUFFER_PADDING_SIZE);
-            if (!filtered_buf)
+            if (!filtered_buf) {
+                ret = AVERROR(ENOMEM);
                 goto fail;
+            }
             filtered_data = filtered_buf->data;
         }
 
         extradata = av_malloc(extradata_size + AV_INPUT_BUFFER_PADDING_SIZE);
         if (!extradata) {
             av_buffer_unref(&filtered_buf);
+            ret = AVERROR(ENOMEM);
             goto fail;
         }