[FFmpeg-cvslog] avcodec/vp3: Add missing check for av_malloc

[Jiasheng Jiang]

ffmpeg | branch: release/4.1 | Jiasheng Jiang <jiasheng at iscas.ac.cn> | Tue Feb 15 17:58:08 2022 +0800| [51efa68ec0b4f42b5b124b8987fb68f60a929c4f] | committer: James Almer

avcodec/vp3: Add missing check for av_malloc

Since the av_malloc() may fail and return NULL pointer,
it is needed that the 's->edge_emu_buffer' should be checked
whether the new allocation is success.

Fixes: d14723861b ("VP3: fix decoding of videos with stride > 2048")
Reviewed-by: Peter Ross <pross at xvid.org>
Signed-off-by: Jiasheng Jiang <jiasheng at iscas.ac.cn>
(cherry picked from commit 656cb0450aeb73b25d7d26980af342b37ac4c568)

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=51efa68ec0b4f42b5b124b8987fb68f60a929c4f
---

 libavcodec/vp3.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/libavcodec/vp3.c b/libavcodec/vp3.c
index 25d0bbb758..76d2ef70d6 100644
--- a/libavcodec/vp3.c
+++ b/libavcodec/vp3.c
@@ -2136,8 +2136,13 @@ static int vp3_decode_frame(AVCodecContext *avctx,
     if (ff_thread_get_buffer(avctx, &s->current_frame, AV_GET_BUFFER_FLAG_REF) < 0)
         goto error;
 
-    if (!s->edge_emu_buffer)
+    if (!s->edge_emu_buffer) {
         s->edge_emu_buffer = av_malloc(9 * FFABS(s->current_frame.f->linesize[0]));
+        if (!s->edge_emu_buffer) {
+            ret = AVERROR(ENOMEM);
+            goto error;
+        }
+    }
 
     if (s->keyframe) {
         if (!s->theora) {