[FFmpeg-cvslog] avcodec/decode: Check size before opening iconv

[Andreas Rheinhardt]

ffmpeg | branch: master | Andreas Rheinhardt <andreas.rheinhardt at gmail.com> | Thu Mar  4 09:36:15 2021 +0100| [a272f382d3eb143e9da99537aec25dbbe5778614] | committer: Andreas Rheinhardt

avcodec/decode: Check size before opening iconv

Avoids closing iconv when the size check fails.

Reviewed-by: James Almer <jamrial at gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at gmail.com>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=a272f382d3eb143e9da99537aec25dbbe5778614
---

 libavcodec/decode.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/libavcodec/decode.c b/libavcodec/decode.c
index db6ee9cb04..c976795311 100644
--- a/libavcodec/decode.c
+++ b/libavcodec/decode.c
@@ -884,18 +884,17 @@ static int recode_subtitle(AVCodecContext *avctx,
         return 0;
 
 #if CONFIG_ICONV
-    cd = iconv_open("UTF-8", avctx->sub_charenc);
-    av_assert0(cd != (iconv_t)-1);
-
     inb = inpkt->data;
     inl = inpkt->size;
 
     if (inl >= INT_MAX / UTF8_MAX_BYTES - AV_INPUT_BUFFER_PADDING_SIZE) {
         av_log(avctx, AV_LOG_ERROR, "Subtitles packet is too big for recoding\n");
-        ret = AVERROR(ENOMEM);
-        goto end;
+        return AVERROR(ERANGE);
     }
 
+    cd = iconv_open("UTF-8", avctx->sub_charenc);
+    av_assert0(cd != (iconv_t)-1);
+
     ret = av_new_packet(&tmp, inl * UTF8_MAX_BYTES);
     if (ret < 0)
         goto end;