[FFmpeg-cvslog] avformat/concatdec: fix the h264 annexb extradata check

[James Almer]

ffmpeg | branch: release/3.3 | James Almer <jamrial at gmail.com> | Tue Apr 25 20:23:12 2017 -0300| [cfca0b91399bfc72218bd47412bc8e57c19742b9] | committer: James Almer

avformat/concatdec: fix the h264 annexb extradata check

The start code can be either in the first three or four bytes.

(cherry picked from commit b4330a0e02fcbef61d630a369abe5f4421ced659)

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=cfca0b91399bfc72218bd47412bc8e57c19742b9
---

 libavformat/concatdec.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/libavformat/concatdec.c b/libavformat/concatdec.c
index 8649916ff2..dd52e4d366 100644
--- a/libavformat/concatdec.c
+++ b/libavformat/concatdec.c
@@ -199,8 +199,11 @@ static int detect_stream_specific(AVFormatContext *avf, int idx)
     AVBitStreamFilterContext *bsf;
     int ret;
 
-    if (cat->auto_convert && st->codecpar->codec_id == AV_CODEC_ID_H264 &&
-        (st->codecpar->extradata_size < 4 || AV_RB32(st->codecpar->extradata) != 1)) {
+    if (cat->auto_convert && st->codecpar->codec_id == AV_CODEC_ID_H264) {
+        if (!st->codecpar->extradata_size                                                ||
+            (st->codecpar->extradata_size >= 3 && AV_RB24(st->codecpar->extradata) == 1) ||
+            (st->codecpar->extradata_size >= 4 && AV_RB32(st->codecpar->extradata) == 1))
+            return 0;
         av_log(cat->avf, AV_LOG_INFO,
                "Auto-inserting h264_mp4toannexb bitstream filter\n");
         if (!(bsf = av_bitstream_filter_init("h264_mp4toannexb"))) {