[FFmpeg-devel] [PATCH] libavcodec/hevc_mp4toannexb_bsf: insert extradata before non-AUD unit

[Xiang, Haihao]

From: Haihao Xiang <haihao.xiang at intel.com>

It is possible that an IRAP frame in input AVPacket contains VPS, SPS
and PPS, and these headers should take effect. However the prepended
extradata might override these headers. This patch inserts extradata
before non-AUD unit, hence VPS, SPS and PPS from the input AVPacket will
take effect if they are present.

This should fix #7799

Signed-off-by: Haihao Xiang <haihao.xiang at intel.com>
---
 libavcodec/hevc_mp4toannexb_bsf.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/libavcodec/hevc_mp4toannexb_bsf.c b/libavcodec/hevc_mp4toannexb_bsf.c
index 790dfb0394..77551ba221 100644
--- a/libavcodec/hevc_mp4toannexb_bsf.c
+++ b/libavcodec/hevc_mp4toannexb_bsf.c
@@ -124,6 +124,7 @@ static int hevc_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *out)
 
     int got_irap = 0;
     int i, ret = 0;
+    int prev_nalu_is_aud = 0, extradata_offset = 0;
 
     ret = ff_bsf_get_packet(ctx, &in);
     if (ret < 0)
@@ -169,14 +170,21 @@ static int hevc_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *out)
 
         prev_size = out->size;
 
+        if (prev_nalu_is_aud)
+            extradata_offset = prev_size;
+
         ret = av_grow_packet(out, 4 + nalu_size + extra_size);
         if (ret < 0)
             goto fail;
 
-        if (extra_size)
-            memcpy(out->data + prev_size, ctx->par_out->extradata, extra_size);
+        if (extra_size) {
+            memmove(out->data + extradata_offset + extra_size, out->data + extradata_offset, prev_size - extradata_offset);
+            memcpy(out->data + extradata_offset, ctx->par_out->extradata, extra_size);
+        }
+
         AV_WB32(out->data + prev_size + extra_size, 1);
         bytestream2_get_buffer(&gb, out->data + prev_size + 4 + extra_size, nalu_size);
+        prev_nalu_is_aud = nalu_type == HEVC_NAL_AUD;
     }
 
     ret = av_packet_copy_props(out, in);
-- 
2.17.1