[FFmpeg-devel] [PATCH 06/11] avcodec/h264_parser: Use smaller buffer

[Andreas Rheinhardt]

The H.264 parser currently allocates a buffer of the size of the input
frame for every input frame to store the unescaped RBSP of a NAL unit
(it is actually only used if a 0x03 escape is encountered at all).
For performance reasons only a small part of slices is actually
unescaped; and given that an unescaped NAL unit is only used once (if
ever), the really needed size of the buffer is the maximum of the
possible length of a NAL unit in front of the first slice and the length of
the part of the first slice that is actually unescaped.
For AVC (mp4/Matroska) content, the length of a NAL unit is known, so
that if the NAL units in front of the slices are small (as they usually
are), said maximum is small so that one can reduce memory usage by
allocating less.
For Annex B, this is unfortunately not so, because the code has to
assume the worst wrt the NAL unit sizes (it even assumes that an
access unit delimiter extends until the end of the input buffer).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at gmail.com>
---
 libavcodec/h264_parser.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/libavcodec/h264_parser.c b/libavcodec/h264_parser.c
index c200a2ab8e..5f7aea0c76 100644
--- a/libavcodec/h264_parser.c
+++ b/libavcodec/h264_parser.c
@@ -268,11 +268,6 @@ static inline int parse_nal_units(AVCodecParserContext *s,
     if (!buf_size)
         return 0;
 
-    av_fast_padded_malloc(&rbsp->rbsp_buffer, &rbsp->rbsp_buffer_alloc_size, buf_size);
-    if (!rbsp->rbsp_buffer)
-        return AVERROR(ENOMEM);
-    rbsp->rbsp_buffer_size = 0;
-
     buf_index     = 0;
     next_avc      = p->is_avc ? 0 : buf_size;
     for (;;) {
@@ -310,6 +305,12 @@ static inline int parse_nal_units(AVCodecParserContext *s,
             }
             break;
         }
+
+    av_fast_padded_malloc(&rbsp->rbsp_buffer, &rbsp->rbsp_buffer_alloc_size, src_length);
+    if (!rbsp->rbsp_buffer)
+        return AVERROR(ENOMEM);
+    rbsp->rbsp_buffer_size = 0;
+
         consumed = ff_h2645_extract_rbsp(buf + buf_index, src_length, rbsp, &nal, 1);
         if (consumed < 0)
             break;
-- 
2.21.0