[FFmpeg-devel] [PATCH] avformat/mpegts: use buffer pools for allocating packets

[Marton Balint]

On Sat, 4 Apr 2020, James Almer wrote:

> On 4/4/2020 11:47 AM, Marton Balint wrote:
>> This brings a performance improvement when demuxing files, most of the
>> improvement comes from buffer pooling unbound packets.
>
> Yes, as i mentioned in my last reply this was my experience as well.
>
>> 
>> time ffprobe -i samples/ffmpeg-bugs/trac/ticket6132/Samsung_HDR_-_Chasing_the_Light.ts -show_packets >/dev/null 2>&1
>> 
>> Before:
>>     real    0m1.967s
>>     user    0m1.471s
>>     sys     0m0.493s
>> 
>> After:
>>     real    0m1.497s
>>     user    0m1.364s
>>     sys     0m0.129s
>> 
>> Based on a patch of James Almer.
>> 
>> Signed-off-by: Marton Balint <cus at passwd.hu>
>> ---
>>  libavformat/mpegts.c | 23 +++++++++++++++++++----
>>  1 file changed, 19 insertions(+), 4 deletions(-)
>> 
>> diff --git a/libavformat/mpegts.c b/libavformat/mpegts.c
>> index 7f56bacb2c..fbafc1d257 100644
>> --- a/libavformat/mpegts.c
>> +++ b/libavformat/mpegts.c
>> @@ -170,6 +170,7 @@ struct MpegTSContext {
>>      int current_pid;
>>
>>      AVStream *epg_stream;
>> +    AVBufferPool* pools[32];
>
> Isn't 32 way more than required?

The number of required pools is 
log2(MAX_PES_PAYLOAD+AV_INPUT_BUFFER_PADDING_SIZE)+1, so currently 18, but 
since as far as I know there is no compile time log2 I did not want to 
limit it here so MAX_PES_PAYLOAD can still be changeable without adjusting 
this.

>
>>  };
>>
>>  #define MPEGTS_OPTIONS \
>> @@ -1103,6 +1104,18 @@ static int read_sl_header(PESContext *pes, SLConfigDescr *sl,
>>      return (get_bits_count(&gb) + 7) >> 3;
>>  }
>> 
>> +static AVBufferRef *buffer_pool_get(MpegTSContext *ts, int size)
>> +{
>> +    int index = av_log2(size + AV_INPUT_BUFFER_PADDING_SIZE);
>> +    if (!ts->pools[index]) {
>> +        int pool_size = FFMIN(MAX_PES_PAYLOAD + AV_INPUT_BUFFER_PADDING_SIZE, 2 << index);
>> +        ts->pools[index] = av_buffer_pool_init(pool_size, NULL);
>> +        if (!ts->pools[index])
>> +            return NULL;
>> +    }
>> +    return av_buffer_pool_get(ts->pools[index]);
>> +}
>> +
>>  /* return non zero if a packet could be constructed */
>>  static int mpegts_push_data(MpegTSFilter *filter,
>>                              const uint8_t *buf, int buf_size, int is_start,
>> @@ -1177,8 +1190,8 @@ static int mpegts_push_data(MpegTSFilter *filter,
>>                          pes->total_size = MAX_PES_PAYLOAD;
>>
>>                      /* allocate pes buffer */
>> -                    pes->buffer = av_buffer_alloc(pes->total_size +
>> -                                                  AV_INPUT_BUFFER_PADDING_SIZE);
>> +                    pes->buffer = buffer_pool_get(ts, pes->total_size);
>> +
>>                      if (!pes->buffer)
>>                          return AVERROR(ENOMEM);
>> 
>> @@ -1351,8 +1364,7 @@ skip:
>>                      if (ret < 0)
>>                          return ret;
>>                      pes->total_size = MAX_PES_PAYLOAD;
>> -                    pes->buffer = av_buffer_alloc(pes->total_size +
>> -                                                  AV_INPUT_BUFFER_PADDING_SIZE);
>> +                    pes->buffer = buffer_pool_get(ts, pes->total_size);
>>                      if (!pes->buffer)
>>                          return AVERROR(ENOMEM);
>>                      ts->stop_parse = 1;
>> @@ -3200,6 +3212,9 @@ static void mpegts_free(MpegTSContext *ts)
>>
>>      clear_programs(ts);
>> 
>> +    for (i = 0; i < FF_ARRAY_ELEMS(ts->pools); i++)
>> +        av_buffer_pool_uninit(&ts->pools[i]);
>> +
>>      for (i = 0; i < NB_PID_MAX; i++)
>>          if (ts->pids[i])
>>              mpegts_close_filter(ts, ts->pids[i]);
>
> LGTM. I assume the pools for the smaller sizes will not end up being
> slower than just allocating new buffers each time.

I tried to measure this but there was no significant difference.

Will apply.

Thanks,
Marton