[FFmpeg-devel] how to use own buffers for input data with av_read_frame()

[Reimar Döffinger]

On 31 Aug 2012, at 03:10, Michael Niedermayer <michaelni at gmx.at> wrote:
> On Thu, Aug 30, 2012 at 08:34:06AM +0200, Jan Pohanka wrote:
>> Hello,
>> 
>> I'm implementing support for hardware video decoder on DM365 and I
>> have some problems with buffers. I have simple testing application
>> with following code (simplified)
>> 
>> ...
>> avformat_open_input(&fctx, filename, NULL, NULL);
>> ...
>> av_find_stream_info(fctx);
>> ...
>> codec = avcodec_find_decoder_by_name("libdm365_h264");
>> ...
>> ret = avcodec_open(avctx, codec)
>> ...
>> picture = avcodec_alloc_frame();
>> ...
>> for (i = 0; i < 500; i++) {
>>    int nb;
>>    char fname[32];
>> 
>>    if (av_read_frame(fctx, &pkt) < 0)
>>        break;
>> 
>>    nb = avcodec_decode_video2(avctx, picture, &got_pic, &pkt);
>>    if (nb < 0) {
>>        av_log(avctx, AV_LOG_ERROR, "error in decoding\n");
>>        goto decode_cleanup;
>>    }
>>    printf("Decoded frame: %d\n", i);
>>    sprintf(fname, "frame%02d.jpg", i);
>>    save_image(picture, avctx->pix_fmt, avctx->width, avctx->height,
>> fname);
>> }
>> 
>> The problem is that the decoder needs to have both input and output
>> data in a buffer which is continuous in physical memory. I can
>> fulfill this constraint for the output data (AVFrame *picture)
>> because the buffer is allocated by codec itself so I can use my own
>> allocator there.
>> Unfortunately I do not know how to handle data of AVPacktet pkt,
>> which are filled in by av_read_frame() - these are allocated by
>> av_malloc which fallbacks to malloc and the buffers probably won't
>> be continuous.
>> 
>> I see two solutions here, but I'm not happy with neither of them:
>> - copy the whole packet to the continuous buffer (but this will
>> significantly slower the application)
>> - provide own av_malloc wrapper that will use my own allocator (many
>> things from libav* internals will be also allocated there and this
>> is not exactly what I want)
>> 
>> 
>> Do please someone know some more elegant solution?

I guess "get a better hardware decoder" does not count as a helpful answer, even though I do consider the problem here that the decoders are technologically stuck about 10 years in the past when it was ok to design hardware without bothering how messy it is for software to use.

> I dont think that a single memcpy of the compressed data will have a
> speed effect that could easily be meassured. Instead it will be
> overshadowed by both obtaining this data from network/disk and the
> actual decoding. If you still wish to avoid this memcpy you would have
> to add some callback mechanism for the AVPacket allocation.
> Such a optimization would of course be welcome in main ffmpeg if you
> implement it and it sclean & simple & faster

I think that such a solution is a very bad idea in this case.
The fact that packet sizes vary a lot together with the fact that FFmpeg can buffer quite a lot of them sometimes means you run into serious risk of issues with memory fragmentation when you allocate and deallocate them constantly as continuous physical memory regions.
Depending on the details, some hack like using huge pages and only copying when the buffer by bad luck crosses a page boundary _might_ be a way to improve average performance...