[Libav-user] Direct rendering or avoiding copying locally a decoded AVFrame before rendering

[Julian Herrera (TVGenius)]

On 7 Oct 2013, at 14:28, Paul B Mahol <onemda at gmail.com> wrote:

> On 10/7/13, Julian Herrera (TVGenius) <julian.herrera at tvgenius.net> wrote:
>> Hello all,
>> 
>> I have developed a video player based on ffmpeg for iOS, which is intended
>> to play MPEG-2 (SD) and H264 (HD) streams from a DVB-S source. The rendering
>> of the decoded video frames is performed by an OpenGL component also
>> responsible for converting YUV frames to RGB.
>> 
>> I used the code in ffplay.c as a reference for the steps necessary to decode
>> and display the video frames, but I was wondering whether I can improve the
>> process by eliminating the need of copying locally each decoded frame before
>> enqueuing them for rendering.
>> 
>> Basically I would like to implement the fix proposed in line 1612 of
>> ffplay.c (ffmpeg v. 2.0) to bypass the local copy of the decoded frame. So
>> the idea is to allocate a brand new AVFrame every time before calling
>> avcodec_decode_video2() and then send the frame to the renderer, instead of
>> passing a single reusable AVFrame to the decoder and copying locally the
>> result frame (this single copying step consumes around 10% of the processor
>> time on an iPad 4 whilst decoding h264-based 1080i streams).
>> 
>> I tried such idea unsuccessfully. The resulted frames were rendered out of
>> order, so the video was played as if the images were going back and forth in
>> time. If I call av_frame_clone(videoFrame) before sending the frame to the
> 
> I guess your code does not use reference counting and thus that call
> copies all data.

I do not use reference counting indeed. I could not find an example on the Internet on how to use it. What I basically need is to allocate a the AVFrame buffer locally, pass it to the decoder and release it only after rendering. It seems though the decoder writes either incomplete data for some frames or rewrites the buffer within subsequent calls. I haven't been able to understand yet how the decoder handles the decoded data in AVFrame.data[] so the lifecycle of this buffer is not clear to me.

> 
>> renderer, the video is played correctly but the same high processor usage
>> occurs, which is what I am trying to prevent.
>> 
>> Could you shed some light on how to solve this issue if it is possible?
>> 
>> Regards,
>> Julian Herrera
>> _______________________________________________
>> Libav-user mailing list
>> Libav-user at ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/libav-user
>> 
> _______________________________________________
> Libav-user mailing list
> Libav-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user