[Libav-user] avcodec_decode_video2(pCodecCtx, pFrame, &frameFinished, &packet) why most of the time frameFinished is zero ?

[wm4]

On Tue, 27 May 2014 11:50:58 -0400
Joshua Kordani <jkordani at lsa2.com> wrote:

> On 5/27/14 11:28 AM, wm4 wrote:
> > On Tue, 27 May 2014 17:10:55 +0530
> > sithruk sana <get2jils at gmail.com> wrote:
> >
> >> thanks for the reply.
> >>
> >> I am doing the same by queueing the packet  as described in dranger
> >> tutorial http://dranger.com/ffmpeg/ffmpegtutorial_all.html
> > This tutorial is prehistoric, and following it will cause you more pain
> > than not. Many things in it are just wrong or uses deprecated APIs. I
> > recommend pretending that this tutorial doesn't exist.
> >
> > Normally, libavcodec decodes one frame for one packet. But you need
> > to deal with "codec delay": at first you won't get any decoded frames,
> > even if you put in some packets. Only later (when enough frames are
> > queued internally), you will get 1 decoded frame for 1 packet. (You also
> > need to feed it null-packets at the end of the video to get the last
> > frames.)
> If I have a buffer of encoded units, which I know contains a complete 
> frame of encoded input, are you saying that I may have to pass multiple 
> sets of these before I receive the decoded results of the first buffer?

Probably. Especially if you use threads, the delay (in number of
frames) will rise. Also have a look at AVCodecContex.has_b_frames and
max_b_frames. This _should_ be the inherent delay used by the decoder.

In this context, it might also be important to explain why this
happens: the use of B-frames means that decoding of frames can depend on
the decoding of _future_ frames, so a future frame is decoded first,
and only then an "earlier" frame is decoded. This means that even if
the decoder decodes this future frame, it can't return it to the user
yet - because the user expects frames in display order. So the frame is
buffered internally, and no output is returned yet.

> Or are you saying that I can submit chunks of a frame's worth of encoded 
> input, and (understandably) won't get a complete frame back until I pass 
> in enough data for 1 frame?
> 
> Does the term "packet" in your reply correspond to something specific, 
> as in, one "packet" of h264 encoded data corresponds to one NALU, or do 
> you mean, one "packet" refers to enough encoded data, no matter the 
> encoder, to make one frame?

I'm not really sure. "packet" corresponds to libavcodec's understand of
a packet. I'm pretty sure you must not pass more than 1 frame worth of
data to the decoder, though. The API can't handle that. (Imagine you'd
feed 2 frames on each avcodec_decode_video2 call - the decoder can
return only 1 frame, so frames would accumulate internally. Obviously
this can't happen, and I assume the call will either return how much
data was decoded, similar to audio decoding - or just drop the data.)

> > There are others reasons why it wouldn't output a frame. If the video
> > frame is "damaged", you may need to set CODEC_FLAG2_SHOW_ALL or
> > CODEC_FLAG_OUTPUT_CORRUPT (forgot which one) to make lavc output them.
> > Also, it's possible to drop frames by setting skip_frame, but you
> > probably don't use this.
> >
> this is useful, thank you
> 
> Joshua Kordani