[MPlayer-dev-eng] [PATCH] mencoder, B-frames, and video/audio delay

[Corey Hickey]

Michael Niedermayer wrote:
>>>>>>--- libmpcodecs/ve_lavc.c	10 Jan 2006 11:09:02 -0000	1.125
>>>>>>+++ libmpcodecs/ve_lavc.c	21 Jan 2006 05:44:32 -0000
>>>>>>@@ -750,6 +750,9 @@
>>>>>>        mux_v->bih->biSize= sizeof(BITMAPINFOHEADER) +
>>>>>>lavc_venc_context->extradata_size;
>>>>>>    }
>>>>>>
>>>>>>+    if (lavc_venc_context->max_b_frames)
>>>>>>+        mux_v->encoder_delay = lavc_venc_context->max_b_frames + 1;
>>>>>>+
>>>>>
>>>>>Why the +1? This seems incorrect,
>>>>
>>>>I don't know. That's what Michael suggested and I've been assuming it to
>>>>be correct. I know I can't really notice 1/24 second sync difference.
>>>>
>>>>http://www1.mplayerhq.hu/pipermail/mplayer-dev-eng/2005-December/039425.html
>>>
>>>I think the +1 is for decoding delay. So it is true that total delay =
>>>bframes+1, but it is incorrect to put the +1 in the muxer.
>>
>>Huh. That makes sense. If I'm reading things right, then that's the way x264
>>does it:
>>
>>x264/encoder/encoder.c:562
>>h->frames.i_delay = h->param.i_bframe;
>>
>>...however, libavcodec adds the +1 whenever B-frames are present.
>>
>>libavcodec/mpegvideo.c: 1195
>>s->low_delay= s->max_b_frames ? 0 : 1
>>avctx->delay= s->low_delay ? 0 : (s->max_b_frames + 1);
>>
>>I really don't know if either is incorrect. That would have to be up to
>>Michael or yourself.
> 
> 
> well "avctx->delay= s->low_delay ? 0 : (s->max_b_frames + 1);" is the encoder
> input to decoder output delay
> 
> the delay between encoder input and encoder output (=pts difference) is
> variable and between 0 and s->max_b_frames + 1 if b frames are used or 0 if not
> 
> 0 1 2 3 4 5
> I B P B P
> |  \|  \|
> |   |\  |\ 
> I   P B P B
> 
> if the encoder can output several frames at once with no input then its delay
> can be between 0 and s->max_b_frames
> 
> 
> the difference between encoder input pts and encoder output dts / decoder
> input dts should be "avctx->delay= s->low_delay ? 0 : (s->max_b_frames + 1);"
> too

I understand your explanation (and I really appreciate the
illustrations, by the way). My remaining question is whether it's
correct to compensate the muxer stream timer in anticipation of that +1
decoding delay with libavcodec. ...because Loren said that it isn't.

But first, let me digress to x264. My tentative understanding of the way
x264 handles decoding delay is to put it into a header*. Loren pointed
me to x264/encoder/set.c:187

sps->vui.i_num_reorder_frames = param->b_bframe_pyramid ? 2 :
param->i_bframe ? 1 : 0;

So, it looks like I don't have to worry about the decoder delay for
x264. Like I said, though, that's only a tentative understanding on my
part. This is the first time I've ever looked at x264 source.

Anyway, does MPEG-4 have any kind of similar facility for putting
anticipated decoding delay into a header? If not, then adjusting the
muxer stream timer seems like the only reasonable recourse.


* I can see that it's an SPS, but despite lots of googling I can't find
what an SPS is. Can someone give me a quick definition?

Thanks,
-Corey