[Ffmpeg-devel] Problems with output picture reordering in H.264 decoder

[Loren Merritt]

On Wed, 9 Aug 2006, Santa Cruz, Diego (GE Indust, Security) wrote:

> I'm having problems with output picture reordering in H.264 decoder
> (great piece of work BTW). I have a purely IPPPP stream, where the POC
> increases by 4 with each frame.
>
> The first IDR is immediately decoded. However, upon decoding the first P
> the decoder thinks that there is picture reordering required since the
> POC increases by more than 2. The code in question is in h264.c,
> decode_frame():
>
> 08233  else if((out_of_order && pics-1 == s->avctx->has_b_frames && pics
> < 15)
> 08234     || (s->low_delay &&
> 08235      ((!cross_idr && prev && out->poc > prev->poc + 2)
> 08236       || cur->pict_type == B_TYPE)))
>
> As far as I understand the H.264 spec the POC can increase by whatever
> amount on two successive output pictures and this test seems too
> restrictive. What is the reason for such a test? I would like to remove
> it, but I am afraid I do not understand all the implications of it on
> the H.264 decoder.

The reason for the test is that H.264 streams are not required to say how 
much decoding delay they need. x264 does, but many other encoders don't.
I see the following possible solutions:
* Follow the spec and assume delay=16 if it's not specified. And then 
implement reordering of timstamps, because this utterly breaks A-V sync.
* Assume delay=1 if it's not specified. And break decoding if it needed 
more. (This may be the most correct most of the time, but IIRC fails on 
some stuff encoded by VSSH.)
* Detect out-of-order frames. There will be a visible jerk the first time 
it sees one, and this will break regression testing on many of the 
official conformance clips.
* Detect the frame before the out-of-order frame. No artifacts in the 
normal case, but breaks if the stream does funny things with POC. (the 
current implementation)

  --Loren Merritt