[Ffmpeg-devel] Problems with output picture reordering inH.264 decoder

[Santa Cruz, Diego GE Indust, Security]

> -----Original Message-----
> From: ffmpeg-devel-bounces at mplayerhq.hu 
> [mailto:ffmpeg-devel-bounces at mplayerhq.hu] On Behalf Of Uoti Urpala
> Sent: Wednesday, August 09, 2006 21:51
> To: FFMpeg development discussions and patches
> Subject: Re: [Ffmpeg-devel] Problems with output picture 
> reordering inH.264 decoder
> 
> On Wed, 2006-08-09 at 21:31 +0200, Michael Niedermayer wrote:
> > why not analyze the first few frames in av_find_stream_info() / the 
> > AVParser to guess the delay? that seems to be the best solution 
> > especially if you consider h264 -> foobar encoding for 
> which correct 
> > timestamps will generally be needed
> 
> Looking at just a few first frames probably isn't too 
> reliable. I tested
> 3 videos and one of them had the delay increase from 0 to 1 
> after about 40 frames (the others were delay 1 and 2 after 
> the first frames).
> 

I agree with the above. H.264 is pretty special since it totally
decouples the picture reordering from the picture types (btw, there is
no such thing as a B or P picture in H.264, it just have slices and all
the slices of a picture may be P only or B only, or whatever; also P
slices may have forward prediction, requiring reordering, and B slices
may have only backward predictions requiring no reordering).

I guess the best solution would be to limit the amount of reordering by
using the num_reorder_frames if present and use the dpb_output_delay in
the SEI picture timing to know when to output each picture and the POC
to actually reorder the pictures. I think that is the way it is supposed
to work in H.264. Without the dpb_output_delay a decoder can only
guarantee output order conformance but not *timing*, so it would not be
possible to guarantee a "no jerk" operation if dpb_output_delay is not
used.

BTW, if I understood well the reordering algorithm currently in h264.c
an IDR, P, B, B stream ... (with POCs 0, 6, 2, 4) that is decoded frame
by frame will output the IDR right away and then output no picture
(since it detects the POC gap from 0 to 6) and then output the pictures
normally. This already creates a jerk (although small), but correct me
if I misunderstood.

Would relying on the dpb_output_delay be reasonable? Does x264 output
them?

-- 
Diego Santa Cruz, PhD
GE Security
Engineered Systems EMEA
Research Scientist

T +41 21 695 0019
F +41 21 695 0001
E diego.santacruz at ge.com
www.gesecurity.net

Victoria House
Route de la Pierre, 22
1024 Ecublens
Switzerland
VisioWave sarl