[MPlayer-dev-eng] Nut and B frames

[Michael Niedermayer]

Hi

On Monday 26 April 2004 13:38, D Richard Felker III wrote:
> Nut is supposed to have a "perfect interleaving" rule, i.e. if frame B
> comes after frame A, regardless of whether they're part of the same
> stream or different streams, frame B must have a later timestamp than
> frame A (or the same). This is to avoid situations like AVI where
> demuxers have to deal with badly interleaved (or noninterleaved) files
> and thus end up rapidly seeking in order to be able to play the movie.
> But how do B frames (or out-of-order frames in general) fit into this
> picture?
>
> I see two possibilities:
>
> 1. In files with B frames, out-of-order frames may be stored anywhere
>    between the previous frame and the next frame in decoding order.
>
> 2. No special treatment for files with B frames. Perfect interleaving
>    still applies.
>
> Before you flame and choice #2 sucks, let's hear the arguments for
> both:
>
> Choice 1 is appealing because it gives you frames in the order that
> most decoders expect. But it also has some drawbacks. Let's say you
> have the following sequence of frames in a file: I0 P3 B1 B2 (decode
> order). When you encounter the P frame, you have to be aware that the
> file has B frames and look ahead. Otherwise you'll see the (well in
> the future) timestamp on the P frame, and think there's nothing to do
> for a while! By the time you see the B frames, it'll be too late to
> present them. Also, after seeking you have to take special precautions
> to reset the codec so you don't decode B frames that belong before the
> I frame you seek to.
>
> Some of the problems of choice 1 could be remedied by an out-of-order
> flag in the frame header (but that's wasteful) or a global header that
> says out-of-order frames are present and that you need to look ahead N
> frames to find next pts...but that's all complicated.
>
> Choice 2 immediately looks _bad_, because it requires you to buffer
> one or more B frames before actually passing them to the decoder. I've
> been a very vocal opponent of doing stuff that requires buffering in
> NUT. But unlike in some other cases (audio subpackets, forward
> pointers, etc.), the buffering here doesn't increase latency beyond
> the latency already present in B frames. Further, any device that has
> to cope with B-frame codecs already has to have some degree of
> buffering for decoded frames, to display them in the right order, so
> requiring some buffering of encoded frames (much smaller) isn't much
> of a big deal.
this is wrong IMHO, just imagine 100 b frames in a row (ok not terribly 
likely, but legal and just to demonstrate the problem)
choice 1. demuxer gets I frame, P frame, B frame; decoder outputs 2 frames, 
and will output another for each b frame it gets
choice 2. demuxer gets I frame, B frame, B frame; decoder outputs nothing, it 
cant, it needs the P frame first which will come after 98 more B frames

what we do need is some definition of dts (decoding time stamps) which is the 
time at which a packet needs to be passed into the decoder, for them we could 
enforce monotonicity

and its easy too
1. add a pts_reorder_style flag into each stream_header

pts_reorder_style=0 
	rpts=pts

pts_reorder_style=1
	if(max_pts < pts){
		rpts= max_pts;
		max_pts=pts;
	}else
		rpts= pts;

example:
type&pts: I0 P3 B1 B2 P4 P6 B5
max_pts :  -  0  3  3  3  4  6
rpts    :  -  0  1  2  3  4  5

[...]
-- 
Michael
level[i]= get_vlc(); i+=get_vlc();		(violates patent EP0266049)
median(mv[y-1][x], mv[y][x-1], mv[y+1][x+1]);	(violates patent #5,905,535)
buf[i]= qp - buf[i-1];				(violates patent #?)
for more examples, see http://mplayerhq.hu/~michael/patent.html
stop it, see http://petition.eurolinux.org & http://petition.ffii.org/eubsa/en