[MPlayer-dev-eng] Nut and B frames (and much more) NODAEMON

[Michael Niedermayer]

Hi

On Monday 26 April 2004 19:40, D Richard Felker III wrote:
[...]
> > 4) group of frames (that is GGG, where every G encodes, for example, 25
> > frames).
> >
> > Correct me if I'm wrong, but avi was designed with 1) and 2) in mind and
> > some trickery was necessary to have streams of type 3).
> > Nut will support well 1), 2) and 3) (the existance of this thread is a
> > proof of that), but was 4) ever considered until now?
> >
> > Just to explain it better, the G type encoder takes 25 frames and
> > outputs some bits which describe all the frames. If you miss one of the
> > bits none of the frames is recoverable (in theory).
>
> Yes it's considered. But really you need some way of knowing the
> presentation times for all the frames. My thought is that such a codec
> should have one large packet for the first of the 25 pictures
> containing all the data, then small "presentation packets" for each of
> the remaining 24 that just serves as a placeholder to tell when the
> picture should be shown. When these presentation packets get sent to
> the decoder, it outputs the next frame in the sequence.
>
> (Note that I'm using the terms picture and packet instead of frame
> because frame in nut terminology means a single coded unit while in
> video it means a single picture.)
>
> The other approach would be to just have the one packet for all 25,
> with the first PTS, and then have the codec layer generate the rest
> with appropriate timestamps. But personally I'm against codecs ever
> touching timestamps since most handle time very stupidly.
even another approach is to store the individual wavelet transformed frames 
with some reordering, and iam almost certain this is what the standard 
committees will do if they decide to standardize such a thing
why?
simple, its much more flexible, just discard the temporal high-frequency 
subband(s) and u got a video with with 1/2,1/4, ... fps or discard the 
spatial high freq subband(s) and u got 1/2, ... the resolution

>
> Anyway I understand your thought that in some ways, video in the time
> domain is similar to audio. But it's a lot different too. Audio is
> oversampled (44100 or even 48000 or 96000 Hz when you only need about
> 38000), but video is grossly undersampled (24-30 Hz when you need
> about 500 Hz). So doing frequency transforms on video in the time
> domain doesn't really make a lot of sense until we have equipment that
> can work at 500 fps (or at least 150 fps or so...).
thats surely true, unless the temporal transform is done along the motion 
trajectories

>
> > It would be good to debate on this issue. For example, the pts
> > is supposed to represent the istant in which the frame starts
> > or the istant at the middle of the frame duration (in my example,
> > the G would shift 0.5 seconds if things aren't well defined).
>
> This was already answered. In nut, pts MUST ALWAYS represent the start
> time of the frame. Otherwise it's useless for most purposes.
>
> > Another thing, what if the codec does this:
> > 1) takes frames 0-24 and generates a G bitstream
> > 2) takes frames 20-44 and generates another G bitstream
> > and the decoder is supposed to blend frames 20,21,22,23,24 together
> > to avoid "block artifacts" in the time domain?
> > In other words, the group of frames are partially overlapped.
> > (this already happens in audio coding, right?)
>
> This should work fine with the sort of things I described above.
yes for video, but what about audio, with a delay? should the pts represent 
the first sample the decoder will output for the given bitstream packet? i 
guess so, it seems to be the most obvious choice

[...]
-- 
Michael
level[i]= get_vlc(); i+=get_vlc();		(violates patent EP0266049)
median(mv[y-1][x], mv[y][x-1], mv[y+1][x+1]);	(violates patent #5,905,535)
buf[i]= qp - buf[i-1];				(violates patent #?)
for more examples, see http://mplayerhq.hu/~michael/patent.html
stop it, see http://petition.eurolinux.org & http://petition.ffii.org/eubsa/en