[MPlayer-dev-eng] FINALIZE NUT SPEC!!

[Rich Felker]

On Mon, Feb 27, 2006 at 02:04:26AM +0100, Michael Niedermayer wrote:
> Hi
> 
> On Sun, Feb 26, 2006 at 02:17:22PM -0500, Rich Felker wrote:
> > On Sun, Feb 26, 2006 at 06:26:16PM +0100, Alexander Strasser wrote:
> > > Oded Shimon wrote:
> > > > The info frames patch I just sent to nut-devel is the last withstanding 
> > > > problem I know of for NUT. libnut seeking has been rather heavily tested, 
> > > > even with EOR. We could still add (remove?) some fields for Info 
> > > > packets/frames, but that can be done after finalizing.
> > > > 
> > > > libnut/nututils is far from ready for production, but I think mpcf.txt 
> > > > _IS_.
> > > > 
> > > > Is there anything else? Is anyone displeased? (even Ivan is happy and 
> > > > stopped flaming! :)
> > > 
> > >   WTF! Ivan, where are you when you are needed :(
> > >   
> > >   Anyway nice that you decide it is locked, i am looking forward to
> > > my own NUT fork.
> > 
> > WTF, I hope you're kidding...
> 
> i thought the same when i read that "NUT is locked speak now or be silent 
> forever" right after some large changes and before any of the following 
> have been done

Well I agree that Oded's email was extremely premature.

> * error recovery tested (playback, seeking with and without index, dealing 
>   with damaged syncpoints, main header, stream header, info packets/frames, ...)
> * systematic errors (zero or 0xFF runs) tested

Oded has been doing some of this. The extent of testing I do not know.

> * various decode delay cases (h.264 b pyramid) tested

..? What is there to test here?

> * extendability of all headers tested
> * tested nut over http, ftp, cdrom

> * wait 1-3 month at least where noone finds an issue and no changes are done
>   if some change has to be done reset the timer

This is nice in principle but no one will use a non-final spec and
thus no testing will happen. :(

> * testing support for various codecs 
> * publish all test-results
> 

> some specific issue iam aware of
> 1. metadata which becomes known after the time to which it applies cannot
>    be stored in info streams, i already threatened to write a info2 
>    extension if oded-rich info streams/packets end up with some silly
>    limitations ...

I don't object to the existence of such data, but it's not
stream-like. Violating the stream nature of a stream to store
information that does not go with the pts of the frame is not a proper
solution.

> 2. PCM/ADPCM and other very small and constant sized packet codecs
>    have huge overhead (50% or so) if stored in a spec compliant way, we 
>    need to add something like Fixed-size lacing (AVI, MOV, MKV support that)

If you interpret the definition of frame for them as single samples,
then yes. :) This has been discussed recently on IRC. A few ideas:

1. PCM audio must be broken into frames smaller than 2 timebase units
   for any timebase in the file. Naturally this is bad if some idiot
   uses ridiculously small timebase; it may not even be possible to
   satisfy.

2. PCM audio must be framed such that no frame entirely overlaps more
   than 1 whole frame from any other stream.

Idea 2 is not too bad and 1=>2. However, it's hard to test/verify and
still may not be possible if the samplerate is low.

Also neither of these prevents some idiot from storing the whole file
in a single audio frame for audio-only files.

I also commented on IRC the other day that some codecs may be poorly
specified and require supplemental specs from us for storage in NUT.
Some examples would be codecs that don't define the notion of
decode_delay or something like Vorbis with broken extradata format.
IMO the best solution might be to just treat PCM as one of these, and
specify a that a frame of PCM is anywhere between 1 and 1024 samples
(samples meaning sampling times, i.e. 10 left-channel samples and 10
right-channel samples count as 10, not 20).

IMO PCM is already such a format that needs additional specification,
since we need to specify the extradata format (or else many fourcc's)
for all the different sample formats.

> 3. index is fragile, 1 byte damaged and its completely lost, writing the index

I'm still not convinced that having redundant data being fragile is a
problem, but if you can fix it in a way that's graceful I'll be happy
to accept a fix.

>    in the way its currently will require 2 passes, libnut always uses 2 passes
>    IIRC, but lavf didnt, the issue is that the forward ptr must be updated
>    if packets are small like they all should have been then thats no problem
>    with a small buffer but with the huge monolithic index its no longer 
>    possible

Sorry, I forget how index works. Maybe I never understood to begin
with..

> 4. with the "info ends at the next other packet" rule we also have a

IMO this is not a problem: If the length is known in advance, then
any competent file author will store it in the frames, or better yet
not use an info frame at all and store normal info packets at the
start of the file.

>    issue with timebases and it violates the 
>    "damaged files can be played back with minimal data loss" goal

As far as I can tell this is really minimal.
Like I said before, if you want extra safety you can add extra EOR
frames regularly to make sure it's ended. But somehow I doubt
broadcasters will flag commercials for you anyway! ;) And if you're
talking about your own file on disk for editing, you should be using
info packets, not info streams.

BTW the same problem applies to subtitles! If the clear sub/next
subtitle (for nonoverlap formats)/EOR frame is lost, then the subtitle
will remain displayed for a long time unless you do something to
prevent it. Likewise, if the subtitle start frame is lost, the
subtitle will remain undisplayed for the whole time it was supposed to
be there.

We could go further and say the same applies to video too, since if a
keyframe is lost you're screwed for the next 10-15 seconds.

IMO in comparison these things, losing the ending time of something
that is _purely metadata_ is hardly a problem.

> summary of MHO
> there are issues left which will some time to fix and

Agree.

> then 1-3 month should pass before locking, that assumes no new issues are
> found in that time which is unlikely

I really doubt this will help... :(

Rich