[NUT-devel] Info packets in NUT stream (spec bugs?)

[Michael Niedermayer]

Hi

On Mon, Nov 20, 2006 at 07:41:48PM +0200, Oded Shimon wrote:
> On Mon, Nov 20, 2006 at 12:37:22PM -0500, Rich Felker wrote:
[...]
> > > that too, iam not sure if theres anything in the spec which forbids it or
> > > if theres anything which would break, iam just mentioning it ...
> > 
> > Starting at arbitrary timestamp is totally legal and it was always my
> > intent for it to be legal. Obviously streaming is not possible either
> > if you don't allow for this..

IIRC someone originally wanted timestamps to start from 0, we then realized
thats not possible and then we agreed that it should be close to 0 or
something (and no iam not at all trying to say that i am against arbitrary
start timestamps ...)


> 
> I've never actually tested it, but AFAIK libnut is completely safe and 
> non-breaking on this issue.

theres at least one issue with random start timestamps
try 1e9999 as start timestamp and tell me if that worked :)
while the fileformat of course has no problem with arbitrary integers,
implementations will ...
making it clear that 0 should be used as start where possible reduces
the issue but doesnt solve it

something like:
timestamps SHOULD start at or close to 0, and MUST fit into 63bit, note
this limit may be raised in the (distant) future
should solve the issue ...

this should be enough for a few hundread years at nanosecond resolution


[...]
> > > > I have no idea for a solution. Your proposed solution doesn't work as 
> > > > there is no way to find the last info packet. even with your stuffing 
> > > > info packet in headers idea: 1. how would you find the headers? 2. who 
> > > > said the headers are necessarily after the last info packet in the file? 
> > > > Our main assumption is that in realtime stream dumping, they can be 
> > > > cut/truncated at absoloutely any arbitrary point in the file.
> > > 
> > > 1. every midstream info packet (and only them) MUST have a pointer to the
> > >    previous non redundant/repeated midstream info packet
> > > 2. the distance between midstream info packets MUST be <= C*max_distance
> > >    unless that is impossible (too large stream header / frame) in which
> > >    case the distance MUST be as small as possible for the specific case
> > > 3. a info packet MUST either be part of the normal info packets or it
> > >    MUST be repeated like described in 2. within the area to which it applies
> > >    and it MUST be in that area at least once
> > 
> > Seems like the sanest proposal for this so far, but.... introduces
> > huge ugly complexity to implementation.
> 
> It's actually not that bad in implementation - just keep last_info and 
> last_info_redundant (to know if to place another one now), a few more if's 
> together with syncpoint writing, and you're done.
> 
> Or do you mean complexity in demuxer? In which case, yes, it is somewhat 
> ugly... But I don't really agree in demuxer searching for info packets 
> anywhere after main headers anyway.

thats perfectly fine, i never proposed that demuxers should search for them
its just that if a demuxer wants to or needs to then it should be technically
possible to find them
also remuxing such streamdumps into midstream info less nut files benefits
from the ability to qickly enumerate all midstream info packets ...


> 
> I have two questions though:
> How do you know if to search for these redundant packets or 
> not? I assume you do NOT write them at all if there are no mid-stream info 
> packets at all, in which case, when looking for them, how do you know if 
> you simply haven't scanned enough of C*max_distance from EOF, or if there 
> aren't mid-stream info packets at all?
> Also, for the begginning of file, do you write these info_redundant until 
> you have one info packet or not?

dunno, probably yes


> Sounds like this would need another flag in the main header.

yes


> 
[...]
> [...]
> > I'd like to request a minor
> > revision: instead of requiring positive chater id's to start from 1,
> > just require them to be contiguous. This keeps implementation simple
> > (array storage can be used), but the first element need not reflect
> > "chapter 1" in a user sense.
> 
> I preffer this.

same issue as with timestamps, may i limit that to 31 bit or more
precissely int32_t?

and is it really that simple? i mean with the start from 0 its a
count=MAX(count, id); realloc(count); store
with the contiguous ones lets say you receive
100,101,102,103,104,105,99,100,101,...

here the first info packet with 99 was damaged ...

then what about mixing that with normal info packets? i mean info packets
with chapter id 1,2,3
and then midstream info packets 4,5,6,7, ...
and if you now cut the file, you loose contiguousity ...

i think these things need to be resolved before we can think about allowing
chapter ids to start at >0

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

In the past you could go to a library and read, borrow or copy any book
Today you'd get arrested for mere telling someone where the library is