[NUT-devel] Info packets in NUT stream (spec bugs?)

[Rich Felker]

On Sun, Nov 19, 2006 at 06:58:59PM +0200, Oded Shimon wrote:
> On Fri, Nov 17, 2006 at 01:02:22PM +0100, Michael Niedermayer wrote:
> > On Fri, Nov 17, 2006 at 09:31:40AM +0200, Oded Shimon wrote:
> > > This doesn't seem like a streaming protocol, more like a butchering of the 
> > > nut spec (most obvious, allowing syncpoints not be written - this is an 
> > > obvious violation of max_distance). This doesn't necessarily make it a bad 
> > > idea (if you are completely error free, and not seeking, syncpoints are 
> > > redundant and non-trivial to calculate for their back_ptr), but it does 
> > > not seem like something to be written in a seperate document and being 
> > > called a "streaming protocol" - it's a modified NUT format spec for 
> > > streaming... And I think it's probably a bad idea to have several modified 
> > > specs (it's somewhat like mpeg-ts, mpeg-ps, mpeg-bla).
> > 
> > no mpeg-ts and mpeg-ps are much more different then what i proposed
> > 
> > the problem simply is that different environments/systems have different
> > requirements, if there is packet loss then some additional things are needed
> > if not they are a waste, surely some minimum error recovery stuff can always
> > be added but then again you where the one saying you will ignore the spec
> > and not repeat headers as its not needed in your case! so what do you
> > argue against? seems like its what you yourself wanted?
> 
> I don't consider it a butchering of the spec to omit repeated headers in 
> the case of realtime streaming as it is an impossible rule to inforce and 
> illogical in the case of realtime streaming. The only way to truely 
> enforce this rule in realtime streams is by making all 3 header copies 
> right at the begginning, with no frame inbetween - otherwise the user can 
> stop the dumping immediately after it begun and it would be impossible to 
> get all 3 copies even if the streamer repeated headers.
> Even if they were repeated - they would not be useful for finding in 
> dumped files as they would not be in 2^n locations (unless you expect the 
> streaming server to start doing calculations for EACH client..)
> 
> It's a butchering of the spec only because you can semantically consider 
> the file truncated and incomplete - in your suggest of max_distance 
> butchering, you're explicitly butchering the spec to the point that 
> compliant demuxers will not be able to play it without different modes for 
> streaming and files.

You realize, right, that the issue here is that a truncated nut file
is not a valid nut file, according to the spec? If we acknowledge the
existence of truncated files but note that they do not constitute
"complete" files (by virtue of missing header copies and missing
index) the problem goes away, I think.

> > if you buther the spec by omiting repeated header then butcher it enough
> > so it cannot be played if dumped raw otherwise such broken streams will
> > spread not neccessarily originating from stream dumps but maybe rather
> > people who like to safe 0.1% space
> 
> Well, I somewhat doubt that will actually happen, but I accept the
> argument...

It's quite easy to say that incomplete streams are valid but
incomplete, and further to require that IF the stream contains and
index, it MUST be complete (i.e. include header copies).

> BTW, if we ultimately decide on disallowing mid-stream info packets, we 
> need to remove these 2 parts from the spec:
> ---
> If a demuxer has seen several info packets with the same chapter_id and
> stream_id then it MUST ignore all but the one with the highest position in
> the file
> 
> demxuxers SHOULD NOT search the whole file for info packets
> ---

I'm find with keeping this possibility as long as the last condition
(don't search) is made abundantly clear. But really I don't care. It's
a stupid bikeshed issue.

> BTW2, If we decide we do allow mis-stream info packets, then this rule 
> somewhat fails:
> ---
>     chapter_id n MUST not be used unless there are at least n chapters in the
>     file
> ---
> 
> As a streamer would start at chapter_id=1, and linearly grow - you can 
> join at any arbitrary point, even when chapter_id=5926.

IMO this should remain illegal. You can use negative chapter_id or
something.. In any case the info system is probably not ideal for the
sort of streaming uses you're describing. There are much better out of
band ways to accomplish this info delivery..

Rich