[NUT-devel] NUT?

[Michael Niedermayer]

Hi

On Sun, Nov 05, 2006 at 06:22:34PM -0800, Ralph Giles wrote:
> On Mon, Nov 06, 2006 at 01:56:53AM +0100, Michael Niedermayer wrote:
> 
> > btw, ralph if you disagree here, iam curious about the cases where you think
> > that a crc field on frames makes sense
> 
> The main thing, I think, is that most decoder implementations are not 
> robust against garbage data (and in some cases cannot be, entirely) so 
> a crc on the frame data helps catch corruption that could crash or 
> mangle playback. It's not a security issue per se, since malicious code 
> could of course set correct crcs, but it still improves robustness.

well, there are 3 things here
first is harmless crashes due to out of array reads, null pointer 
dereference and such
second are security issues like writes over the end of an array or
various integer overflows causing writes to unallocated memory
third are audible distortions caused by errors


there are also 2 different types of codecs

ones which code a frame as a single monolithic piece so that a single error
causes the whole frame to become useless and ones which code a frame so
that an error only affects part of the data,
mp3, vorbis and i think many other audio codecs fall in the first category

OTOH, mpeg1/2/4 h263/4 fall in the second, they do support spliting a frame
in several independant slices (in case of mpeg2 1 slice per 16pixel row is
mandatory at minimum), also mpeg4 and h264 further allow a slice to be
split in 3 partitions which in case of mpeg4 can be encoded with reversible
vlcs, so that decoding can be done from start to end and from end to start
recovering alot of data surounding a single error (backward decoding though
isnt supported by anyone AFAIK, but using undamaged slices and undamaged
partitions of damaged slices is supported by libavcodec for example)

some audio codecs designed for "noisy" environments, i think AMR is one of
them also use various tricks to recover data from damaged frames

all that makes a crcs less usefull, a demuxer cannot just drop damaged frames,
what it could do is pass a is_damaged flag to the decoder but that then
would require decoders to have support for such a thing, and decoders dont
currently (we could ask everyone to change their decoders and corresponding
APIs but i doubt anyone would, and sure it can still be used somehow with
wrapers around decoders or a list of decoders and their error recovery
capabilities, it just IMHO becomes somewhat messy for little gain ...)


about crashes, a crc of course pretty much prevents them from occuring on 
random or damaged data, but it also prevents users from sending bugreports 
for such crashes (there are no crashes ...) which has 2 effects
1. the decoder will be full of bugs which make it crash on random data
   so it pretty much depends on either error free data or a container
   with checksums
2. some of the crashes can be security issues, which normally would have
   been found due to bugreports about the crashes ...


another problem is rawvideo is large
and highbitrate video too, and calculating crcs takes time, which leads
us to yet another question, should these crcs be mandatory or optional
if optional, then all the advantages pretty much disapear, but mandatory
seems unflexible and also adds alot of overhead for small packets
and having a checksum over several packets also means extra delay vs.
ignoring the checksum, so in the end checksums on frames cause more
problems then they solve IMHO
optional ones dont help the decoders preventing random input, dont
help the users checking files
mandatory ones mean more overhead, slower muxing for rawvideo or high
bitrate video which could be a problem for anything realtime
also crashes security or not should be fixed IMHO not be hidden by crc
checks

also nut is sufficiently extendible so that we can always add crcs
on frames later without breaking compatibility with old decoders

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

In the past you could go to a library and read, borrow or copy any book
Today you'd get arrested for mere telling someone where the library is