[FFmpeg-soc] RTP packetizers

[Martin Storsjö]

On Wed, 11 Aug 2010, Luca Abeni wrote:

> On 10/08/10 10:39, Martin Storsjö wrote:
> 
> [...]
> > > But I see that the "VP8 in RTP" draft is still under development... It
> > > would
> > > be nice if someone with some knowledge of the VP8 bitstream and a good RTP
> > > understanding tries to make it smarter than the theora draft (by
> > > suggesting to
> > > fragment frames in appropriate points, so that the effect of lost packets
> > > can
> > > be minimised).
> > 
> > Well, the issue there seems to be that you can't decode anything from the
> > rest of the GOP if you miss the first data partition of any frame
> Well, I do not know theora and VP8, but for mpeg{1,2,4} and for H.264 I think
> this is not true... If a frame is composed by multiple slices (or NALs, or
> similar things), then missing one slice (or NAL, or similar) will not be
> critical for the whole frame (and for the whole GOP) - assuming that some
> error concealment techniques are used in the decoder. You will (of course) see
> artefacts in the video, but it will be decoded. So, having a "smart
> fragmentation" in RTP (that is, fragmenting frames according to slices, NALs,
> etc...) will improve error resilience.


To quote David Conrad from earlier in the discussion on this:

> VP8's probability contexts and segment map aren't reset between 
> interframes, so if an entire frame is lost the rest of the GOP will 
> likely decode to total junk.
> 
> If you want to do better than the proposal, you can read the vp8 frame 
> header and ignore the rfc if the first data partition is intact; that's 
> the only guarantee that arithmetic probabilities and the segment map 
> remain correct.

And my tests with it showed the same - if anything else than the first 
packet of a frame is broken, the frame itself might not decode totally 
correct, but following frames are ok at least. If the first packet of a 
frame is broken/missing, the whole rest GOP just becomes garbage.


> Now, if a VP8 frame can be composed by something similar to NALs, this feature
> should be exploited in RTP fragmentation (so, the RTP muxer should split the
> frame dividing it in these small units, as the H.264 and the MPEG packetisers
> do, etc...). I do not know if VP8 is done like this or not, but fortunately we
> have experts, here :)
> And since the draft is still being modified, I think it is worth rising this
> issue.

Don't know if there's something such for VP8. David, Ronald?

// Martin