[FFmpeg-devel] Scalable Video Encoding and FGS

[Erik van Zijst]

Michael Niedermayer wrote:
>> I think streaming video could also benefit from this
>> platform/protocol when you apply some form of scalable encoding:
>> have a small base-layer and multiple enhancement layers. I've
>> looked at MPEG2 FGS which comes closest to my needs, but this has
>> not been implemented in any serious codec AFAIK.
> 
> no, and i think almost everyone is very glad that it hasnt, same for
> the equivalent mpeg4 scalability stuff
> 
> have you considered using several streams where each is optimized for
> a specific bitrate and then just switch between them depending on
> network state or even encode a stream in realtime depending on the
> available bitrate?

You're referring to "simulcasting" which I believe is done by
RealNetworks and Windows Media Server.

Simulcasting works great in point-to-point distribution, letting the
server switch between streams in response to the size of a send buffer,
but it's not very suitable in large-scale multicast environments where a
server publishes the stream just once and there's no flow-control.
"Self-adapting" (layered) streams with incremental quality enhancements
are more elegant there IMHO.

> also you should keep in mind that all the scalability stuff will be 
> worse quality per bitrate than streams optimized for a single bitrate

No such thing as a free lunch, I fully realize that, but the increased
resilience to congestion ought to be worth something :)

> also maybe b frames provide already enough scalability for your needs
> and they are fully supported on all decoders

True, but we feel that gently lowering the spatial resolution or
signal-to-noise ratio tends to give a smoother user experience than
cutting down on the framerate.

We've built a proof-of-concept encoder that scatters DCT coefficients
over layers (we didn't do P- or B-frames, so we were cheating a bit) and
that really works well. As little as 20% of the required bandwidth still
gives a smooth 25fps playback. Much more blurry, but no artifacts.
If the input is high quality (we use DVD), then a significant decrease
in signal-to-noise can still be quite acceptable.

> PS2: iam of course not against adding FGS/... support to ffmpeg if 
> someone submits a VERY clean patch which also can be disabled at 
> compile time

Happy to contribute patches if we manage to make it work, but I doubt
they'll be able to match the efficiency and cleanness of the ffmpeg
codebase..

Maybe the overall skepticism towards complex scalable encoding is
justified, but at the moment I think they're able to offer an elegant
solution to live streaming bandwidth fluctuations, so I do want to give
it a try.

cheers,
Erik
-- 
You have acquired a scroll entitled 'irk gleknow mizk'(n).--More--

This is an IBM Manual scroll.--More--

You are permanently confused.
		-- Dave Decot