[MPlayer-dev-eng] [PATCH] A/V sync improvement for TV streams

[Laurent]

On 10/30/08, Uoti Urpala <uoti.urpala at pp1.inet.fi> wrote:
> On Mon, 2008-10-27 at 16:57 -0400, Laurent wrote:
> > On 10/26/08, Uoti Urpala <uoti.urpala at pp1.inet.fi> wrote:
> > > On Mon, 2008-10-20 at 19:23 -0400, Laurent wrote:
> > > If the status is permanent then this is not a good fix. A better simple
> > > workaround would be to pause playback for a while if there isn't enough
> > > data.
> > >
> > I would rather adjust the playspeed to compensate for the audio drift.
>
> That could be better when you have a source like a TV card that returns
> data at a precise rate, but it's also trickier to implement and probably
> no better for network streams where varying lag can make accurate speed
> estimates impossible. Something like it is needed though if desync in
> the other direction is a problem (MPlayer falling increasingly far
> behind the source).

Certainly. Adapting playspeed makes only sense for "live" streams,
such as TV cards and network Live streams. It does not for On Demand
streams/files where mplayer can read ahead the data.

>
> > Though, this does not fix the sync issue.
>
> Why do you say that? Any mechanism that prevents MPlayer from getting
> "too far ahead" so that there is not enough data to fill the audio
> buffers avoids the need to handle sync with partially filled buffers.
>
> Which demuxer is used in your case btw? I think most of them block if
> there isn't enough data instead of returning (temporary) EOF.

Yes. This is another issue I would like to address later.
Blocking in the TV acquisition code, when using MPlayer and
immediatemode, does not solve the problem: the output audio buffer can
still not be filled.
This is like trying to fill a leaky bucket while the leak rate is
greater than the fill rate: it cannot work unless you modify the
rates, which blocking does not.

What I experienced, is that sleeps in the TV demuxer (waiting for
audio, as audio is always read before video) brakes the sync system.
As a result I have degradated video framerate: basically, mplayer
waits longer in the demuxer than in the video sync sleep function.
(IMO, waiting anywhere but in mplayer.c should never happen).

As someone said earlier, v4l2 inserts blank audio frames as a mean to
increase the audio input rate (when using mplayer+immediate mode and
waits when using mencoder). This sounds like the fastest workaround.
DShow TV does not do so.



>
> The amount buffered depends on the audio output driver used. If it's
> much larger than needed to avoid underrun then it could be beneficial to
> reduce the buffer size (if the output API supports that). The drift
> speed depends on the difference between source and audio card playback
> rates; I've heard of audio cards with quite significant differences from
> nominal playback rates.
>

At one point, reducing the buffer size would be beneficial to me.
Currently, changing a TV channel takes one second, while 250ms or less
would be much closer to an acceptable value for interactivity,
whatever the audio driver.


> > However, during all those hours, MPlayer will loose sync, unless the
> > EOF issue is fixed.
>
> I agree it will lose sync, but I don't agree with the way you're trying
> to avoid it. MPlayer shouldn't keep playing the file in a constant
> degraded state with EOF set and audio buffers filled below the standard
> amount. I also suspect that most other people would benefit
> significantly less from your hack; typical audio output buffering at
> least on Linux is much less than a second, and I also expect higher
> drift rates to be common.

I made additional tests. Bottom line is that the demuxer EOF condition
cannot be replaced by a detection of end-of-buffer at the audio driver
level. This is primarily due to the fact the audio drivers are taking
counteractions to avoid buffer underrun.

For example, Alsa's get_delay() is not const in its behavior and
modify the audio buffer pointers when approaching buffer underrun (to
perform a replay), .
Also, in the DirectSound driver, get_delay() assumes that the buffer
has been filled, and is unreliable otherwise.

So, my patch related to EOF has nasty side effects. I'll find out
another way to achieve my goal.


Generally speaking, the audio buffer flow controls are widely spread
in the MPlayer code, (from demuxer to audio output), which makes it
extremely hard to control it propertly.
I guess there is code duplication because of this too (eg blank frames
insertions in TV demuxers, underrun prevention methods in audio output
drivers).


What I think would be a good step forward, is a unified audio buffer
flow control, in mplayer.c:fill_audio_out_buffers, for example, that
would:
* control buffer size (delay from demuxer to output).
* add blank frames to avoid buffer underrun
* replay old frames to avoid buffer underrun
* drop frames to avoid buffer overflow
* adjust play speed to avoid both underrun and overflow (Live stream)
All this with configurable parameters to define what the user would
expect in case of buffer over/under-flow.

Does this make sense?



For the moment, could we just commit the ad_pcm patch?

Thanks,