[FFmpeg-devel] [PATCH] ALSA: fix timefilter divergence

[Michael Niedermayer]

On Sat, Feb 18, 2012 at 01:33:38PM +0100, Nicolas George wrote:
> L'octidi 28 pluviôse, an CCXX, Michael Niedermayer a écrit :
> > of course
> > what we have are noisy time values at arbitrary time intervals
> > we also know these time intervals to a pretty good approximation.
> > There is a systematic error in the time intervals due to clocks not
> > being exact while i assumed that this systematic error is sufficiently
> > close to constant over time. That is 44100 hz may actually be
> > 44098 but wont drift slowly around between that and 44102. If this
> > assumtation is wrong the filter becomes less optimal but
> > should not catastrophically fail.
> > 
> > The way i implemented that is quite simply to take the first and
> > last timestamps and the "number of samples" to get the true sample
> > rate and use this instead of a exponentially decaying average.
> 
> I have some plots that will, hopefully, prove helpful. The attached
> standalone program (I copy-pasted the timefilter implementations to easily
> have both versions in the same program) produce data that can be plotted
> using gnuplot for example. The first gnuplot command in the comments is what
> I am looking at.
> 
> I can post PNGs on my webpage, but it is probably best to be able to zoom
> around and adjust parameters.
> 
> The input is a 48 kHz stream recorded by chunks of 512 samples (this is a
> typical default ALSA setting), with a ±16 samples (0.33 ms) white noise on
> the measured times.
> 
> After 10 seconds, the system clock leaps forward by 0.2 seconds.
> 
> After 15 seconds, the system clock speed increases by 10%.
> 
> The plot shows the difference between the "real" time (which includes the
> leap forward and the speed change: the purpose of the timefilter is to
> eliminate the noise, nothing else) and, in that order, the unfiltered input,
> the output of the current timefilter and the output of the timefilter as
> modified by your patch.
> 
> If you look at the first 10 seconds, you see that both filters perform
> equally well; that is feedback2_factor working, and they share the same
> logic here.
> 
> If you zoom on the time discontinuity, at first glance the modified
> timefilter seems to smooth it faster. But you need to zoom enough on the
> ordinates, and keep enough abscissas, and optionally turn the noise off:
> 1.5 second after the discontinuity, the original timefilter has gone back to
> 0. On the other hand, at the same time the modified timefilter is still at
> +1 ms (about one audio packet), and after 5 seconds, still at +0.8 ms.
> 
> Now let us zoom on the speed change (possibly turning the discontinuity
> off): this time, there is no ambiguity: the original timefilter settles in
> about 1.5 second while after 5 seconds, the modified timefilter is still
> worse than the worst result of the original one. Note that the result are
> exactly in the same proportion with a much more reasonable speed change
> (+1/1000).
>

> > It might be better to detect this case and force a (partial?) reset of
> > the internal state.
> 
> I believe the point of using a second-order timefilter is to avoid this:
> instead of various heuristics for the problem we thought of, it's all in the
> mathematical model of the filter.

Well, but this mathematical model doesnt really match reality

My patch would exactly calculate the rate while leaving the actual
values to a first order filter. In the model that the rate is constant.
If its not constant but instead piecewise constant, like in your test
here, it of course doesnt work nearly as well.

One could create a filter that performs better on such piecewise
constant rate input than any clasical second-order filter.
That no doubt would perform worse then on for example piecewise
polynomials

And any functional classic second-order filter would catastrophically
break down in case some large discontinuities or outliers are
introduced.

I think we should first find a reasonable model for the data and noise
before trying to find a filter that seperates them

The 2nd order Timefilter seems to assume that noise is mostly high
frequency and data is mostly low frequency.

My patch would in addition have assumed a near constant rate.

The discontinuities in your test would indicate though that there is
some non zero high frequency data in addition to the low frequency
and neither filter handles this well as they try to discard noise by
discarding high frequency. Blurring the data.
This should become quite catastrophic in case of large discontinuities
as  these would have very large high frequency components that actually
would far exceed the noise.
I thus think if the filter is intended to handle discontinuities it
has to do more than just a 2nd order filter. Like detecting such
discontinuities and reseting its state or adjusting its coefficients.



> 
> > Also theres another issue, if theres a significnat correction from
> > NTPD that may mean the previous hours of recording used wrong
> > timestamps. And if there are 2 streams (audio & video) which use
> > seperate timefilter, they may with either implementation recover at
> > different rates makeing the whole issue even worse
> 
> Indeed. But we can not do anything about it, timefilter or no timefilter. If
> someone wants to sync streams with different time sources, it is their
> responsibility to keep the time sources in sync.

iam not sure if we misunderstand each other here.
but if we have audio and video both from the same time source (wall
clock) and this gets shifted by a few hours due to NTP then the 2 will
recover very differntly because each has a seperate timefilter
one run per video frame and one run per audio packet.
the AV sync errors between the packets after such NTP discontinuity
should be in the order of minutes and could oscilate around both audio
and video being late before converging to the correct value


[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

While the State exists there can be no freedom; when there is freedom there
will be no State. -- Vladimir Lenin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20120219/c0f5e8e1/attachment.asc>