[FFmpeg-devel] libavdevice: PTS stability and risks of drift

[Luca Abeni]

Hi,

Olivier Guilyardi wrote:
> Luca Abeni wrote:
> 
>> I guess you cannot synchronise dv1394 video with OSS audio. But a DV stream
>> coming from firewire contains both audio and video, so you should not need such
>> a synchronisation.
> 
> This is untrue. We are actually working on a university broadcasting
> installation, where the video will come from a DV device, and the audio from
> other devices.

Ok, sorry; I did not consider this setup. Anyway, in this case I suspect
audio/video synchronisation might become problematic. As far as I can see,
using the DV camera clock as a reference for the dv video timestamps is the
only way to assign proper timestamps (which represent the time when the video
data has been sampled, not when the computer received it - I have seen some
DV cameras introducing a noticeable delay before sending the video on the
firewire).
And I do not see an easy way to synchronise the DV camera clock with the
computer's clock. Maybe you can just assume that there is some offset between
the dv video timestamps and the audio timestamps, and hand-tune such offset
so that audio and video are synchronised.


>> Yes, I suspect you can have a drift between the DV timestamps and system time,
>> but such drift can be compensated at the application level (something similar
>> to ffmpeg's vsync and async).
> 
> If I understand correctly, you recommend that each of libavdevice's demuxers
> compute the PTS out of the specific hardware device clock it handles, which will
> result in inevitable drifts.

Yes; I believe that libavdevice should return the timestamps provided by the
hardware (or by the drivers), without trying to be too smart.
All kinds of compensations can be performed at application level.


> For your information, we have measured the drift between the audio and system
> time using 5 different audio sound cards, and obtained linear drifts ranging
> from -3ms to +4ms / minutes. That means about 200ms after one hour of recording.

I guess your computer is not properly cooled :).
Yes, I've seen similar drifts in the past, and the "wrong" clock was the one on
the computer mainboard. My experience is that audio cards are generally more
reliable than the system clock...


> And these are extremely varying, so that I can't think of anything more
> difficult than figuring out the right value for -async or -vsync.

Uh?


> To my understanding, one of the roles of libavdevice is to provide demuxers that
> play well *together*. If each demuxer uses its own clock, then this task is not
> fulfilled IMO.

I think you are just proposing to move in libavdevice something that I believe
should be at an upper level.


> The solution is to use a single clock that both the audio and video demuxers
> have access to: the system clock. This does cause jitters but they can be
> reduced by using a Delay Locked Loop filter:
> 
> http://www.kokkinizita.net/papers/usingdll.pdf
> http://lalists.stanford.edu/lad/2009/01/0095.html

And what's the problem in doing this at application level? The system clock
can be accessed by the application without problems. If you move this mechanism
in libavdevice, you discard the timestamps coming from the device/driver, and I
do not see the advantage in doing this.



				Luca