[FFmpeg-devel] libavdevice: PTS stability and risks of drift

[Luca Abeni]

Hi,

Olivier Guilyardi wrote:
> Hi everyone,
> 
> I'm currently writing a JACK indev for libavdevice, and trying to understand how
> PTS are/must be computed, and how they're used for A/V sync.
> 
> First, in the packets returned by the libavdevice read_packet() function, is the
> PTS used as a running counter, or does its absolute value matter?

In my understanding, the absolute time matters (otherwise, it would be impossible
to synchronise a video input with an audio input, for example).


I do not know what to say about the "gettimeofday() vs monotonic clock" issue,
since I do not know NTP too much (can it really cause such large "jumps" in the
system time?).

Anyway, I believe that directly using gettimeofday() (or some other call
providing the current system time) is problematic, because we need to know
the time when the data was sampled, not the time when we read it from the
device... In this regard, I believe that both the v4l2 input and the ALSA
input use the correct value for PTS. Does the Jack API provide some similar
call for reading the time when the data was sampled?

> If the PTS absolute value doesn't matter, why not use the POSIX
> clock_gettime(CLOCK_MONOTONIC, ..) function, or, on osx, mach_absolute_time()
> [1] if available?

This has the same problem I pointed out above.


> This problem also seems to affect the v4l2 demuxer, which computes the pts out
> of the timestamp returned by the VIDIOC_QBUF ioctl, which is itself based on
> gettimeofday() [2]

Well, v4l2 is using the timestamp returned by the driver, which is the best thing
we can use... In my experience, it is pretty accurate (I had some problems only
with some USB drivers) and I've never seen the time going backward.
Anyway, I'd say that if such thing happens the problem must be fixed in video4linux2
(and in the driver), not in libavdevice.


> I took a look at the dv1394 demuxer (libavdevice/dv1394.c), and I'm unsure how
> the video PTS is computed. Is it based on the system clock in any way or on the
> dv hardware clock?

I do not know dv1394 very much, but it seems to me that the timestamps are read
from the dv stream, so I suspect the dv hardware clock is used.


> My first impression is that the PTS is computed out of the number of retrieved
> frames from the DV hardware, which, if this is interrupt-driven, means the dv
> hardware clock is used. This is likely to drift away from the system time, and
> thus get out of sync of, say, audio recorded using oss (NTP or not). Isn't it?

I guess you cannot synchronise dv1394 video with OSS audio. But a DV stream
coming from firewire contains both audio and video, so you should not need such
a synchronisation.

Yes, I suspect you can have a drift between the DV timestamps and system time,
but such drift can be compensated at the application level (something similar
to ffmpeg's vsync and async).


				Luca