[FFmpeg-user] Unable to sync audio and video

[Livio Tenze]

Dear Nicolas,

I am following your suggestion. I added one thread to acquire the audio
stream. Currently I have only the thread for the audio stream; I will
implement threads also for video acquisition.
The see a strange behaviour:
===============================================
Crop frame system is 1
Dummy read one frame
VIDEO PTS START from video1
AUDIO: void AudioInputThread::startLoop()
AUDIO: audioThread started NOW!
PTS START video: 83565572 audio: -1
stream v:1/15360 a:1/22050
AUDIO: audio: 0.000 0.185759 = 0.186 1614099985303395
set record for 15.00 minutes
AUDIO: head 0, tail 1
start recording
AUDIO: audio: 1.840 0.185759 = 2.026 1614099987143524
AUDIO: head 0, tail 2
AUDIO: audio: 2.068 0.185759 = 2.253 1614099987371002
AUDIO: head 0, tail 3
AUDIO: audio: 1.735 0.185759 = 1.920 1614099987037933
AUDIO: head 0, tail 4
AUDIO: audio: 1.476 0.185759 = 1.662 1614099986779509
AUDIO: head 0, tail 5
AUDIO: audio: 1.293 0.185759 = 1.479 1614099986596620
AUDIO: head 0, tail 6
AUDIO: audio: 1.116 0.185759 = 1.302 1614099986419398
AUDIO: head 0, tail 7
video1: 0.032 83597620 33333
>>> video timer0 0
>>> video timer1 20
===============================================
Please, take into account lines starting with "AUDIO: " tag.
This is a log of my application. I notice that the audio estimated position
(it is estimated taking into account the first received pts) starts
increasing, then decreases, and, after some packets, the behaviour is
monotonic and right. Moreover, the distance from the starting PTS and the
second one is almost 2 seconds.
Have you any suggestions about this strange behaviour?

Thanks again.
Livius

On Wed, Feb 17, 2021 at 2:20 PM Nicolas George <george at nsup.org> wrote:

> Livio Tenze (12021-02-17):
> > No, it is not, because the audio and the video streams come from two
> > different sources: one webcam and an external microphone. The starting
> PTS
>
> Different sources do not mean different timestamps origins. In fact, you
> NEED the same timestamps origin if you want to sync.
>
> > values are different for two audio and video sources. How should I  get
> the
> > same origin with this configuration? Please suggest how to treat this
> case.
>
> Ideally, the documentation of the device you are using should be stating
> it. For example:
>
> http://ffmpeg.org/ffmpeg-all.html#video4linux2_002c-v4l2
>
> "Depending on the kernel version and configuration, the timestamps may
> be derived from the real time clock (origin at the Unix Epoch) or the
> monotonic clock (origin usually at boot time, unaffected by NTP or
> manual changes to the clock)."
>
> Unfortunately this is rather the exception than the norm.
>
> If the documentation does not say it, then you need to experiment:
> actually look at the timestamps, look at at the actual time when you
> capture, do some arithmetic to find the origin, and try to guess what it
> corresponds to.
>
> And then propose a patch to document the origin of timestamps for that
> particular device.
>
> Regards,
>
> --
>   Nicolas George
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".