[FFmpeg-user] Audio/video stream are out of sync?

[Erik Dobberkau]

Am Donnerstag, 9. Februar 2017 schrieb christina zou :

> I've made some progress thanks to advice from the IRC channel. I increased
> my pipe max buffer size as high as I could:
>
> sysctl fs.pipe-max-size=1048576
>
> I also lowered my video resolution and bitrate.
>
> Now, with the below command I've managed 3 minutes of fully synced
> audio/video livestream to Youtube from my Pi Zero. At the 3 minute mark,
> there's a slight desync, and the audio goes about 0.5s ahead of the video.
>
> That's my next question. I don't mind if there's a stutter every 3 minutes,
> but I need the A/V sync to recover after that. So if I lose a few audio
> frames, I need the video to catch up (or audio to slow down). Any
> suggestions about how to do this (either ffmpeg param, or script)?
>
> Here's my current command:
>
> sysctl fs.pipe-max-size=1048576
>
> *sudo rm temp_audio.v*
>
> *sudo rm temp_video.h264*
>
> *mkfifo temp_audio.v*
>
> *mkfifo temp_video.h264*
>
> *arecord -Dmic_sv -c2 -r48000 -fS32_LE -twav temp_audio.v & \*
>
> *raspivid *-w *640* -h *480** -fps 10 -v -b 1000000 -o temp_video.h264 -t 0
> & \*
>
> ~/special/ffmpeg/ffmpeg* \*
>
> *    -framerate 10 \*
>
> *    -i temp_video.h264 \*
>
> *    -i temp_audio.v \*
>
> *    -ab 24k \*
>
> *    -c:v copy \*
>
> *    -c:a aac \*
>
> *    -report \*
>
> *    -f flv **rtmp://209.85.230.23/live2/KEY*
>
> On Wed, Feb 8, 2017 at 3:50 PM, christina zou <zou.christina at gmail.com
> <javascript:;>>
> wrote:
>
> > Carl,
> >
> > I spent a lot of time trying to use ffmpeg alone (alsa/v4l2), but it
> > produces extreme stuttering on the audio track. Using my approach of
> > writing to named pipes and sending those into ffmpeg, I add a bit of
> > latency but ensure that there's no loss of frames. This was the approach
> > suggested to me by the ffmpeg IRC channel. I'm on a Pi Zero (single
> core),
> > by the way.
> >
> > Is there any way to align the two input streams' timestamps, if they are
> > both being read from named pipes?
> >
> > Here is my full console output: http://pastebin.com/phmiphaL
> >
> > Thanks,
> > Christina
> >
> >
> >
> > On Wed, Feb 8, 2017 at 3:02 PM, Carl Eugen Hoyos <ceffmpeg at gmail.com
> <javascript:;>>
> > wrote:
> >
> >> 2017-02-08 22:57 GMT+01:00 christina zou <zou.christina at gmail.com
> <javascript:;>>:
> >>
> >> > *arecord -Dmic_sv -c2 -r48000 -fS32_LE -twav temp_audio.v & \*
> >>
> >> Why?
> >> (See below)
> >>
> >> > *raspivid -fps 10 -v -b 3000000 -o temp_video.h264 -t 0 & \*
> >>
> >> Is this not possible with ffmpeg alone?
> >>
> >> > ~/special/ffmpeg/ffmpeg* \*
> >> > *    -framerate 10 \*
> >> > *    -re \*
> >> > *    -i temp_video.h264 \*
> >> > *    -i temp_audio.v \*
> >>
> >> (Complete, uncut console output missing.)
> >> How are these two input streams supposed to be synced?
> >>
> >> Normally, you would use (for example) alsa and v4l2 input
> >> and hope that the two drivers both provide wallclock timestamps.
> >>
> >> Carl Eugen
>

My suspicion is the transcoder the stream is being fed to doesn't cope well
with the framerate. In my experience it's most stable when using either 25
or 30fps with a keyframe interval of 50 or 60 frames maximum. Also best to
use standard bitrates, especially with audio, to avoid drifting.

Erik