[FFmpeg-user] Unable to sync audio and video

Livio Tenze ltenze at gmail.com
Tue Feb 16 09:56:25 EET 2021


Thank you for your answer.

On Mon, Feb 15, 2021 at 7:13 PM Nicolas George <george at nsup.org> wrote:

> Livio Tenze (12021-02-15):
> > -The audio source comes from alsa: here I see really strange behaviour.
> The
> > pts value of the second packet with respect to the first pts packet seems
> > to be "delayed" from 0.4s to 1.8s. I don't understand this behaviour: I
> > checked the time from start to the second packet and the elapsed time
> > cannot be justified (ms and not s).
>
> Can you observe the same phenomenon using ffprobe?
>
> ffprobe -f alsa -i default -show_packets -of compact | head -n 50 | less -S
>
> would be a good way to look.
>
> Do you observe warnings on the console?
>

Ok, I will check.

>
> How does your application handle parallelism between encoding and
> reading from devices?
>

At the moment I have only one thread (the main one, as in doc/examples).
The process I implemented is the following:
0) Initialize PTS output frame (one PTS for video and another for audio)
1) check PTS difference between PTS video and audio
2) if ptsvideo>ptsaudio then decode one video frame, and encode it in the
output stream
    otherwise decode audio frame, and it in the output stream.
3) go to point 1

This is a chunk of my code.

AVRational video_timebase = out_video_stream->time_base;

        AVRational audio_timebase = out_audio_stream->time_base;

        if (av_compare_ts(audio_pts, audio_timebase,

                          video_pts, video_timebase) < 0) {


            if (decodeAudio(picture))

                encode_audio(picture, audio_pts, out_audio_stream);

        }

        else {

            // https://stackoverflow.com/questions/49280566/ffmpeg-c-convert-compress-a-single-image-out-of-buffer

            // https://stackoverflow.com/questions/49446335/ffmpeg-h264-encode-each-single-image

            if (decodeVideo(picture, (picture2) ? &picture2 : NULL))

                encode_video(picture, picture2, video_pts, out_video_stream);

        }



> > -Finally, when I write audio and video in the output MP4 stream (I use
> > libx264 and libfdk_aac for video and audio respectively) the audio stream
> > is delayed.
>
> Do the timestamps between audio and video match before encoding?
>

Yes, please check above. I use two PTS values: one for audio and one for
video. The PTS from the input stream differ from the ones of the output
stream. I check the distance between the first packet.pts and the current:
here I saw that strange behaviour where the audio elapsed time is very high
with respect to the video input stream.
Do you think that this behaviour is due to missing multithread? I found the
above flowchart in doc/examples where no threads are used.

>
> Regards,
>
> --
>   Nicolas George
>

Many thanks for your help.
Livius


> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".


More information about the ffmpeg-user mailing list