[FFmpeg-user] Can't get a/v synchronisation right with mpeg transport stream that has gaps in it

[Erik Slagter]

Hi people,

I'm having this problem for years and years (so it's not a recent
problem or regression).

I make recordings on my enigma2 based settopbox. It saves the recordings
as the original transport stream, but with filtering of only the
requested service pids.

There is a feature to edit the recorded services, in a basic way. Very
helpful to edit out commercials. Problem is that it makes cuts in a very
"rude" way, it justs cuts the file at a certain position, in the middle
of all kinds of open packets and streams. On playback it isn't such a
big deal, because the damaged packets are discarded and the DVB hardware
resyncs on the timestamps pretty quickly.

After the crude editing, I like to perform a real editing run, using
kdenlive (or any other NLVE, for that case). I've found that all of them
have problems when fed with a "damaged" transport stream, like this. So
what I do, is transcode the stream to something with mjpeg and pcm
codecs, intra frames only, very easy on the editor.

This is the problem: after every timestamp discontinuity ("cut" or
"gap"), ffmpeg resyncs audio and video BUT it doesn't do it 100%
correctly. Sound remains a bit "in front" of the video, skewed a bit.
Also, it gets worse at every cut, everytime about 200-300 ms is added to
the skew. That's not what I'd expect?

I've tried these options to force synchronisation:

- -vsync 1 (not per sé required, but I want the audio and video streams
in the container in such a way that they're not skewed, so if the editor
decides to demux and throw away the timestamps, it still works)
- -r 25 (material is 25 fps, tried with and without, doesn't matter)
- filter:a "
aresample=async=10000:min_comp=0.1:min_hard_comp=0.1:max_soft_comp=10000:first_pts=0"

None of this helps. I can see ffmpeg dropping and dupping frames, but it
doesn't help to fix the slow drift.

I was under the impression that ffmpeg would automatically base the
synchronisation of the (single) audio stream on the (single) video
stream. Is this a correct assumption or do I need to specify an explicit
"-map" statement?

It looks like ffmpeg takes care that frames are dupped/dropped so the
frame rate keeps "solid" 25 fps (or whatever the source frame rate is
specified as or is specified with -r), but it seems it does not keep the
audio stream synchronised.

Thanks for help,
Erik.