[FFmpeg-user] Please help with weird video problems

[Anatoly]

On Wed, 8 Feb 2023 20:41:46 +0300
Alexander Gribanov <sunrisecoder at gmail.com> wrote:

> ср, 8 февр. 2023 г. в 17:53, Anatoly <anatoly at kazanfieldhockey.ru>:
> 
> > On Wed, 8 Feb 2023 14:46:04 +0300
> > Alexander Gribanov <sunrisecoder at gmail.com> wrote:
> >  
> > > ср, 8 февр. 2023 г. в 02:11, Anatoly
> > > <anatoly at kazanfieldhockey.ru>: 
> > > > On Tue, 7 Feb 2023 22:00:35 +0300
> > > > Alexander Gribanov <sunrisecoder at gmail.com> wrote:
> > > >  
> > > > > Hello!
> > > > >
> > > > > Here is a downloaded Facebook Livestream video which I'm
> > > > > trying to process:
> > > > >  
> > > >  
> > https://drive.google.com/file/d/1mVkQbzVSzhOlQbT2mB1J2yvPFV5kl5Dq/view?usp=share_link
> >  
> > > >  
> > > > >
> > > > > I can play it correctly in MPC-HC Media Player Classic, but
> > > > > when I try to add this video to Adobe Premiere 2022, there
> > > > > around 3:00:00 position audio is about 40 seconds ahead of
> > > > > video...
> > > > >
> > > > > I tried to remux and re-encode this video using ffmpeg, it
> > > > > showed a lot of warnings like this:
> > > > > [aac @ 000001c1bfe68e40] env_facs_q 254 is invalid
> > > > >
> > > > > I tried different options like "-vsync cfr", "-fps_mode drop"
> > > > > and some other stuff which I don't remember already, but
> > > > > nothing helped :( In MPC-HC it was ok, but in Adobe Premiere
> > > > > still 40-seconds a/v unsync.
> > > > >
> > > > > I assume that the problem is with timestamps on the audio
> > > > > track.
> > > > >
> > > > > I would like to make using ffmpeg just a simple cfr video
> > > > > 29.97 fps / audio 48kHz without any timestamp gaps and
> > > > > offsets.
> > > > >
> > > > > Could anybody please help me to understand how to analyze and
> > > > > diagnose such problems in such videos to be sure what the
> > > > > exact problem is?
> > > > >
> > > > > Could You please help me find the solution (i.e. options or
> > > > > subcommands to ffmpeg).
> > > > >
> > > > > Thank You very much in advance :)  
> > > >
> > > > Maybe re-encode it with constant framerate using 'framerate'
> > > > filter? ffmpeg -i in.mp4 -vf framerate=fps=29.97 -c:a copy -c:v
> > > > libx264 -b:v 800k out.mp4
> > > >  
> > >
> > > Hello Anatoly!
> > > Thank You very much for the reply, but it didn't help, the problem
> > > remains :(
> > >
> > > Any other ideas?  
> >  
> 
> Hello Anatoly! Thank You very very much for the reply.
> Handbrake 1.6.1 helped me, but still, I would like to figure out what
> the exact problem with my file is.
First of all I'm not deep expert, just a user like you, so I may be
wrong somewhere.
While playing your file with mplayer, launched from console, I've
noticed warnings:
[aac @ 0x802b535f0]env_facs_q 255 is invalid
and I hear audio dropout (silence, about 1 frame) with every message.
From that I've got this idea.
mplayer and ffmpeg share same library 'libavcodec.so' on my system.
This library contains 'default' aac codec for both. I see same warnings
while transcoding with ffmpeg, but ffmpeg behaves differently. Instead
of silence broken chunks of audio, it skips them, making audio shorter.
I have ffmpeg linked to another aac decoding library 'fdk-aac', and
tried to use it ac decoder:
ffmpeg -c:a libfdk_aac -i in.mp4...
No any messages, but result was the same.
> 
> > I took a deeper look at your file. It seems that aac audio stream
> > contains undecodable frames (by all decoders I have aac, aac_fixed,
> > libfdk_aac).  
> 
> 1. Do You think Handbrake has the decoder which was able to decode or
> it just muted broken frames?
Just muted. I clearly hear this. I think I'll try to make a picture and
send as attachment in next mail on topic. As I understand it, handbrake
has same aac decoding libraries, maybe slightly different versions,
linked into it statically.
> 2. I have an assumption that those frames could be really broken due
> to bad internet during the live stream
> and incomplete frame chunks were saved as they are (incomplete) in the
> audio stream, what do You think?
That would be strange if only audio stream were damaged, unless you
already transcoded video part of this file (and thus fixed broken video
frames). Btw as I understand it, video part contains less frames than
it should be, but there's no broken frames, so it's ok for software
that can handle variable framerate.
% ffprobe -i in.mp4
...
  Duration: 03:23:47.69, start: 0.000000, bitrate: 47 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p,
    432x768, 492 kb/s, 29.42 fps, 29.97 tbr, 16k tbn, 2k tbc (default)
> 
> And ffmpeg just drops undecodable audio parts instead of
> > silence them.  
> 
> Is there any way to hint to ffmpeg not to drop them but keep silent?
I'd like to know too. Maybe one with better knowledge come up and tell
us.
> 
> This results in audio track becomes shorter than video and
> > out of sync.  
> 
> Yes, I agree
> 
> 
> > Mplayer handles this properly, playing audio in sync, but
> > with very short and rare audio droputs.  
> 
> How is it possible?
> If ffmpeg dropped about 50 seconds of audio, it doesn't look like
> "some very rare samples"...
your video has lenght of 3[h] * 3600 + 23[m] * 60 = 12180[s]
let's assume average dropout lenght is 17ms
then we should have 1 dropout per 12180 / (50 / 0.017) = 4.14s average
> So I assume, after it dropped some samples, it should add kind of
> timestamp to the next audio block,
If you mean ffmpeg then, imagine we transcoding audio
to uncompressed pcm_s16le, our output is just sample by sample. For most
muxers (or filters), as I can understand it this stream should be
continuos, so only two options: cut out (skip) broken part or replace
with silence. Your broken source, being transcoded with ffmpeg, will
play wrong in any software, and MPC too. Audio decoded to plain
samples, then (possible) encoded back to something, but missing samples
already just skipped.
> which MPC-HC can digest, but Adobe Premiere can not, is it right? Or
> how does it work?
If we'll give your broken source (original broken source, not improperly
transcoded source) to MPC, Mplayer or Handbrake, they can handle
situation properly, replacing broken samples with silence.
But Premiere and ffmpeg can not. They start to skip broken samples.
> Could You please explain in a few words or give me a link to an
> article to read about it?
> 
> 
> > But mencoder does same way as
> > ffmpeg.
> >  
> What is the mencoder? Google says that it is a part of MPC-HC, but
> I'm not sure... is it right?
Mencoder is a part of Mplayer
http://www.mplayerhq.hu
Not sure how MPC-HC relates to Mplayer, I think they are completely
different projects.
> 
> 
> > I succeed transcoding this file with Handbrake
> > https://handbrake.fr, I have version 1.3.3 installed with FDK aac.  
> 
> Yes, this worked fine, I tried with Hardbrake 1.6.1, then used the
> output as input for Adobe Premiere 2022,
> and the result file was not out of sync.
> 
> 
> > This also results in short
> > audio dropouts like playing with Mplayer.
> >  
> Yes, I will listen to it carefully to see whether there are some
> dropouts or not.
> 
> Anyway, thank You very very much, at least, I found a working
> solution :)