[PATCH] -oac copy with ac3 sound and lavf ts demuxer issues (Was: Re: [MEncoder-users] Video/Audio asynchronous)

[Matthias Hopf]

Hi,

I noted (and participated) in a discussion on mencoder-users, that
probably fits much better in mplayer-dev-eng now.
You'll find a summary of the discussion that lead to this at the end of
this mail. You might want to read it first if the summary doesn't make
sense to you.


Summary:

- "-demuxer lavf" breaks "-oac copy" subtly - still works with mplayer
  and xine, but not on any other player I tried
- remuxing doesn't help
- the extracted ac3 looks good (5.1 channel ac3, no broken packets)
- the extracted ac3 from the broken avi is exactly the same as the
  extracted ac3 from the original file
- Sample uploaded as lavf_copy_broken_audio_hdtv.ts
- Patch attached, but not to be applied without *major* revision

The sample is only 1MB, at 1080i, which gives you only approx 1 second
of video and audio, but that's enough for the tests. Not for A/V sync
issues, but that seems to work out fine as later tests show.


Now I've compared the avi files that were produced by choosing lavf and
mpegts demuxers, and it showed, that mpegts produced a constant bitrate
stream (dwSampleSize = 1), while lavf produced a VBR stream. Also,
nBlockAlign, wBitsPerSample, nSamplesPerSecond wasn't set by lavf.
nChannels is set differently (2 by mpegts, 6 by lavf), but that is
irrelevant, and lavf is IMHO actually doing the right thing here.

The culprit for not finding any audio at all in the other players I
tried (compared to xine and mplayer which both found audio) was the
missing nBlockAlign. Additionally, without setting/changing the constant
bitrate information (dwSampleSize, dwRate, dwScale), stream duration was
way off, and no sound was actually heard (though at least detected). I
didn't really try to create correct VBR settings here, though, but just
copied the ones from the avi created by mpegts.

With this information I finally created the attached patch, which fixes
the issues for me. Though I hardly know what I'm doing in this area. In
fact, it should be considered rather a discussion base than a real patch.
But it *did* fix *my* problems (except for wrong wBitsPerSample, which
is cosmetic only) so far, and I get perfectly synchronous video and
audio with it. So far.


So the remaining question is basically twofold:

- Why is nBlockAlign, wBitsPerSample not set correctly?

- Why is dwRate/dwScale/dwStart/dwLength/dwSuggestedBufferSize wrong?
  What are the right values?
  I assume codec->frame_size and/or codec->sample_rate are wrong.
  lavf gets it right for constant bit rate, but an unknown ac3 stream is
  inherrently variable, so a VBR approach is the right thing (TM) to do.

Can anyone with a deeper understanding how demuxing in mplayer actually
works comment on my findings?


Thanks

Matthias

===========

Now for the background discussion:

Markus Bonet:
> I am doing some conversions between mpeg2 (TS) to mpeg4 (vxid) via
> mencoder.
> The TS isn't demultiplexed as I am using the TS directly as input.
>
> When done, the result is slightly asynchronous. mencoder itself reports
> a very slight delay (0,048s) but the result is even more asynchronous.
>
> Setting -delay 0 didn't help.

Nico Sabbi:
> try with -demuxer lavf

Markus Bonet:
> Nice! Seemed to help. Thank you.

Me:
> Good to know that others have the same issues. I get delays between
> +250ms and -200ms.
> 
> > try with -demuxer lavf
> 
> This doesn't work correctly for -oac copy and ac3 streams. The produced
> file plays fine with mplayer, but with no other player (I think I
> remember xine worked as well). Media player classic (guliverkli)
> complains that there's no audio, VirtualDub even crashes as soon as
> anything is to be done with audio.
> FWIW, -demuxer mpegts (default) works (modulo the delay).
> 
> Is this a known issue (actually two known issues), or should I upload a
> sample?

Nico Sabbi:
> Yes, please.

This resulted in the sample upload and further analysis:
mencoder -oac copy                  -ovc xvid -xvidencopts 'fixed_quant=4' -o tst0_mpegts.avi lavf_copy_broken_audio_hdtv.ts
mencoder -oac copy    -demuxer lavf -ovc xvid -xvidencopts 'fixed_quant=4' -o tst1_lavf.avi lavf_copy_broken_audio_hdtv.ts
mencoder -oac mp3lame -demuxer lavf -ovc xvid -xvidencopts 'fixed_quant=4' -o tst2_lavf_lame.avi lavf_copy_broken_audio_hdtv.ts
mplayer  -dumpaudio                 -dumpfile tst3_mpegts.ac3 lavf_copy_broken_audio_hdtv.ts
mplayer  -dumpaudio   -demuxer lavf -dumpfile tst4_lavf.ac3 lavf_copy_broken_audio_hdtv.ts
mplayer  -dumpaudio -dumpfile tst5_mpegts_fromavi.ac3 tst0_mpegts.avi
mplayer  -dumpaudio -dumpfile tst6_lavf_fromavi.ac3 tst1_lavf.avi
mencoder -oac copy -ovc copy -o tst7_mpegts_copy.avi tst0_mpegts.avi
mencoder -oac copy -ovc copy -o tst8_lavf_copy.avi tst1_lavf.avi
mplayer  -dumpaudio -dumpfile tst9_lavf_fromcopyavi.ac3 tst7_lavf_copy.avi

The resulting file sizes:

 223242  tst0_mpegts.avi
 232748  tst1_lavf.avi
 221900  tst2_lavf_lame.avi
  26112  tst3_mpegts.ac3
  26112  tst4_lavf.ac3
  24746  tst5_mpegts_fromavi.ac3
  26112  tst6_lavf_fromavi.ac3
 223218  tst7_mpegts_copy.avi
 230184  tst8_lavf_copy.avi
  26112  tst9_lavf_fromcopyavi.ac3

tst0, tst2, tst7: play fine everywhere

tst1, tst8:
 - media player classic (guliverkli), M$ media player 9: no sound (no
   audio detected)
 - virtualdubmod crashes (6 Channel Stereo?!? 17 blocks audio only?!?
   inditerminable length, rate?!?)
 - if I embed one of the .ac3 files with virtualdubmod (disabling the
   already embedded audio) the files play fine (but I won't comment on
   A/V synchronity)
 - mplayer, xine: ok

The .ac3 files tst3, tst4, tst6, and tst9 actually exactly the same, not
only same size. Interestingly, tst5 is different, but it is one of the
files that work.

Both the analysis at the begining of this mail and the patch are new.

-- 
Matthias Hopf <mhopf at suse.de>       __        __   __
Maxfeldstr. 5 / 90409 Nuernberg    (_   | |  (_   |__         mat at mshopf.de
Phone +49-911-74053-715            __)  |_|  __)  |__  labs   www.mshopf.de
-------------- next part --------------
Index: libmpdemux/demux_lavf.c
===================================================================
--- libmpdemux/demux_lavf.c	(revision 22261)
+++ libmpdemux/demux_lavf.c	(working copy)
@@ -295,7 +295,7 @@
             }
             sh_audio->wf= wf;
             sh_audio->audio.dwSampleSize= codec->block_align;
-            if(codec->frame_size && codec->sample_rate){
+            if(0 && codec->frame_size && codec->sample_rate){
                 sh_audio->audio.dwScale=codec->frame_size;
                 sh_audio->audio.dwRate= codec->sample_rate;
             }else{
Index: libavformat/mpegts.c
===================================================================
--- libavformat/mpegts.c	(revision 8020)
+++ libavformat/mpegts.c	(working copy)
@@ -961,6 +961,12 @@
         st->priv_data = pes;
         st->codec->codec_type = codec_type;
         st->codec->codec_id = codec_id;
+	switch (codec_id) {
+	case CODEC_ID_AAC:
+	case CODEC_ID_AC3:
+	case CODEC_ID_DTS:
+	    st->codec->block_align = 1;
+	}
         st->need_parsing = 1;
         pes->st = st;
     }