[FFmpeg-devel] Audio Video Sync issue with dvr-ms files

[Michael Niedermayer]

Hi

On Tue, May 01, 2007 at 02:47:31PM -0400, Jonathon Elwood wrote:
> On 4/3/07, Jonathon Elwood <jaelwood at gmail.com> wrote:
> >
> >
> >There is an AV sync issue with the latest SVN (8618).   I've uploaded a
> >sample file that demonstrates this issue to:
> >
> >ftp://upload.mplayerhq.hu/MPlayer/incoming/Backpackers -
> >_23_03_2007_23_00_03_comms.dvr-ms
> >
> >Michael Niedermayer recommended that I test with previous versions of 
> >ffmpeg
> >and find the patch that broke the AV sync.   I traced the problem to patch
> >#7905.  ffmpeg 7904 worked fine, but 7905 and above demonstrate this AV 
> >sync
> >issue.   I modified the latest SVN to reverse the changes made in patch
> >7905, and the AV sync looks good.  However, this obviously isn't a proper
> >fix.   Also without the changes in patch 7905 I have other dvr-ms shows
> >where the fps is detected incorrectly, so this patch is obviously needed.
> >Could you please look at this?   I'm not sure what the code in patch 7905 
> >is
> >doing, so I won't be able to create the appropriate patch.
> >
> 
> I have been working with John Donaghy trying to solve the problems I
> reported above (thanks John for all your help).  I've made some
> progress fixing the timestamps (the video isn't "freezing" and the fps
> is detected correctly on all shows now); however, the video frames
> before the first keyframe are being discarded.  This is leading to a
> bad AV sync issue because the audio starts from the beginning of the
> stream, but the video starts from the first Key Frame.
> I've attached two patches (one to asf.c and the other to asf.h)
> showing the changes I've made so far.  These patches obviously aren't
> ready for submission so please be kind, this is obviously still a work
> in progress.  I've also attached a debug log file showing the output
> of my debug statements.
> My questions are:
> 1) If you look at the debug.txt file you will see the timestamps I'm
> now allocating to each segment.  However, I'm not sure what to do with
> the video segments before the first Key Frame.  My guess is I need to
> do something special to these timestamps so the frames aren't
> discarded, but what?  Right now I'm just setting them to
> AV_NOPTS_VALUE, 

AV_NOPTS_VALUE is correct if you dont know their timestamp


> however, they are being discarded.  I tried setting a
> value on the new segments, however, there was a big jump in the
> timestamp values once I got to the first Key Frame and they were
> discarded anyway.  So, I need to know what to do with these sequences.
> John suggested I just discard both the audio and video sequences
> until I get to the Video Key Frame.  I don't know how to do this, is
> there a mechanism in place to make this easy?

a demuxer should not discard frames at random, if this causes a problem
then the bug is not in the demuxer


> 2) If you look at the segments marked "audio timestamp" you will
> notice that the audio is split into two segments.  I'm assigning the
> timestamp to the first segment, and setting the second segment to
> AV_NOPTS_VALUE.  Is this correct, or should the second segment also
> have a timestamp?  If the second segment has to have a timestamp, then
> what timestamp do I use (if I read the timestamp from the file, it is
> the same timestamp as the first segment)?

i dont know that, but i dont think setting the timestamp for both is
correct, either first or second ... but iam just guessing, it depends on
what dvr-ms really does ...


> 3) Should both audio segments have the Key Frame flag set, or should I
> remove the Key Frame flag from the second segment?
> 
> If there is anything else you see wrong regarding the timestamps I've
> allocated please let me know.  I've learned a lot, but all of this is
> still new to me.

yes i will NOT accept a patch with some odd interpolation code without
fully understanding why that is needed
that is if your only explanation is like "it doesnt work without" then
the patch is rejected

also all float/double calculations must be removed they are unacceptable
due to breaking binary reproduceable output between different architectures

the debug output you provide below is fairly hard to read its not obvious
from it what timestamps where in the file, what would be wrong with them or
what better timestamps you set

also without having looked at the files, did you try to simply set the
frame rate correctly if it was wrong? the internal timestamp code uses
the frame rate and will not work if its wrong ...

[...]

> +            audio_video_type = get_byte(pb); //This value appears to identify the type of audio and video.

can this be used to identify ac3/mp3 properly?

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

There will always be a question for which you do not know the correct awnser.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20070501/a1d74ea9/attachment.pgp>