[MEncoder-users] regarding skipped/duplicate frames during framerate conversion

[pfile at pacbell.net]

On Thu Dec 7 06:12:13 CET 2006, Trent Piepho wrote:

> On Tue, 5 Dec 2006, rob pfile wrote:
> > but even with this filter, i am seeing a repeated pattern of
> > duplicate and skipped frames, with a skip following just 2 or 3
> > frames after mencoder decided it had to duplicate one. i looked into
> > this a little bit and found that the culprit is the code that's
> > looking at v_timer_corr. for whatever reason, v_timer_corr goes
>
>
> I'm seeing this problem too.  I enhanced my filter to create a 2- 
> pass mode
> that tries to keep the best frame.  For example, the 1-pass mode  
> where D =
> drop and K = keep:
>
> [description of filter and behavior of v_timer_corr]

although i didnt express it in frame times, i was going to post the  
exact same sequence to the list last week. so we really are seeing  
the same thing. filter wise, we have probably converged on equivalent  
code.

> It seems like the problem is that mencoder doesn't cope with a  
> filter chain
> that also adjusts v_timer_corr.  mencoder looks at v_timer_corr and  
> decides
> it needs to make an adjustment (adding or skipping a frame) and  
> then calls
> the decode/filter/encode layer to process the next input frame.   
> This also
> adjusts v_timer_corr but mencoder didn't take that into account  
> when it
> made its dup/skip decision.  You end up with a double adjustment  
> that moves
> v_timer_corr too far.
>

is that really true though? if it decided to skip a frame, it  
increments skip_count and adjusts v_timer_corr. then it calls  
decode_video() and if that returns 0 (dropped frame), then it adjusts  
v_timer_corr again, but only if skip_count is <= 0. so it seems like  
you dont actually get a double correction to v_timer_corr, but you do  
get a skipped frame next time. so maybe the right thing to do is to  
decrement skip_count if it is > 0 in the case that decode_video()  
returned 0.

> By making mencoder wait for v_timer_corr to be off by 1.5 frames I  
> was able
> to stop the dup/skip cycle.  It gives the inverse telecine filter  
> another
> frame to bring thigns back into sync before causes a skip/dup.
>

i made a similar hack, and this seemed to work okay. however,  
probably due to the pts hack described below, at edit points i was  
getting a whole burst of pts duplicate frames. in the end i've  
resorted to encoding the audio without the video and using -noskip,  
and then merging the audio back in to avoid all this nonsense. but  
this method seems to have a big drawback: some kinds of dropouts in  
the original .ts file cause mencoder to do something funny to the  
video such that when its remultiplexed, mplayer says the file is  
badly interleaved and stops playing the audio. clearly i'd prefer to  
just have mencoder handle the audio as well, which is why i'm still  
messing around with this stuff.

> This doesn't give the best results I think.  For example, say  
> v_timer_corr
> gets to 1 and then just stays there.  It never gets to 1.5 and so  
> no frames
> are skipped.  It would obviously be better if one frame were  
> skipped, so it
> would be saying at 0 instead of 1.
>

yeah, along the way i hit this too. my "solution" was to decrease the  
pts_corr threshold to 1.2x the display time from 2x, which causes the  
pts code to hone in more closely on the a/v sync and skip/dup that  
extra frame that v_timer_corr didnt. but that may have screwed up the  
video around edits.

> It's not clear to me what the difference between v_timer_corr and
> v_pts_corr is.  Why does v_pts_corr not cause a correction until  
> it's is
> greater than two frame times, while v_timer_corr only needs one  
> frame time?
> Would it make any sense to add v_timer_corr+v_pts_corr and make a  
> decision
> based on that?

no idea on the rationale for how these two counters are treated.

my very limited understanding of this is that v_timer_corr is there  
to enforce the video framerate, and v_pts_corr is there to maintain  
proper a/v sync. however, i dont understand at all how v_timer_corr  
is really supposed to work. its computed from both the input frame  
display time and the output frame display time; how is the input  
frame display time relevant when you are changing the framerate?

anyway, around line 1406 of mencoder.c, it does look like the code  
that computes v_pts_corr does actually take the sum of v_timer_corr  
and v_pts_corr.

do we have any idea who the author of mencoder.c is, or at least who  
hacked on the skip/dup code last? i havent looked at the CVS logs  
(can we even anymore?) or the SVN logs. at this point it would be  
very helpful if that person had comments.

rob