[MEncoder-users] regarding skipped/duplicate frames during framerate conversion

[Trent Piepho]

On Tue, 5 Dec 2006, rob pfile wrote:
> but even with this filter, i am seeing a repeated pattern of
> duplicate and skipped frames, with a skip following just 2 or 3
> frames after mencoder decided it had to duplicate one. i looked into
> this a little bit and found that the culprit is the code that's
> looking at v_timer_corr. for whatever reason, v_timer_corr goes


I'm seeing this problem too.  I enhanced my filter to create a 2-pass mode
that tries to keep the best frame.  For example, the 1-pass mode where D =
drop and K = keep:

KDD KD KDD KD KDD KD

In 2-pass mode the filter might do this instead for the same frames:

DKD KD DDK KD KDD DK

In each case one frame out of each 2/3 frame group is kept.  Sometimes the
first frame in the group isn't a good as the later frames, so it's kept and
the first frame is dropped.

The later method is giving mencoder problems.  It drops a frame, then later
decides to duplicate one.  Or the other way around, it's hard to say which
is the mistake and which is the correction.  This is an example of the
pattern I see of v_timer_corr, in units of input frame times (~ 1/60th
second).

v_timer_corr value / mencoder's action, 60->24 filter action
-1	drop
-2	drop
-3	keep, mencoder insert dup
+1	keep
+2.5	drop, mencoder ask for skip
+1.5	keep (skipped because of mencoder)
+0.5	drop
-0.5

Each time an input frame is dropped the audio advances by 1 frame and the
video stays the same, so v_timer_corr -=1.

When a frame is output, the audio advances by 1 frame time and the video by
2.5 (since one 24fps frame == 2.5 60fps frames).  The net change is
v_timer_corr += 1.5.

When mencoder sees that v_timer_corr is <= -2.5 it will duplicate a frame,
and when it sees v_timer_corr >= 2.5 will request a frame skip.  It's worth
nothing that frame skip is saved until the next _output_ frame.  For
example, in the fifth input frame v_timer_corr is +2.5 and mencoder asks
for a skip.  But the inverse telecine filter drops that frame and so the
skip is saved up.  The sixth frame is kept by the filter, but gets dropped
anyway because of the skip mencoder requested for the previous frame.

It seems like the problem is that mencoder doesn't cope with a filter chain
that also adjusts v_timer_corr.  mencoder looks at v_timer_corr and decides
it needs to make an adjustment (adding or skipping a frame) and then calls
the decode/filter/encode layer to process the next input frame.  This also
adjusts v_timer_corr but mencoder didn't take that into account when it
made its dup/skip decision.  You end up with a double adjustment that moves
v_timer_corr too far.

By making mencoder wait for v_timer_corr to be off by 1.5 frames I was able
to stop the dup/skip cycle.  It gives the inverse telecine filter another
frame to bring thigns back into sync before causes a skip/dup.

This doesn't give the best results I think.  For example, say v_timer_corr
gets to 1 and then just stays there.  It never gets to 1.5 and so no frames
are skipped.  It would obviously be better if one frame were skipped, so it
would be saying at 0 instead of 1.

It's not clear to me what the difference between v_timer_corr and
v_pts_corr is.  Why does v_pts_corr not cause a correction until it's is
greater than two frame times, while v_timer_corr only needs one frame time?
Would it make any sense to add v_timer_corr+v_pts_corr and make a decision
based on that?