[MPlayer-dev-eng] New inverse-telecine filter

[D Richard Felker III]

On Tue, Dec 02, 2003 at 10:10:08AM -0600, Zoltan Hidvegi wrote:
> This is a new inverse telecine filter, similar to detc, ivtc and pullup,
> but better.  Here is a summary how it works.

The name is very misleading. It implies the filter is better than
pullup, which is certainly not true, since it's impossible to do
correct inverse telecine without buffering fields ahead. At the very
least, you need one field into the future. And there's a reason pullup
buffers 6 fields...

The pullup module was already designed to handle all the buffer
management and field merging with minimal copy overhead. It's also
meant to allow plugging in new decision algorithms, although this code
isn't in place yet. If you think you have better algorithms for
choosing how to put frames together, I would suggest making them
switchable in pullup, rather than writing an entirely new filter. We
already have too many separate filters for playing around with ivtc
algorithms...

> The filter tries to detect frame boundaries by comparing fields.  Only
> the luma plane is used for this comparison, since the chroma field is
> usually too noisy and has lower resolution. 

Definitely a good choice.

> The filter works on
> even-first (a.k.a. top-field first) frames, for a bottom-first frame the
> first line is skipped to make it top-first.

This is rather awkward. Why not just invert stride and start at the
bottom?

> Similar to the other filters, first a metric is calculated for each 8x8
> pixel block not adjacent to the edge of the image.  Just like in the
> other filters, the four components of the metrics are even, odd, noise,
> temp. 

Only the bad ones (detc and ivtc) have noise/temp. It's actually not a
very good metric for detecting combing since it will give the exact
opposite result on certain cases (one-pel-per-frame vertical scroll).

> Even and odd are the SAD (sum of absolute difference) between the
> current field, and the same field in the previous frame.  Noise is the
> difference between the two fields within the same frame.  It measures the
> distance of the 4 odd lines in the 8x8 block from the even lines.  First
> the even lines are used to approximate the image half a pixel below the
> even lines, and the odd lines are used to approximate the image half a
> pixel above the off lines, and for each pixel the difference between
> these two are calculated. 

OK, this is not the same as the noise/temp metric, but the "licomb"
metric in pullup. This seems to be a good one. Confusing to mix names
though.

> Then the difference for half a pixel below the
> odd lines is calculated, and the smaller of these two are kept for each
> pixel.  This process eliminates false noise from sharp horizontal edges.

Interesting approach.

  ...
> The interlaced_low member counts the number of blocks where the noise and
> temp metrics are close to each other (neither is 1.5 times bigger than the
> other), and both are above the threshold.  The interlaced_high count is
> similar, but the metrics must be above twice the threshold.
> 
> To allow correct de-telecine of pulldown TV content with interlaced
> overlay, the interlaced blocks (those counted in interlaced_low) are not
> counted in the other statistics.

Something like this is probably a good approach. I didn't write
anything like this because I'm interested only in maximal accuracy for
DVD ripping, not TV captures polluted by logos.

> Using these statistics, the find_breaks routine tries to find frame
> breaks, see the code for details.  This routine is probably not perfect
> yet.

Again, you reuse terminology from the other filters to mean something
radically different.

> The above methods can detect frame boundaries quite reliably, but they
> fail on still motionless frames.  When there is no clear motion, the
> filter falls back using the telecine pattern, or the ratio of
> input/output frames to mark the frames.  put_image will use the breaks
> detected by find_breaks to output the frames, however, it will also
> keep a strict input/output frame ratio, dropping or duplicating frames
> as needed. 

If you'd like to add this functionality to vf_pullup.c (optionally), I
think many users would be quite appreciative...

> If frame timestamps were available, the filter could be
> easily modified to use those instead of frame counting.  The

Yes. My pullup is designed for G2, where it will not have to do any
framerate management at all. Instead a separate filter could regulate
the output fps if desired, but preferably one would encode to
variable-fps container formats like nut.

  ...
> It has built-in crop support.  This is necessary for two reasons: the
> external crop filter does not pass on the field flags, thus cannot be
> used for soft-telecined content.  The external crop filter also messes up
> the chroma interlacing on YV12 content when cropping non-multiple of 4
> lines, which the built-in crop allows any even cropping.  The use of

If you're sure it's done correctly, this is good.

Rich