[MPlayer-dev-eng] New inverse-telecine filter

[Zoltan Hidvegi]

> > For mencoder-g1 it is better than pullup, since pullup cannot keep the
> > framerate. 
> 
> I guess you mean force 24 fps output, and you're right. But it's still
> not better to output incorrect frames. I can upload some samples where
> your code will output incorrect frames if you'd like. :)

I do not think I am outputing incorrect frames, but if I do, I'd like
to see a sample, so I'd be very much interested to try those samples.
But I do not think I have download access from mplayerhq incoming.

> Anyway, I'll be happy to commit your filter to MPlayer, but the name
> pullup2 is very misleading and I won't commit with that name. Pullup
> is the engine I wrote independent of MPlayer to be very versatile and
> reusable in other programs, and "pullup2" is not version 2 of that,
> but a very different approach.

OK, I really do not care about the name, I just did not have any
better idea.

> output. The only thing I don't consider is nasty broadcast noise or
> watermarks (imo a form of broadcast noise :).

Yes, but unfortunately, I use this mostly for broadcast.  From your
timezone I assume that you are in Europe, so you do not have to deal
with broadcast telecine there.

> > It may be true that you need lookahead if you want to be absolutely
> > correct, but once you find a frame boundary, you can display the next
> > two fields as long as those two fields seem to match well. 
> 
> The question is: can you find the frame boundary? Lots of times it's
> harder than you'd expect. I imagine you're working with trivial
> content (live action) rather than difficult stuff like animation.

I've encoded hundreds of hours from various sources, including
animation.

> > And for watching live TV
> > it is an advantage that you have no delay, and it was one of my goals,
> > to have a filter good for live TV.
> 
> I agree this is useful. But you absolutely need at least one field
> delay or you _WILL_ mess up at scene change splices.

Sometimes I do delay, when the match does not look good, my filter
will delay.  But if you have a frame with two fileds matching
perfectly, and there is a big difference between the last field of the
previous frame and the first field of the current frame, you can show
the whole frame as is.  Also, if the two current fields match well
with the last field of the previous frame (a 3-field sequence), I can
also show the current frame without delay.  If there is a scene change
in the middle of the frame, it would show up as a bad match.  My field
compare is more accurate than licomb, so I can do better decisions
just by comparing adjacent fields.

> > > The pullup module was already designed to handle all the buffer
> > > management and field merging with minimal copy overhead. It's also
> > 
> > My pullup2 has zero copy for most frames, only the merged frames,
> > usually 1 out of 4 are copied.  The pullup filter can do direct
> > rendering, but not after a crop filter, which used EXPORT.  I'm
> 
> The solution is to set the boundary sizes for pullup to ignore and
> then perform the actual crop after pullup. There's no way to do this
> yet from the command line, but adding it is trivial.

Yes, that works too to speed up the metrics, but the merging is still
slower if you do not crop.

> > I'll try to understand pullup better, but some parts of it, especially
> > the buffer management, seemed to be more complicated than necessary,
> > maybe because it is really for G2?
> 
> Because it has to buffer fields ahead to make accurate decision. Also
> the API makes it possible to use field-at-a-time source instead of
> frame-based source, in case your capture device or codec gives fields
> instead of fake frames. And if you read the code a little more, I
> think you'll see that factoring out the 'complicated' buffering layer
> makes the actual decision-making code very concise and readable, and
> it's naturally independent of input field order without any hacks.

I believe you, I'll see if I can understand what's going on, but I do
not think I'll have time for this before the end of the year.  And for
sources wilth field-at-a-time, mplayer-g1 does not support that, but I
agree it's nice to have that for the future.

> > > > The filter works on
> > > > even-first (a.k.a. top-field first) frames, for a bottom-first frame the
> > > > first line is skipped to make it top-first.
> > > 
> > > This is rather awkward. Why not just invert stride and start at the
> > > bottom?
> > 
> > Sure, that works too, but I do not see how adding the stride once is
> > more awkward that reversing it?  You already skip the pixels close to
> > the edges, skipping one more line at the top does not make any
> > difference.  This is only for calculating the metrics.  I also skip on
> > the left to make sure I start at an 8-byte aligned address to speed up
> > MMX.
> 
> I'm not quite sure how your code works on mixed input where some input
> frames are TFF and others BFF (mixed hard+soft telecine)...

I have a flag in vf_priv_s called swapped (bad name, I know), which is
set it the current frame is BFF.  And if it is set, the metric routine
will reduce the frame hight by 1 and it will crop one line from the
top, then the frame becomes TFF.  And I assume that repeat-first-field
frames are never interlaced.  The HDTV broadcast is full of mixed
telecine, sometimes switching very rapidly between hard and soft TC,
like most os the stuff is telecined with mpeg flags, but a few frames
around every scene change is hard telecined, sometimes with bad edits
etc.  If you only do DVDs you probably rarely see such mpegs.

> > > > Using these statistics, the find_breaks routine tries to find frame
> > > > breaks, see the code for details.  This routine is probably not perfect
> > > > yet.
> > > 
> > > Again, you reuse terminology from the other filters to mean something
> > > radically different.
> > 
> > I'm not that familiar with mplayer, so I was really not aware that I
> > was confusing the terminology, please tell me what terminology I
> > should use.
> 
> Maybe I should type my handwritten design docs for pullup and commit
> to DOCS/tech... Basically, in my terminology, a break means a point
> where fields necessarily can't go together. Breaks arise from
> duplicate fields, so they only appear adjacent to duration-3 frames
> with motion relative to the previous/next frame.

Yes, such a document would be useful.

> Pullup also has a notion of field "affinity", whereby each field is
> said to "prefer" one or neither of its two potential partners
> according to the "licomb" metric.

I see.  My metric for the neighbouring fields (like yours, but yours
is 2x bigger because of the 16-line bug), is trying to measure the
distance between the neighbouring fields just like the diff metric
measures the distance between the even or odd fields.  They should be
comparable, so for example, the distance between the odd fields should
be close to noise+temp, where noise is the distance between the two
fields in the current frame, and temp is the distance between the
current first field and the previous last field.  And you can do
pretty good decisions just by using these `affinity' measures.  I can
handles 4-field long frames as well, like here sometimes they do
2-3-2-4 on TV to slow down video (I do not know why, 2-3-3-3 would be
much better of course, but those TV people do not care much about
quality).

Zoli