[FFmpeg-user] Wanted: Fields-to-frames filter that does not add cosmetics

[Mark Filipak (ffmpeg)]

On 2021-03-05 21:31, pdr0 wrote:
> Mark Filipak (ffmpeg) wrote
>> On 2021-03-05 11:13, James Darnley wrote:
>>> On 05/03/2021, Mark Filipak (ffmpeg) wrote:
>>>> I seek a fields-to-frames filter that does not add cosmetics. In my
>>>> pursuit,
>>>> I look for such a
>>>> filter every time I peruse the filter docs for anything. I've yet to
>>>> find
>>>> such a filter.
>>>>
>>>> Do you know of a fields-to-frames filter that does not add cosmetics?
>>>
>>> separatefields - "splits each frame into its components fields,
>>> producing a new half height clip with twice the frame rate and twice
>>> the frame count"
>>
>> Yes, I could do (and have done) that, followed by 'shuffleframes=00',
>> followed by 'tinterlace' [1].
>> But that seems like a lengthy (slow) way to do what should be a simple
>> (faster) thing [2].
>>
>> [1] [A+b] ==> [A][b] ==> [A][A] ==> [A+A]
>> [2] [A+b] ==> [A+A]
>>
>>
>> If you're curious about what I'm doing, look:
>> [A+a][B+c][C+d][D+d][D+d]   ...SOURCE is a (consistent) mess [3]
>> [A+A][a+a][B+B][c+c][C+C][d+d][D+D][d+d][D+D][d+d]
>> ...yadif=mode=send_field
>> [A+A][a+a][B+B][B+B][C+C][c+c][D+D][d+d]   ...shuffleframes=0 1 2 2 4 3 6
>> 7 -1 -1
>> [A+a][B+B][C+c][D+d]   ...tinterlace=mode=interleave_bottom to make TARGET
>> [4]
>>
>> [3] Telecined (=30fps) ==> frame 1 discard (=24fps) ==> frame 3 repeat
>> (=30fps).
>> [4] The TARGET is beautiful. No cosmetic filtering needed (or possible).
>>
>>
>> [A+a][B+c][C+d][D+d][D+d]   ...SOURCE
>> [A][a][B][c][C][d][D][d][D][d]   ...separatefields
>> [A][a][B][B][C][c][D][d]   ...shuffleframes=0 1 2 2 4 3 6 7 -1 -1
>> [A+a][B+B][C+c][D+d]   ...weave=first_field=top to make TARGET
>>
>> Hmmm... That appears to work. I'll try it.
>>
>> I guess I got stuck on using tinterlace as the last step and couldn't see
>> that separatefields &
>> weave would be simpler (and faster) than yadif and without yadif's
>> cosmetics.
>>
>> Thanks!
> 
> 
> Yes, yadif is not the right filter for what you're doing, because of the
> spatial interpolation. Yadif is a deinterlacer, and as a general rule you
> don't deinterlace progressive content (that has matching field pairs), or
> you'll degrade it

I think it would be helpful if I clarified. I've got 2 balls in the air. 1, go with what I outline 
above (specifically [B+B]), or 2, allow [c] to mate with [B] (in addition to mating with [C]) and 
apply yadif to just the [B+c] frame, thusly:

[A+a][B+c][C+d][D+d][D+d]   ...SOURCE
[A][a][B][c][C][d][D][d][D][d]   ...separatefields
[A][a][B][c][C][c][D][d]   ...shuffleframes=0 1 2 3 4 3 6 7 -1 -1
[A+a][B+c][C+c][D+d]   ...yadif(frame n%4==1 only),weave=first_field=top

The video is an actor making a comment. Her name fades in-out, below and to the right.

The more I look at the bob of that text in the [B+B] frame (otherwise the bob is not noticeable), 
the more I'm leaning towards some sort of cosmetics even if it introduces a very, very slight judder 
(i.e. c][C+c] is effectively 1+1/2 frames).

In my first tries (a couple of days ago), I applied yadif to all frames and didn't like the results. 
I hate fuzzy and I hate judder even more, but now I'm considering both for what is effectively +1/2 
frame.

> Mark Filipak (ffmpeg) wrote
>> [4] The TARGET is beautiful. No cosmetic filtering needed (or possible).
> 
> Cosmetic filter is wanted and possible on B+B.
> 
> B is an orphaned field, missing it's partner "b" . B+B is going to be full
> of aliasing/stairstepping...

Since it's for only 1/24th second, aliasing & bob really are not noticeable (except for the 
aforementioned text). (BTW, the ffmpeg docs have only 3 references to the word 'alias'. All 3 are in 
'8.6 acrusher'. Are the devs avoiding the word 'alias' for a reason?)

>... The field interpolation algorithm used to
> generate the pseudo "b" makes a difference. For example, -vf nnedi=field=t
> applied selectively on that B+B frame will look substantially better, almost
> like a fake B+b . Or temporally filtered B+b (from A+a and C+c data), such
> as with QTGMC in vapoursynth or avisynth will look better than either. If
> you want demos or more info let me know

Thanks for the offer. You're very generous. I'll try hard to not burden you.

I experimented with 'nnedi' a couple of months ago and will retrieve my notes. And I do pipe to/from 
VapourSynth for other tasks (mostly to use InterFrame) so I'm somewhat familiar with it. I'll try 
both your suggestions (and I'll try to not be too stupid).

You've given me a good push. Thanks so much for the guidance!