[FFmpeg-user] minterpolate performance & alternative

[Mark Filipak (ffmpeg)]

Synopsis:

I seek to use minterpolate to take advantage of its superior output. I present some performance 
issues followed by an alternative filter_complex. So, this presentation necessarily addresses 2 
subjects.

Problem:

I'm currently transcoding a 2:43:05 1920x1080, 24FPS progressive video to 60FPS via minterpolate 
filter. Apparently, the transcode will take a little more than 3 days.

Hardware:

There are 4 CPU cores (with 2 threads, each) that run at 3.6 GHz. There is also an NVIDIA GTX 980M 
GPU having 1536 CUDA cores with a driver that implements the Optimus, CUDA-as-coprocessors architecture.

Performance:

During the transcode, ffmpeg is consuming only between 10% & 20% of the CPU. It appears to be 
single-threaded, and it appears to not be using Optimus at all.

Is there a way to coax minterpolate to expand its hardware usage?

Alternative filter_complex:

minterpolate converts 24FPS to 60FPS by interpolating every frame via motion vectors to produce a 60 
picture/second stream in a 60FPS transport. It does a truly amazing job, but without expanded 
hardware usage, it takes too long to do it.

A viable alternative is to 55 telecine the source (which simply duplicates the n%5!=2 frames) while 
interpolating solely the n%5==2 frames. That should take much less time and would produce a 24 
picture/second stream in a 60FPS transport -- totally acceptable.

The problem is that motion vector interpolation requires that minterpolate be 'split' out and run in 
parallel with the main path in the filter_complex so that the interpolated frames can be plucked out 
(n%5==2) and interleaved at the end of the filter_complex. That doesn't make much sense because it 
doesn't decrease processing (or processing time) and, if the fully motion-interpolated stream is 
produced anyway, then output it directly instead of interleaving. What's needed is an interpolation 
alternative to minterpolate.

Alternative Interpolation:

55 telecine with no interpolation or smoothing works well even though the n%5==2 frames are combed 
but decombing is desired. The problem with that is: I can't find a deinterlace filter that does 
pixel interpolation without reintroducing some telecine judder. The issue involves spacial alignment 
of the odd & even lines in the existing filters.

Some existing filters align the decombed lines with the input's top field, some align the decombed 
lines with the input's bottom field. What's desired is a filter that aligns the decombed lines with 
the spacial mean. I suggest that the Sobel might be appropriate for the decombing (or at least, that 
the Sobel can be employed to visualize what's desired).

Sobel of line y:   ______/\_____________/\_________ (edges)
Sobel of line y+1: __________/\_____________/\_____
Desired output:
          line y:   ________/\_____________/\_______ (aligned to mean)
          line y+1: ________/\_____________/\_______ (aligned to mean)
I could find this:
          line y:   ______/\_____________/\_________
          line y+1: ______/\_____________/\_________ (aligned to top line edges)
and I could find this:
          line y:   __________/\_____________/\_____ (aligned to bottom line edges)
          line y+1: __________/\_____________/\_____

Does such a decomb filter exist? I've tested every deinterlacing filter, and I've failed to find one.

Thank you for reading all this. I'm sorry it's so long.

Regards,
Mark.
-- 
Someone's sneaking in and turning up the range so that my food burns.
I'm sure of it.
And the older I get, the more sure of it I become.