[FFmpeg-user] 1080i AVC-I 100 frame rate conversion

[Mark Filipak (ffmpeg)]

On 2021-03-29 07:11, Bruce Roberts wrote:
> Yes, my apologies - I don't get the luminance bands either when I don't try
> to deinterlace the progressive source or if I do deinterlace and
> reinterlace interlaced inputs so it seems there was no problem with the
> framerate filter after all.
> 
> I would still be very interested to hear any optimisation tips and I look
> forward to seeing the results of Mark's work.

Hey Bruce,

The 1-pass, fractional picture interpolation I'm trying to develop depends on the 'shuffleframes' & 
'mix' filters in order to do a weighted 119.880fps rendition of 23.976fps. For example,

given the input (processed 2 frames at a time):
[a_____________________________________][b_____________________________________]

and 'fps' boosting to 119.880fps and 'split'ting that into 2 substreams:
[1a5___][1b2___][1b4___][1b6___][1b8___][1b5___][1a2___][1a4___][1a6___][1a8___]
[2a5___][2a8___][2a6___][2a4___][2a2___][2b5___][2b8___][2b6___][2b4___][2b2___]

and creating multiplex shuffleframes (at 11.988fps):
[1A5___________________________________________________________________________]
_______][1B2____________________________________________________________________
_______________][1B4____________________________________________________________
_______________________][1B6____________________________________________________
_______________________________][1B8____________________________________________
_______________________________________][1B5____________________________________
_______________________________________________][1A2____________________________
_______________________________________________________][1A4____________________
_______________________________________________________________][1A6____________
_______________________________________________________________________][1A8____
[2A5___________________________________________________________________________]
_______][2A8____________________________________________________________________
_______________][2A6____________________________________________________________
_______________________][2A4____________________________________________________
_______________________________][2A2____________________________________________
_______________________________________][2B5____________________________________
_______________________________________________][2B8____________________________
_______________________________________________________][2B6____________________
_______________________________________________________________][2B4____________
_______________________________________________________________________][2B2____
(note: to here, the 'a's & 'A's are identical, all the 'b's & 'B's are identical.)

then summing them with the weights (a5+a5)/10, (a8+b2)/10, (a6+b4)/10, etc.
[A5A5__________________________________________________________________________]
_______][B2A8___________________________________________________________________
_______________][B4A6___________________________________________________________
_______________________][B6A4___________________________________________________
_______________________________][B8A2___________________________________________
_______________________________________][B5B5___________________________________
_______________________________________________][A2B8___________________________
_______________________________________________________][A4B6___________________
_______________________________________________________________][A6B4___________
_______________________________________________________________________][A8B2___
(note: now, they are different, they are weighted sums.)

and interleaving to produce the 119.880fps output:
[A5A5__][B2A8__][B4A6__][B6A4__][B8A2__][B5B5__][A2B8__][A4B6__][A6B4__][A8B2__]

Here is the filter_complex:

settb=720000,setpts=N*30030,fps=fps=24000/1001,
fps=fps=120000/1001,split=2[1][2],
   [1]shuffleframes=0 5 6 7 8 9 1 2 3 4,split=10[1a5][1b2][1b4][1b6][1b8][1b5][1a2][1a4][1a6][1a8],
     [1a5]shuffleframes=0 -1 -1 -1 -1 -1 -1 -1 -1 -1[1A5],
     [1b2]shuffleframes=-1 1 -1 -1 -1 -1 -1 -1 -1 -1[1B2],
     [1b4]shuffleframes=-1 -1 2 -1 -1 -1 -1 -1 -1 -1[1B4],
     [1b6]shuffleframes=-1 -1 -1 3 -1 -1 -1 -1 -1 -1[1B6],
     [1b8]shuffleframes=-1 -1 -1 -1 4 -1 -1 -1 -1 -1[1B8],
     [1b5]shuffleframes=-1 -1 -1 -1 -1 5 -1 -1 -1 -1[1B5],
     [1a2]shuffleframes=-1 -1 -1 -1 -1 -1 6 -1 -1 -1[1A2],
     [1a4]shuffleframes=-1 -1 -1 -1 -1 -1 -1 7 -1 -1[1A4],
     [1a6]shuffleframes=-1 -1 -1 -1 -1 -1 -1 -1 8 -1[1A6],
     [1a8]shuffleframes=-1 -1 -1 -1 -1 -1 -1 -1 -1 9[1A8],
   [2]split=10[2a5][2a8][2a6][2a4][2a2][2b5][2b8][2b6][2b4][2b2],
     [2a5]shuffleframes=0 -1 -1 -1 -1 -1 -1 -1 -1 -1[2A5],
     [2a8]shuffleframes=-1 1 -1 -1 -1 -1 -1 -1 -1 -1[2A8],
     [2a6]shuffleframes=-1 -1 2 -1 -1 -1 -1 -1 -1 -1[2A6],
     [2a4]shuffleframes=-1 -1 -1 3 -1 -1 -1 -1 -1 -1[2A4],
     [2a2]shuffleframes=-1 -1 -1 -1 4 -1 -1 -1 -1 -1[2A2],
     [2b5]shuffleframes=-1 -1 -1 -1 -1 5 -1 -1 -1 -1[2B5],
     [2b8]shuffleframes=-1 -1 -1 -1 -1 -1 6 -1 -1 -1[2B8],
     [2b6]shuffleframes=-1 -1 -1 -1 -1 -1 -1 7 -1 -1[2B6],
     [2b4]shuffleframes=-1 -1 -1 -1 -1 -1 -1 -1 8 -1[2B4],
     [2b2]shuffleframes=-1 -1 -1 -1 -1 -1 -1 -1 -1 9[2B2],
   [1A5][2A5]mix=inputs=2:weights=5 5[A5A5],
   [1B2][2A8]mix=inputs=2:weights=2 8[B2A8],
   [1B4][2A6]mix=inputs=2:weights=4 6[B4A6],
   [1B6][2A4]mix=inputs=2:weights=6 4[B6A4],
   [1B8][2A2]mix=inputs=2:weights=8 2[B8A2],
   [1B5][2B5]mix=inputs=2:weights=5 5[B5B5],
   [1A2][2B8]mix=inputs=2:weights=2 8[A2B8],
   [1A4][2B6]mix=inputs=2:weights=4 6[A4B6],
   [1A6][2B4]mix=inputs=2:weights=6 4[A6B4],
   [1A8][2B2]mix=inputs=2:weights=8 2[A8B2],
[A5A5][B2A8][B4A6][B6A4][B8A2][B5B5][A2B8][A4B6][A6B4][A8B2]interleave=nb_inputs=10

This simple scheme depends on shuffleframes behaving (or at least behaving predictably). That's 
where the problem lies. I'm still trying to figure out why shuffleframes acts unexpectedly. It seems 
to have 3 interacting modes of unexpected behavior that are factors of whether the input #frames is 
odd or even, whether the input #frames can be spanned by a whole number of map elements (for 
example, the shuffleframes above have 10 map elements), and how it handles cases where the ending of 
the input results in fractional map-spans. That doesn't completely account for the behavior, so I 
reckon there's another unexpected mode that needs to be discovered.

The scheme is working except that some of the output frames in the 10-frame sequence above move 
backward for a few frames before leaping back to the forward cadence it should have. That of course 
indicates that the presentation time stamps (PTS) of the backward frames are wrong, but I haven't 
been able to figure out why. That's why I'm testing shuffleframes. The task is rather mind wracking.