[FFmpeg-devel] Politics

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Michael
> Niedermayer
> Sent: Sunday, December 19, 2021 4:16 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
> Subject: Re: [FFmpeg-devel] Politics
> 
> On Sat, Dec 18, 2021 at 08:41:09PM +0000, Soft Works wrote:
> >
> >
> > > -----Original Message-----
> > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Daniel
> > > Cantarín
> > > Sent: Saturday, December 18, 2021 9:05 PM
> > > To: ffmpeg-devel at ffmpeg.org
> > > Subject: Re: [FFmpeg-devel] Politics
> > >
> > >  >
> > >  > Then you have never seen anime translations where signage in the
> > >  > videos are translated. If the subtitles are off even by one frame in
> > >  > such a case, you will see it, especially when the translated sign is
> > >  > moving with the video, and one new subtitle event is generated for
> > >  > every video frame.
> > >  > You can't sync to audio when the element you are translating is in the
> > >  > video itself, and not audio.
> > >  >
> > >  > - Hendrik
> > >
> > > This is correct, thank you.
> > >    1. If you're translating visuals, you sync to video.
> > >    2. If such visual is moving, you may move the translation in sync.
> > >
> > > I've ignored those cases, and it's correct to remark them.
> > >
> > > Yet, I understand this is done with video editing UIs, not ffmpeg
> filters,
> > > specially as it requires to visually match XY coordinates.
> > >
> > > Also, subtitle communities I know of use timings, not frames, even
> > > when doing overlays: "overlay at X:Y:Z.000 time".
> > >
> > > And a single frame means absolutelly nothing, even in this use cases.
> > >
> > > So, overall, I fail to see the serious frame-perfect subtitle-video sync
> > > problem with the patchset.
> >
> > The more the focus is moving to "a single frame" doesn't matter,
> > the more will that conversation create the impression that my patchset
> > would be lacking precision.
> >
> > In fact we're just talking about a fantasy subject instead of an
> > existing problem.
> >
> > > But there's no need to so much debate: get us some such anime sub,
> > > I get the original video somehow, do some tests, and post the results
> > > here. I'm open to do the testing work, if others are willing to help me
> > > with input examples.
> >
> > I would also be interested in an example for this, even when it doesn't
> > prove any issues.
> >
> > I'd be really glad when somebody could provide a sample (even privately),
> > it could be something I haven't seen yet.
> 
> playing devils advocate here, meaning iam not sure if the example below makes
> sense as an argument in this debate but its a interresting case which was
> not mentioned
> 
> consider a subtitle track
> consider 2 video tracks for US 30000/1001 fps and EU 25 fps
> 
> the 6th frame in the US track is at  0.2002 sec, the 5th in the EU track at
> 0.2sec

Yes, and we urgently need to buy carrots, because tomorrow, a unicorn might 
come around the corner and then we wouldn't have anything to feed it.
Also, the whole MPEG code should be re-written because next year, there 
could be a breaking change to the specs, so we should go and rewrite the code 
to prepare for that possibility. Also we should all buy a red T-Shirt, because
aliens might arrive soon at our planet, and they will probably like red colors.
;-)

At the serious side: 

First of all, if such a unicorn video would exist, and you would want to consider
this as a realistic problem, then it would be a problem of its own and independent
of ffmpeg and there's nothing that ffmpeg could remedy.
It's the time resolution of all the subtitle format specifications which do not offer
such a fine granularity. And this won't change. Simply because it's not needed
and there wouldn't be the slightest advantage.

The idea that a high resolution for subtitle timings would be required for being
able to match video timings as closely as possible is a misconception.

In fact, you couldn't do more wrong than that. :-)
(I'm sure some will figure out why)


For the unicorn video, we'd first need to know how it was produced.

From a movie? Then the two tracks would have a different playback speed, and the 
two video couldn't use the same subtitle track anyway.

I don't think that a camera exists that has a framerate high enough so that 
you could produce videos at both framerates just from picking original frames.

In most other cases, one of the videos would have been converted from the other
one. There's high-priced studio and broadcast equipment for doing such conversions,
and it requires a mix of different strategies for this.
But these can't do magic either, and when we go back to the example above, 
we'd have to start by wondering what is happening with hard cuts when such
conversions are done.
Either a hard cut would turn into a slightly smoothed transition over 2 or 3 frames,
then subs wouldn't be affected (still hitting the smooth transition.

On the other hand, when you would do hard cuts at the nearest neighbour frames
in the second video, then that would turn it in do a _different_ video.
That different video would have cuts at different time positions. 
And that means  that a single subtitle stream cannot be used for 
both videos - no matter how high the precision of the subtitle timings 
would be - it would help anything.

Best wishes,
softworkz