[FFmpeg-devel] [PATCH] WIP: subtitles in AVFrame

[Clément Bœsch]

(sorry for initially replying in private by mistake)

On Thu, Nov 03, 2016 at 07:36:29PM +0100, Nicolas George wrote:
> Le duodi 12 brumaire, an CCXXV, Clement Boesch a écrit :
> > ---
> > So this is just a prototype that is starting to work. It needs a split
> > (and completion of the long TODO list) before review but I wanted to
> > share it for early feedbacks.
> >
> > So long story short: AVSubtitle → AVFrame, you have to use the new M:N
> > API, and it's integrated into lavfi.
>
> Yay!
>
> > Now a bit longer: the AVFrame->extended_data are
> > AVFrameSubtitleRectangle (declared in lavu/frame). Frames are
> > refcounted. So far this is not really meaningful as a frame ref will
> > alloc and copy every rectangle but that's a limitation we will have to
> > deal with for now but it allows proper integration into the framework.
>
> Can you briefly explain how you handled the duration and sparseness?
>

I didn't. Duration is somehow broken currently for some reason. I did
nothing for sparseness: the reason I added basic support in lavfi is
because it was much simpler to handle at ffmpeg.c level, so it's currently
just a passthrough mechanism.

> Also, I see you kept the text field a simple string with ASS syntax. Is
> it final?
>

I did't like having multiple fields for text based data. If we want to
decode in another form, we can still add an option to print out verbatim
text instead of ASS markup.

> > It actually simplifies a lot the workflow so I'm optimistic about the
> > end result. Unfortunately, there is a lot of work remaining.
> >
> > End note: despite my obsession with subtitles, I'd like to remind
> > everyone that I actually hate working with them, I just believe it's
> > really important to get that historical burden out of sight. That
> > means,
> > if anyone wants to help with that, they are extremelly welcome.
> >
> > Thanks,
> >
> > TODO:
> > - what to do about sub2video?
>
> The plan was always to remove it as soon as a proper solution in
> libavfilter was implemented and does the same job. Preferably with a
> short transitional period for the users, but not if it is too
> inconvenient.
>

Yeah I guess I'll need to write a filter to blend in the final patchset
submission.

> > - properly deal with -ss and -t (need strim filter?)
> > - sub_start_display/sub_end_display needs to be honored
> > - find a test case for dvbsub as it's likely broken (ffmpeg.c hack is
> >   removed and should be replaced by a EAGAIN logic in lavc/utils.c)
> > - make it pass FATE:
> >   * fix cc/subcc
> >   * broke various other stuff
> > - split commit
> > - Changelog/APIchanges
> > - proper API doxy
> > - update lavfi/subtitles?
>
> Regards,
>
> --
>   Nicolas George

-- 
Clément B.