[FFmpeg-devel] [RFC] Yet another subtitles open question

[Josh Allmann]

On 2 October 2013 13:29, Clément Bœsch <u at pkh.me> wrote:
> On Wed, Oct 02, 2013 at 12:10:38PM -0700, Josh Allmann wrote:
>> On 1 October 2013 11:14, Clément Bœsch <u at pkh.me> wrote:
>> > Hi folks,
>> >
>> > I have definitely better to do, and I still hate subtitles, but we need to
>> > make some changes, and I'm willing to help.
>> >
>> > Shell
>> > -----
>> >
>> > So here it is, I'd like to move on with the API. Currently we use
>> > AVSubtitle (and its AVSubtitleRects) which is defined in lavc. We wanted
>> > at some point to move it to libavutil, eventually with slight changes.
>> > OTOH, the idea of using the AVFrame was disregarded quickly for various
>> > reasons. The thing is, it might have various benefits:
>> >
>> >  - since it doesn't introduce a new struct and its own semantic, it will
>> >    ease a lot its integration with for instance libavfilter, and probably
>> >    allow various code refactoring
>> >
>> >  - AVFrames have a ref counting system: if audio and video are ref
>> >    counted, we expect the subtitles to be as well, otherwise it can become
>> >    a source of pain for users
>> >
>> >  - using AVFrame also suggest the idea that we could embed subtitles data
>> >    within existing AVFrame: think closed captioning. "Freelance" subtitles
>> >    OTOH would just use a empty/zero frame shell. Note that this conflicts
>> >    with the ref counting idea since it can't share the data buffers.
>> >
>> > Opinion on this?
>> >
>>
>> In Libav we're thinking of introducing an opaque field into AVFrame
>> for other purposes, and I'm thinking of leveraging it for
>> subtitle-related things there.
>
> Can you detail a bit more? I'm not sure I follow you with the opaque
> field. It sounds like it will require the AVSubtitle to be moved to
> libavutil anyway.
>

More or less, yes. There is concern about adding extra single-purpose
fields to AVFrame that are only used for subtitles, when the API user
could just as easily access an AVSubtitle (or similar). So this way,
we maintain the API/external semantics of AVFrame while keeping the
internal organization of subtitles distinct from AVFrame. The opaque
field could be re-purposed for other types of streams as well.

This idea is just preliminary, we need to consider more how that would
interact with ref counting, etc.

Josh