[MPlayer-dev-eng] Documenting and extending the subtitles system

[Uoti Urpala]

On Tue, 2009-02-24 at 00:36 +0100, Nicolas George wrote:
[replies to Reimar's comments]
> > >   - Overlaid stuff is pulled from libass, and may be send through, and will
> > >     sometime travel along VFCTRL_DRAW_EOSD.
> > No, VOCTRL_DRAW_EOSD will contain the data, VFCTRL_DRAW_EOSD is only a
> > notification that it is now time to draw the OSD.
> 
> I did not see the difference between V[OF]CTRL, my bad. But, for my defence,
> there are inconsistencies:
> 
> - vf_ass always does the overlay in put_image and ignores V?CTRL_DRAW_EOSD;
> - vo_gl (the only VO implementing EOSD) does the overlay on VOCTRL_DRAW_EOSD;
> - vf_vo emits VFCTRL_DRAW_EOSD whenever it gets VFCTRL_DRAW_EOSD;

emits VOCTRL_DRAW_EOSD, not VF

> - dec_video.c emits VFCTRL_DRAW_EOSD just after put_image.
> 
> It seems to me like two subtle ways to do exactly the same thing, and there
> certainly is room to make things simpler.

I think the main function of sending VFCTRL_DRAW_EOSD though the filter
chain and converting in vf_vo is to allow a filter doing the rendering
to block the request from going further, so that the vo will not try to
duplicate the rendering.


[replies to my comments]
> > It's not clear whether you're considering only the render-on-video part
> > or also how OSD content state is maintained and updated. Those parts are
> > separate; the code doing the rendering need not have any knowledge about
> > how OSD internally maintains its list of objects. Your comments about
> > the current mechanisms generally fail to distinguish the API for
> > updating OSD state and the API for renderers, which are fairly
> > independent.
> 
> I completely agree that these two points should be distinguished. But in the
> current state of the code, they are not, which is a problem.

There is a clear separation even now. The VOs do not need to know how
the OSD is updated. "State of the OSD" is not abstracted away from
"things that want to change OSD", but the more important separation of
"OSD state and its changes" vs "rendering the OSD" is there.

> > This stuff is all related to maintaining OSD state and is not visible to
> > the part doing the rendering.
> 
> Alas, it is: there actually several code paths that lead to the rendering
> function, depending on the origin of the OSD objects.

This is in no way visible to individual renderers. They get calls to the
draw_alpha function they provided, and there is zero distinction between
different kinds of OSD objects.

Maybe you misinterpret where the API boundaries are? What is in the vo_
files and in vf_expand.c is the "renderer side". Compare it with EOSD;
instead of getting a list of bitmaps they get a series of callbacks with
varying parameters. The code doing those callbacks is no more part of
the "renderer side" than the code creating the list of bitmaps is part
of the EOSD "renderer side".

> > > - a callback API, for anyone to get notification of significant events (such
> > >   as timestamps expiration);
> > This isn't so obvious. Currently there is no such API and I see no clear
> > need for one. Subtitle events are probably best handled by the subtitle
> > "codec". OSD objects may have expiration times, but it doesn't seem
> > necessary to have those "in the OSD API".
> 
> I think you are mistaken: there is currently such an API, and it is used:
> the ass and vo filters get their EOSD objects by calling
> ass_mp_render_frame, and they give the current PTS as a parameter; libass
> uses that PTS to compute which subtitles are active.

What does that have to do with callbacks? You separately listed an
"input API", and this functionality is much better placed under that.
Code which decides the contents of the OSD can of course use whatever
information it wants to select those contents, including the timestamp
of the corresponding frame. Why would that be a reason to list this
separately from the "input API"?

> Here is the API of the client side (the function names are not definitive):

I think you're trying to do too much at once. Pick some more concrete
goal than "rewriting the API" which has no clear definition of success.
I think you've still misunderstood too many things to _start_ from
trying to write a complete definition of the overall API; improvements
to that would be better done when limitations hinder some more concrete
goal.