[MPlayer-dev-eng] alpha compositing over video with gpu acceleration

[Nicolas George]

Le quintidi 15 fructidor, an CCXVIII, Chris Morgan a écrit :
> I'm working on a proof of concept for showing an alpha blended osd
> that could be modified at runtime on top of hd video playback. The
> target is an intel atom system, possibly the d510.
> 
> I've considered using a window manager and putting the osd window
> above the video window, this only seems to provide a single alpha
> value for the osd window, so to make a semi-transparent background I'd
> also end up with semi-transparent text.
> 
> I've done a bit of reading and came across vf_overlay. I'm aware that
> it hasn't made it into mplayer proper yet and wanted to find out what
> possible paths are possible here.
> 
> Does using vf_overlay preclude acceleration with vaapi/vadpu or simply
> a performance hit?
> 
> I've heard mention of an alternate path for vf_overlay development in
> terms of a different design and location in mplayer. Is this work
> beyond the design phase?

As the one who was working on it recently, I feel I must answer some of your
points.

As you probably know, hardware acceleration of video decoding nowadays is
most of the time not done by a dedicated device but by the video controller.
That implies that the decoded video is ready to go directly to the screen,
it does not need to go back to the central memory. There is probably a way
to get it back, just as there is a way to do a screenshot, but that is not
the expected procedure, and as such not very optimized and this inefficient.

If the decoded video does not transit in central memory, there is no way to
change it using a filter. In MPlayer words, most video filters just can not
work with hardware decoders.

Fortunately, there are ways to change the video inside the video controller.

Most modern video acceleration systems fit inside the 3D system of the video
controller, the video being some kind of glorified texture you could map on
the walls of your dungeon if you were writing a first person shooter instead
of a movie player. And just as it is possible to add monsters and explosions
in your dungeon, it is possible to add text and logos on top of the movie.

Using these features was probably what your compositing window manager was
doing in your experiment. If you managed to overlay a window with a single
alpha value, then you probably could have done the same with two window,
each with its own alpha value, thus solving part of your problem.

As for what could be done inside of MPlayer: MPlayer has two systems to add
things on top of the video, each with its own limitations and more or less
partial implementation in the accelerated video drivers.

OSD is the oldest one. Its two major problems are that there is a lot of
needlessly duplicated code, and that each new kind of displayed object has
been added directly in the code with special cases, making it that much
harder for the next kind of display.

EOSD is the most recent. It has been added for ASS, but can easily be
adapted to accept several sources of images. I have submitted a series of
patch to implement that recently. The first ones have been applied, but the
next ones need to be reworked somewhat. I will go back to work on it as soon
as I am settled with my new professional time schedule.

EOSD is limited to monochrome overlays (OSD too, but not exactly the same
way). This works fine for ASS, since ASS glyphs are vectorial and
essentially monochrome. To add support for any bitmap overlay, the API must
be more carefully designed, especially about whose task it is to do the
scaling and colorspace conversion of the images. This is a long-term task,
but I think it will come to something eventually.

Regards,

-- 
  Nicolas George
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/attachments/20100905/f67fbffc/attachment.pgp>