[MPlayer-dev-eng] [PATCH] spudec: fix heap overflow in pal2gray_alpha()

[Reimar Döffinger]

On Tue, Jul 01, 2014 at 03:45:31PM +0200, Matthijs van Otterdijk wrote:
> sub/spudec.c:spudec_packet_fill() optionally draws rectangles with an x and
> y offset, which is used by sub/av_sub.c:avsub_to_spudec() in case of
> multiple rects. The way this is done now causes a heap overflow in
> spudec.c:pal2gray_alpha().
> spudec_packet_fill() offsets img and aimg by x before calling
> pal2gray_alpha(). pal2gray_alpha() writes dst_stride pixels for each line
> in the rect. In case the bottom rectangle (and therefore the rectangle
> situated at the end of the packet buffer) has an x offset, this will cause
> x 0s to be written past the end of the packet buffer.
> 
> The attached patch fixes this by making pal2gray_alpha() handle the x
> offset, rather than spudec_packet_fill().

Really sorry for the long delay, with holidays in-between I forgot about
it.
Is there an easy test-case?
I think this change is wrong (or at least, it still leaves issues).
It means that if there are two subtitle rects next to each other,
only one will ever be visible.
I think instead we need to skip the clearing part (avsub_to_spudec
will pre-clear the image anyway).
I would welcome testing of attached patch.

Regards,
Reimar
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pal2gray.diff
Type: text/x-diff
Size: 1637 bytes
Desc: not available
URL: <https://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/attachments/20140904/6cd8910f/attachment.bin>