On Tue, Jul 01, 2014 at 03:45:31PM +0200, Matthijs van Otterdijk wrote:

sub/spudec.c:spudec_packet_fill() optionally draws rectangles with an x and y offset, which is used by sub/av_sub.c:avsub_to_spudec() in case of multiple rects. The way this is done now causes a heap overflow in spudec.c:pal2gray_alpha(). spudec_packet_fill() offsets img and aimg by x before calling pal2gray_alpha(). pal2gray_alpha() writes dst_stride pixels for each line in the rect. In case the bottom rectangle (and therefore the rectangle situated at the end of the packet buffer) has an x offset, this will cause x 0s to be written past the end of the packet buffer.

The attached patch fixes this by making pal2gray_alpha() handle the x offset, rather than spudec_packet_fill().

Really sorry for the long delay, with holidays in-between I forgot about it. Is there an easy test-case? I think this change is wrong (or at least, it still leaves issues). It means that if there are two subtitle rects next to each other, only one will ever be visible. I think instead we need to skip the clearing part (avsub_to_spudec will pre-clear the image anyway). I would welcome testing of attached patch. Regards, Reimar