[MPlayer-dev-eng] decoding of MS-RLE in 4-bit is buggy

[Roberto Togni]

On 2003.06.13 20:13 Michael Günnewig wrote:
> Hallo.
> 
> The decoding of Microsoft-RLE in 4-bit doesn't work correctly under
> all circumstances. As described at
> 
>   http://www.pcisys.net/~melanson/codecs/msrle.txt
> 
> must the code be aligned at a word-boundary when in absolute mode. I
> have an AVI file which decodes correctly under Windows. The
> problematic segment is the following (in hex):
> 
>   00 09 77 70 78 88 80 00 3A 88 00 08 77 07 7F FF 29 FF 00 00
> 
> This code results that the variable rle_code will be 9 / 2, which will
> be 4. This will decode the next 4 bytes into the corresponding 8
> pixels. The 9th pixel will be interpreted by mplayer as an
> encoded-code-segment instead of be a pixel with pal-index 9 (and skip
> the 0 in the lsb) and an extra-byte for alignment. This results in
> very strange decoded frames and warning-messages about violated
> boundaries:
> 
>   MS RLE: stream ptr just went out of bounds (1)
> 
> The fix would be to round up the computation of rle_code and leave the
> for-loop in the middle when rounded up, or? But my problem is, that is
> won't solve the distorted frames. Any hints?
There are at least two problems:
- the extra_byte flag needs to be computed on stream_byte, not on 
rle_code (it must be done on byte count, not on nibble count). To be 
exact, ytou have an extra byte if (stream_byte/2 + stream_byte%2) is odd
- the second problem is the 9th nibble not being decoded at all.

The logical solution is:
- fix extra_byte computation
- do the first half of the for cycle for the last nibble if nibble 
counter is even (can be implemented in various ways, i changed the for 
to be executed 9 times (in this example) and i use the i variable to 
find out if i must decode the high or low nibble of the byte)
- increment the stream pointer afret the for cycle by stram_byte/2 + 
stream_byte%2
- if i have an extra byte, increment stream pointer again.

But it doesn't work!
I dont't have any more errors, but the image is distorted. Looks like 
something goes wrong when it checks if the current pixel position is 
greater than the current line length.

I'm still looking at it.

> 
> The problematic AVI file (60 kB size) is available at:
>   http://ls6-www.cs.uni-dortmund.de/~guennewi/mplayer-msrle-4bit.avi
Do you have any other samples with 4 bit msrle? I'd expecially like to 
have a sample that works now, to check that my fixes don't break 
anything (even if, looking at the code, i wonder if it even worked at 
all).

> 
> 
>   Michael
> 
Ciao,
  Roberto