[MPlayer-dev-eng] [PATCH] more AMD64 swscaler fixes

[Reimar Döffinger]

Hi,
On Sun, May 28, 2006 at 01:08:20PM +0200, Michael Niedermayer wrote:
> On Sun, May 28, 2006 at 12:28:42PM +0200, Reimar Döffinger wrote:
> > for AMD64 some variables passed via "m" constrained are 32bit values
> > whereas they need to be 64 bit values, because otherwise the comparison
> > that checks for e.g. end of a picture line will never come true -
> > causing a segfault sooner or later.
> > Attached patch fixes problems for me so far by replacing some ints by
> > longs.
> 
> the changes to xInc_mask are rejected, this variable should be uint16_t

After reading the code, obviously. Thanks to little-endian it does not
matter though *g*
So, would you prefer the attached patch that changes xInc_mask to
uin16_t, a seperate patch that changes xInc_mask or should xInc_mask
just be left as it is now?

Greetings,
Reimar Döffinger
-------------- next part --------------
--- postproc/swscale_template.c	2006-05-13 14:15:57.000000000 +0200
+++ postproc/swscale_template.c	2006-05-28 12:22:11.000000000 +0200
@@ -873,9 +873,9 @@
  */
 static inline void RENAME(yuv2packedX)(SwsContext *c, int16_t *lumFilter, int16_t **lumSrc, int lumFilterSize,
 				    int16_t *chrFilter, int16_t **chrSrc, int chrFilterSize,
-			    uint8_t *dest, int dstW, int dstY)
+			    uint8_t *dest, long dstW, long dstY)
 {
-	int dummy=0;
+	long dummy=0;
 	switch(c->dstFormat)
 	{
 #ifdef HAVE_MMX
@@ -2223,7 +2223,7 @@
 #endif
 }
       // *** horizontal scale Y line to temp buffer
-static inline void RENAME(hyscale)(uint16_t *dst, int dstWidth, uint8_t *src, int srcW, int xInc,
+static inline void RENAME(hyscale)(uint16_t *dst, long dstWidth, uint8_t *src, int srcW, int xInc,
 				   int flags, int canMMX2BeUsed, int16_t *hLumFilter,
 				   int16_t *hLumFilterPos, int hLumFilterSize, void *funnyYCode, 
 				   int srcFormat, uint8_t *formatConvBuffer, int16_t *mmx2Filter,
@@ -2336,8 +2336,8 @@
 	else
 	{
 #endif
-	int xInc_shr16 = xInc >> 16;
-	int xInc_mask = xInc & 0xffff;
+	long xInc_shr16 = xInc >> 16;
+	uint16_t xInc_mask = xInc & 0xffff;
 	//NO MMX just normal asm ...
 	asm volatile(
 		"xor %%"REG_a", %%"REG_a"	\n\t" // i
@@ -2535,7 +2535,7 @@
 	{
 #endif
 	long xInc_shr16 = (long) (xInc >> 16);
-	int xInc_mask = xInc & 0xffff; 
+	uint16_t xInc_mask = xInc & 0xffff; 
 	asm volatile(
 		"xor %%"REG_a", %%"REG_a"	\n\t" // i
 		"xor %%"REG_b", %%"REG_b"		\n\t" // xx