[MEncoder-users] [BUG]: mencoder segfaults on accurate SW rounding on 64-bit

[Grozdan Nikolov]

Hi,

I just discovered that MEncoder segfaults when enabling accurate rounding of 
the vertical software scaler. I've compiled it with debugging information and 
ran a backtrace on it...

System: openSUSE 10.2 - 64-bit
MEncoder: SVN-r23784
GCC: 4.1.2



gdb mencoder
(gdb): run  dvd://1 -aid 128 -ovc xvid -xvidencopts bitrate=1000 -oac 
mp3lame -lameopts abr:br=120 -vf scale=512:384:0:0:0.00:0.71:::1 -o 
$HOME/test.avi 

...[snip].....

Opening video filter: [scale w=512 h=384 interlaced=0 chr-drop=0 param=0.00 
param2=0.71 arnd=1]

...[snip].....

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 47429030926208 (LWP 25413)]
0x0000000000549bc0 in swScale_MMX2 (c=0x1021a40, src=0x7fffbfcd4320, 
srcStride=0x7fffbfcd4340, srcSliceY=0,
    srcSliceH=576, dst=0x1021670, dstStride=0x7fffbfcd4350) at 
swscale_template.c:937
937                 YSCALEYUV2YV12X_ACCURATE(   0, CHR_MMX_FILTER_OFFSET, 
uDest, chrDstW)

(gdb): bt

#0  0x0000000000549bc0 in swScale_MMX2 (c=0x1021a40, src=0x7fffbfcd4320, 
srcStride=0x7fffbfcd4340,
    srcSliceY=0, srcSliceH=576, dst=0x1021670, dstStride=0x7fffbfcd4350) at 
swscale_template.c:937
#1  0x000000000054218f in sws_scale (c=0x1021a40, src=<value optimized out>, 
srcStride=0x102bcc0,
    srcSliceY=0, srcSliceH=576, dst=0x1021670, dstStride=0x1021690) at 
swscale.c:2545
#2  0x000000000048b86b in scale (sws1=0x1024758, sws2=0x1021a40, src=<value 
optimized out>,
    src_stride=0x102bcc0, y=26538752, h=6, dst=0xfffffffffffffe00, 
dst_stride=0x1021690, interlaced=0)
    at vf_scale.c:351
#3  0x000000000048b96b in put_image (vf=0xf96420, mpi=0x102bc80, 
pts=-9.2233720368547758e+18)
    at vf_scale.c:379
#4  0x00000000004675d7 in filter_video (sh_video=<value optimized out>, 
frame=0x102bc80,
    pts=-9.2233720368547758e+18) at dec_video.c:417
#5  0x0000000000433053 in main (argc=<value optimized out>, argv=<value 
optimized out>) at mencoder.c:1316