[MPlayer-dev-eng] segv in swscaler YV12->YUY2 with SIMD on x86_64

Jason Tackaberry tack at urandom.ca
Sun Apr 16 00:56:34 CEST 2006


I can consistently reproduce a crash in swscaler on amd64 with
YV12->YUY2 conversion.  I have a filter that supports only YUY2:
        
        static int
        query_format(struct vf_instance_s* vf, unsigned int fmt)
        {
            if (fmt == IMGFMT_YUY2)
                return vf_next_query_format(vf, fmt);
            return 0;
        }

Mplayer will helpfully do the colorspace conversion for me, but it
crashes on am64. :)  It will crash with MMX2, 3DNOW, and MMX, but if I
disable all three, there's no crash.  Here's a traceback:


        SwScaler: BICUBIC scaler, from Planar YV12 to Packed YUY2 using MMX2
        VO: [null] 618x464 => 618x464 Packed YUY2
        A:   0.0 V:   0.0 A-V:  0.009 ct:  0.000   1/  1 ??% ??% ??,?% 0 0
        Program received signal SIGSEGV, Segmentation fault.
        [Switching to Thread 46912507208448 (LWP 23628)]
        0x0000000000594654 in swScale_MMX2 ()
        (gdb) bt
        #0  0x0000000000594654 in swScale_MMX2 ()
        #1  0x000000000058b7f3 in sws_scale_ordered ()
        #2  0x0000000000495198 in scale ()
        #3  0x00000000004951f0 in draw_slice ()
        #4  0x000000000046de98 in draw_slice ()
        #5  0x00000000005b4831 in ff_draw_horiz_band ()
        #6  0x00000000006aa4d5 in decode_slice ()
        #7  0x00000000006ab439 in ff_h263_decode_frame ()
        #8  0x00000000005b0e80 in avcodec_decode_video ()
        #9  0x000000000046e6d6 in decode ()
        #10 0x000000000046b404 in decode_video ()
        #11 0x0000000000415f20 in main ()
        
Happy to provide more info if needed.

Jason.




More information about the MPlayer-dev-eng mailing list