[MPlayer-dev-eng] Mencoder crash with x264 - mingw32 on p4 build, maybe other

[Zuxy Meng]

Hi,

2007/3/29, Gianluigi Tiesi <mplayer at netfarm.it>:
> I've been experimenting crashes when trying to encode using x264 or lavc
> h264, this happens on a p4 and not on a amd tb k7, this is strange
> because the code seams to be mmx.

No it's SSE2 code. A 0x66 before an MMX instruction make it an SSE2
one. Please update your unassembler:-)

> The build is on mingw but I don't known if it's really win32 only.
> I don't known too much mmx so I'm not able to locate the code in the
> source, but I have trapped the crash with ollydbg so I have some (maybe)
> usefull infos.
>
> The code where it crashes is here:
> 00AB08F4    0FFDC1          PADDW MM0,MM1
> 00AB08F7    66:0FFDD3       PADDW MM2,MM3
> 00AB08FB    66:0FFDC9       PADDW MM1,MM1
> 00AB08FF    66:0FFDDB       PADDW MM3,MM3
> 00AB0903    66:0FF9C8       PSUBW MM1,MM0
> 00AB0907    66:0FF9DA       PSUBW MM3,MM2
> 00AB090B    66:0FFDC2       PADDW MM0,MM2
> 00AB090F    66:0FFDCB       PADDW MM1,MM3
> 00AB0913    66:0FFDD2       PADDW MM2,MM2
> 00AB0917    66:0FFDDB       PADDW MM3,MM3
> 00AB091B    66:0FF9D0       PSUBW MM2,MM0
> 00AB091F    66:0FF9D9       PSUBW MM3,MM1
> 00AB0923    66:0FEFE4       PXOR MM4,MM4
> 00AB0927    66:0FEFED       PXOR MM5,MM5
> 00AB092B    66:0FF9E0       PSUBW MM4,MM0
> 00AB092F    66:0FF9EA       PSUBW MM5,MM2
> 00AB0933    66:0FEEC4       PMAXSW MM0,MM4
> 00AB0937    66:0FEED5       PMAXSW MM2,MM5
> 00AB093B    66:0FEFE4       PXOR MM4,MM4
> 00AB093F    66:0FEFED       PXOR MM5,MM5
> 00AB0943    66:0FF9E1       PSUBW MM4,MM1
> 00AB0947    66:0FF9EB       PSUBW MM5,MM3
> 00AB094B    66:0FEECC       PMAXSW MM1,MM4
> 00AB094F    66:0FEEDD       PMAXSW MM3,MM5
> 00AB0953    66:0FDDC1       PADDUSW MM0,MM1
> 00AB0957    66:0FDDD3       PADDUSW MM2,MM3
> 00AB095B    66:0FDDF0       PADDUSW MM6,MM0
> 00AB095F    66:0FDDF2       PADDUSW MM6,MM2
> 00AB0963    66:0F71D6 01    PSRLW MM6,1
> 00AB0968    66:0FF535 2C0CC>PMADDWD MM6,QWORD PTR DS:[CE0C2C]
>                         ^^^^^^^^^^^^
> crash here
>
> 00AB0970    0F12FE          MOVHLPS XMM7,XMM6
> 00AB0973    66:0FFEF7       PADDD MM6,MM7
> 00AB0977    F2:             PREFIX REPNE:                            ; Superfluous prefix
> 00AB0978    0F70FE 0E       PSHUFW MM7,MM6,0E

Actually this should be PSHUFLW XMM7, XMM6, 0E.

> 00AB097C    66:0FFEF7       PADDD MM6,MM7
> 00AB0980    66:0F7EF0       MOVD EAX,MM6
> 00AB0984    5B              POP EBX
> 00AB0985    C3              RETN
>
> DS:[00CE0C2C]=0001 0001 0001 0001
> MM6=ABAB ABAB ABAB ABAB
>
> Dump of registers
> EAX 019916A8
> ECX 019913E8
> EDX 00000010
> EBX 00000020
> ESP 0022CC18
> EBP 00000000
> ESI 0198F330
> EDI 0022CEB4
> EIP 00AB0968 mencoder.00AB0968
> C 0  ES 0023 32bit 0(FFFFFFFF)
> P 1  CS 001B 32bit 0(FFFFFFFF)
> A 0  SS 0023 32bit 0(FFFFFFFF)
> Z 0  DS 0023 32bit 0(FFFFFFFF)
> S 0  FS 003B 32bit 7FFDF000(FFF)
> T 0  GS 0000 NULL
> D 0
> O 0  LastErr ERROR_SUCCESS (00000000)
> EFL 00200206 (NO,NB,NE,A,NS,PE,GE,G)
> MM0 8080 8080 8080 8080
> MM1 8080 8080 8080 8080
> MM2 8080 8080 8080 8080
> MM3 8080 8080 8080 8080
> MM4 ABAB ABAB ABAB ABAB
> MM5 ABAB ABAB ABAB ABAB
> MM6 ABAB ABAB ABAB ABAB
> MM7 ABAB ABAB ABAB ABAB
>
> attached stdout/stderr of mencoder -v
> Using x264 or lavc h264 crashes mencoder in the same place
>
> Any clue?
> Regards
>

-- 
Zuxy
Beauty is truth,
While truth is beauty.
PGP KeyID: E8555ED6