[MPlayer-dev-eng] MPlayer linking takes forever on MinGW

[Georgi Petrov]

Problem "solved"!!!

Please read the complete solution without VMware or anything like
that. You will be surprised!!!

When every attempt so far failed, I did set up a XP installation from
inside VMware, which was going to compile/link MPlayer from a shared
folder from the host (Windows 7). The host (Win 7) should provide a
shared tree and the guest (XP) should access the tree and compile/link
mplayer.exe. Let me remind you that ld.exe doesn't hang in XP.

You can't imagine my surprise when linking hanged! Actually my Windows
7 "side" hanged and it was responsible only for providing MPlayer's
tree as a directory share!!! Now the whole VMware process couldn't be
killed, just like when running native ld.exe...

I was in shock, because it didn't make sense.

Using Sysinternals Process Monitor, I was able to isolate the very
same problem when:

1) ld.exe in Windows 7 (in VMware or on my machine) hangs.
2) Windows 7 on my machine hangs while only providing a file share to
the XP, trying to link from inside VMware.

ld.exe doesn't hang. VMware's process (with XP and ld.exe inside)
didn't hang. The NT kernel did!!!

This is the stack from hanged ld.exe (and supposedly VMware's) process:

ntdll.dll!KiFastSystemCallRet
kernel32.dll!WriteFile+0x4e
msvcrt.dll!fprintf+0x235
msvcrt.dll!write+0x73
msvcrt.dll!CIcos+0x248
msvcrt.dll!fscanf_s_l+0x68
msvcrt.dll!fseeki64+0x2a
msvcrt.dll!fsetpos+0x3f
ld.exe+0x96e09
ld.exe+0x34c1b
ld.exe+0x400d1
ld.exe+0x2560a
ld.exe+0x4f2ac
ld.exe+0x50083
ld.exe+0x14543
ld.exe+0x134f1
ld.exe+0x10db
ld.exe+0x1178
kernel32.dll!BaseThreadInitThunk+0x12
ntdll.dll!RtlInitializeExceptionChain+0x63
ntdll.dll!RtlInitializeExceptionChain+0x36

The "hanged call" is "KiFastSystemCallRet" in ntdll.dll!

This explains why I can't kill ld.exe or VMware's process (with
"hanged" ld.exe inside). They are waiting for the NT executive to
finish a system call! This was jaw-dropping.

Obviously XP's function call returned, but Windows 7's one hanged.

In theory this means that I can host the tree in XP inside VMware or
even Linux inside VMware and share it to my machine's Windows 7.

Fortunately, a strange thought went through my head - what about the
file system? Would the same bug in ntdll.dll happen on a FAT32
partition?

So, I created one, copied the tree inside and.. guess what...

It didn't hang. mplayer.exe was built.

Yes, the tree compiles and links just fine on my Windows 7 (no VMware)
on a FAT32 partition...

This doesn't solve the problem itself, but "works for me". I guess
that ld.exe's file write logic somehow triggers a bug in ntdll.dll
under certain circumstances. This is only a speculation from my
uneducated point of view.

See the last few calls:
ntdll.dll!KiFastSystemCallRet
kernel32.dll!WriteFile+0x4e
msvcrt.dll!fprintf+0x235
msvcrt.dll!write+0x73

MSVCRT's write function uses fprintf to write to a stream (file stream
obviously), then we have kernel32.dll's WriteFile and finally
ntdll.dll's hanging KiFastSystemCallRet.

This is insane, but a proven fact. My problem is solved.

Should I contact Microsoft and file something like a bug report?