682788 - link.exe of VS2010 x64 crashes during PGO

Status: RESOLVED INCOMPLETE

(Firefox Build System :: MozillaBuild, task)

Description

[Makoto Kato [:m_kato]]

When turning on PGO on VS2010 SP1 x64, linker (link.exe) seem to crash.

c:\Workspace\hg.mozilla.org\mozilla-win64\netwerk\cache\nsDiskCacheMap.cpp : fatal error C1001: An internal error has occurred in the compiler.
(compiler file 'f:\dd\vctools\compiler\utc\src\p2\main.c[0x000007FEEF7C6882:0x00000000008CCF42]', line 183)
 To work around this problem, try simplifying or changing the program near the locations listed above.
Please choose the Technical Support command on the Visual C++
 Help menu, or open the Technical Support help file for more information

LINK : fatal error LNK1000: Internal error during IMAGE::BuildImage
make[1]: *** [xul.dll] Error 255

Comment 1

[Makoto Kato [:m_kato]]

This may be able to fix using x86_amd64 version...

Comment 2

[Makoto Kato [:m_kato]]

Attachment: fix v1

Comment 3

[(not currently active) Ted Mielczarek]

Comment on attachment 556490 [details] [diff] [review]
fix v1

That seems really unfortunate. That means we'll wind up restricted to only 4GB of address space for the linker. :-/

Comment 4

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Can we file a MS bug and see if we can get a hotfix instead?

Comment 5

[Makoto Kato [:m_kato]]

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #4)
> Can we file a MS bug and see if we can get a hotfix instead?

I have filed a bug to connect.  https://connect.microsoft.com/VisualStudio/feedback/details/686117/64-bit-linker-causes-ice-on-huge-dll-during-pginstrument

Does we have support account to get new hotfix?  They will need a premier contract for new hotfix.

Comment 6

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

We have something with them, I think.  Ted knows more.

Comment 7

[(not currently active) Ted Mielczarek]

I don't know anything about that. I've just filed bugs at connect as a normal user.

Comment 8

[Atsushi Okawa]

Comment on attachment 556490 [details] [diff] [review]
fix v1

Review of attachment 556490 [details] [diff] [review]:
-----------------------------------------------------------------

I think this patch is useful but not enough.

The following line is required to build Thunderbird or SeaMonkey with x64
after 'call "%VC10DIR%\bin\x86_amd64\vcvarsx86_amd64.bat"'.

set "PATH=%SDKDIR%\bin\x64;%PATH%"

Because %SDKDIR%bin\midl.exe makes mailnews\mapi\mapihook\build\msgMapi_p.c
like this.

>#if !defined(_M_IA64) && !defined(_M_AMD64)

This causes error when MapiProxy.dll is linked.

Otherwise %SDKDIR%bin\x64\midl.exe make that C source file 
just below.

>#if defined(_M_AMD64)

Comment 9

[ABE Hiroki (:hATrayflood)]

(In reply to Atsushi Okawa from comment #8)
> Comment on attachment 556490 [details] [diff] [review]
> fix v1
> 
> Review of attachment 556490 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> I think this patch is useful but not enough.
> 
> The following line is required to build Thunderbird or SeaMonkey with x64
> after 'call "%VC10DIR%\bin\x86_amd64\vcvarsx86_amd64.bat"'.
> 
> set "PATH=%SDKDIR%\bin\x64;%PATH%"
> 
> Because %SDKDIR%bin\midl.exe makes mailnews\mapi\mapihook\build\msgMapi_p.c
> like this.
> 
> >#if !defined(_M_IA64) && !defined(_M_AMD64)
> 
> This causes error when MapiProxy.dll is linked.
> 
> Otherwise %SDKDIR%bin\x64\midl.exe make that C source file 
> just below.
> 
> >#if defined(_M_AMD64)

I tried to build comm-central, and build successful.
Can you take a build log ?

Comment 10

[Atsushi Okawa]

Sorry, I have already deleted directories.
I'm going to do it again.

Comment 11

[Atsushi Okawa]

Attachment: Error log

Using v1 patch, this error occurs during building the SeaMonkey 2.3.3

Comment 12

[Atsushi Okawa]

Attachment: C source which MIDL generates.

Comment 13

[ABE Hiroki (:hATrayflood)]

oh, sorry sorry, I set midl flag.

set MIDL_FLAGS=-x64

but comm-central/configure.in is buggy. Bug 685050

Comment 14

[Atsushi Okawa]

I'm alright.I don't aware that the problem of comm-xxx.
So I cancel my review.

Comment 15

[ABE Hiroki (:hATrayflood)]

(In reply to Makoto Kato from comment #5)
> (In reply to Kyle Huey [:khuey] ([e-mail]) from comment #4)
> > Can we file a MS bug and see if we can get a hotfix instead?
> 
> I have filed a bug to connect. 
> https://connect.microsoft.com/VisualStudio/feedback/details/686117/64-bit-
> linker-causes-ice-on-huge-dll-during-pginstrument
> 
> Does we have support account to get new hotfix?  They will need a premier
> contract for new hotfix.

I sent link repro (file name: LINK_REPRO.7z) to msconnect.

Comment 16

[(not currently active) Ted Mielczarek]

Comment on attachment 556490 [details] [diff] [review]
fix v1

I would rather not make this the default. I would be okay with adding a new batch file that sets up the x86->x64 toolchain.

Comment 17

[Atsushi Okawa]

Building Thunderbird 8.0 beta 1 candidate causes the similar error using the compiler of a x86_amd64 version.

make[6]: Entering directory `/c/Users/PCUser/mozilla/thunderbird/thunderbird-8.0
b1.source/obj64/mozilla/js/src'
c:/mozilla-build/python/python2.7.exe /c/Users/PCUser/mozilla/thunderbird/thunde
rbird-8.0b1.source/comm-beta/mozilla/js/src/config/pythonpath.py -I./config /c/U
sers/PCUser/mozilla/thunderbird/thunderbird-8.0b1.source/comm-beta/mozilla/js/sr
c/config/expandlibs_exec.py --uselist -- link -NOLOGO -DLL -OUT:mozjs.dll -PDB:m
ozjs.pdb -SUBSYSTEM:WINDOWS  jsalloc.obj jsanalyze.obj jsapi.obj jsarena.obj jsa
rray.obj jsatom.obj jsbool.obj jsclone.obj jscntxt.obj jscompartment.obj jsdate.
obj jsdbgapi.obj jsdhash.obj jsdtoa.obj jsemit.obj jsexn.obj jsfriendapi.obj jsf
un.obj jsgc.obj jsgcmark.obj jsgcchunk.obj jsgcstats.obj jscrashreport.obj jshas
h.obj jsinterp.obj jsinvoke.obj jsiter.obj jslock.obj jslog2.obj jsmath.obj jsna
tivestack.obj jsnum.obj jsobj.obj json.obj jsonparser.obj jsopcode.obj jsparse.o
bj jsproxy.obj jsprf.obj jsprobes.obj jspropertycache.obj jspropertytree.obj jsr
eflect.obj jsregexp.obj jsscan.obj jsscope.obj jsscript.obj jsstr.obj jstypedarr
ay.obj jsutil.obj jswatchpoint.obj jsweakmap.obj jswrapper.obj jsxdrapi.obj jsxm
l.obj prmjtime.obj sharkctl.obj Debugger.obj GlobalObject.obj Stack.obj String.o
bj ParseMaps.obj Unicode.obj jstracer.obj Assembler.obj Allocator.obj CodeAlloc.
obj Containers.obj Fragmento.obj LIR.obj njconfig.obj RegAlloc.obj avmplus.obj N
ativeX64.obj jsbuiltins.obj VMPI.obj Writer.obj MethodJIT.obj StubCalls.obj Comp
iler.obj FrameState.obj FastArithmetic.obj FastOps.obj StubCompiler.obj MonoIC.o
bj PolyIC.obj ImmutableSync.obj InvokeHelpers.obj Retcon.obj TrampolineCompiler.
obj checks.obj conversions.obj diy-fp.obj v8-dtoa.obj fast-dtoa.obj platform.obj
 utils.obj Assertions.obj ExecutableAllocatorPosix.obj ExecutableAllocatorWin.ob
j ExecutableAllocatorOS2.obj ExecutableAllocator.obj ARMAssembler.obj Logging.ob
j MacroAssemblerARM.obj MacroAssemblerX86Common.obj OSAllocatorOS2.obj OSAllocat
orPosix.obj OSAllocatorWin.obj PageBlock.obj YarrInterpreter.obj YarrJIT.obj Yar
rPattern.obj YarrSyntaxChecker.obj CTypes.obj Library.obj jsperf.obj pm_stub.obj
 TrampolineMasmX64.obj    -LARGEADDRESSAWARE -NXCOMPAT -DYNAMICBASE  -LIBPATH:./
../../dist/lib -NODEFAULTLIB:msvcrt -NODEFAULTLIB:msvcrtd -NODEFAULTLIB:msvcprt
-NODEFAULTLIB:msvcprtd -DEFAULTLIB:mozcrt -LTCG:PGUPDATE   ctypes/libffi/.libs/l
ibffi.lib  c:/Users/PCUser/mozilla/thunderbird/thunderbird-8.0b1.source/obj64/mo
zilla/dist/lib/nspr4.lib c:/Users/PCUser/mozilla/thunderbird/thunderbird-8.0b1.s
ource/obj64/mozilla/dist/lib/plc4.lib c:/Users/PCUser/mozilla/thunderbird/thunde
rbird-8.0b1.source/obj64/mozilla/dist/lib/plds4.lib  kernel32.lib user32.lib gdi
32.lib winmm.lib wsock32.lib advapi32.lib
PGOMGR : 警告 PG0188: 'mozjs!*.pgc' に一致する .PGC ファイルは見つかりませんでし
た。
   ライブラリ mozjs.lib とオブジェクト mozjs.exp を作成中
コード生成しています。
c:\users\pcuser\mozilla\thunderbird\thunderbird-8.0b1.source\comm-beta\mozilla\j
s\src\jscntxt.cpp(355) : fatal error C1001: コンパイラで内部エラーが発生しました
。
(コンパイラ ファイル 'f:\dd\vctools\compiler\utc\src\p2\main.c[0xFFFFFE1D:0xFFFF
FE1D]'、行 183)
 この問題を回避するには、上記の場所付近のプログラムを単純化するか変更してくださ
い。
詳細については、Visual C++ ヘルプ メニューのサポート情報コマンドを
選択してください。またはサポート情報 ヘルプ ファイルを参照してください。

LINK : fatal error LNK1000: Internal error during IMAGE::BuildImage

  Version 10.00.40219.01

  ExceptionCode            = C0000005
  ExceptionFlags           = 00000000
  ExceptionAddress         = FFFFFE1D
  NumberParameters         = 00000002
  ExceptionInformation[ 0] = 00000000
  ExceptionInformation[ 1] = FFFFFE1D

CONTEXT:
  Eax    = FFFFFFFC  Esp    = 002AE7D0
  Ebx    = 1197E500  Ebp    = 002AE7EC
  Ecx    = 0000047A  Esi    = 1197E6F0
  Edx    = 00000008  Edi    = 00000001
  Eip    = FFFFFE1D  EFlags = 00010246
  SegCs  = 00000023  SegDs  = 0000002B
  SegSs  = 0000002B  SegEs  = 0000002B
  SegFs  = 00000053  SegGs  = 0000002B
  Dr0    = 00000000  Dr3    = 00000000
  Dr1    = 00000000  Dr6    = 00000000
  Dr2    = 00000000  Dr7    = 00000000
299 of 16211 (  1.84) profiled functions will be compiled for speed
make[6]: *** [mozjs.dll] Error 232
make[6]: *** Deleting file `mozjs.dll'
make[6]: Leaving directory `/c/Users/PCUser/mozilla/thunderbird/thunderbird-8.0b
1.source/obj64/mozilla/js/src'
make[5]: *** [libs_tier_js] Error 2
make[5]: Leaving directory `/c/Users/PCUser/mozilla/thunderbird/thunderbird-8.0b
1.source/obj64/mozilla'
make[4]: *** [tier_js] Error 2
make[4]: Leaving directory `/c/Users/PCUser/mozilla/thunderbird/thunderbird-8.0b
1.source/obj64/mozilla'
make[3]: *** [default] Error 2
make[3]: Leaving directory `/c/Users/PCUser/mozilla/thunderbird/thunderbird-8.0b
1.source/obj64/mozilla'
make[2]: *** [default] Error 2
make[2]: Leaving directory `/c/Users/PCUser/mozilla/thunderbird/thunderbird-8.0b
1.source/obj64'
make[1]: *** [build] Error 2
make[1]: Leaving directory `/c/Users/PCUser/mozilla/thunderbird/thunderbird-8.0b
1.source/comm-beta'
make: *** [profiledbuild] Error 2

Comment 18

[Atsushi Okawa]

I'm sorry.
Above comments is not valid. 
Please ignore it.

Comment 19

[xunxun]

Even if I use the mozilla-build X64 patch, I fail to build FF8 release using VC2010 SP1 X64.
It will show that when linking xul.dll:

d:\mozilla\js\src\ctypes\ctypes.cpp(1461):  fatal error C1001: An internal error has occurred in the compiler.
(compiler file 'f:\dd\vctools\compiler\utc\src\p2\main.c[0x709037A1:0x00000010]', line 183)
 To work around this problem, try simplifying or changing the program near the locations listed above.
Please choose the Technical Support command on the Visual C++
 Help menu, or open the Technical Support help file for more information

LINK : fatal error LNK1000: Internal error during IMAGE::BuildImage

Version : 10.00.40219.01

ExceptionCode            = C0000005
ExceptionFlags           = 00000000
ExceptionAddress         = 709037A1 (70750000) "d:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN\x86_amd64\c2.dll"
NumberParameters         = 00000002
ExceptionInformation[ 0] = 00000000
ExceptionInformation[ 1] = 00000010

CONTEXT:
EAX ..... (omitted)
3688 of 168332 (  2.19) profiled functions will be compiled for speed
make[1]: *** [xul.dll] Error 232

Comment 20

[Mark Straver]

I've had the same crash (on Image::BuildImage) C0000005 without using PGO but just using the x64 compiler with LTCG on Firefox 8.0 release (specifying -GL as opt flag to improve speed without PGO). The source location where it fails seems to be rather random, although most end up in the /layout modules. The cross-compiler seems to fix it for now, but it seems something rather fundamental that is broken in the compiler DLL.

Perhaps the xul dll just gets too big and complex to deal with? Is there a way to use PGO on a non-libxul build to see if that works any better?

Comment 21

[Makoto Kato [:m_kato]]

(In reply to xunxun from comment #19)

xunxun, could you file a bug to connect.microsoft.com?  I don't know x86 linker crash.


(In reply to Mark Straver from comment #20)

Mark, you should file it to connect.microsoft.com.  I don't want to discuss linker bug on this bugzilla.  When we switch to VS2010 on automation build then this occurs on it, I will investigate it.

Comment 22

[xunxun]

(In reply to Makoto Kato from comment #21)
> (In reply to xunxun from comment #19)
> 
> xunxun, could you file a bug to connect.microsoft.com?  I don't know x86
> linker crash.
> 
> 
I post it on https://connect.microsoft.com/VisualStudio/feedback/details/700201/x86-amd64-linker-causes-ice-on-huge-dll-during-pginstrument-using-vc2010-sp1#details

Comment 23

[Mark Straver]

> (In reply to Mark Straver from comment #20)
> 
> Mark, you should file it to connect.microsoft.com.  I don't want to discuss
> linker bug on this bugzilla.  When we switch to VS2010 on automation build
> then this occurs on it, I will investigate it.

I'll see what I can do.

Comment 24

[xunxun]

I think to use a workaround PGO, we should disable some code using -GL.
For example, if the error shows : d:\mozilla\js\src\ctypes\ctypes.cpp(1461):  fatal error C1001: An internal error has occurred in the compiler.

we should using -GL- recompile ctypes.cpp

Comment 25

[Mark Straver]

(In reply to xunxun from comment #24)
> we should using -GL- recompile ctypes.cpp

It would be a good idea if the location it fails at would be the same source file every time, but I've seen the compile fail in layout/generic, layout/styles, gfx/*, etc. - it is extremely random what source file it decides to finally fail on in the process.

Looking at the Microsoft bug, and the fact that it errors on "readsym" which I gather stands for "read symbol", I wonder if the culprit would be ICF as it scans symbols for identical content and folds it... What if we instruct the linker to not fold with -OPT:NOICF?

Comment 26

[Makoto Kato [:m_kato]]

(In reply to Mark Straver from comment #25)
> Looking at the Microsoft bug, and the fact that it errors on "readsym" which
> I gather stands for "read symbol", I wonder if the culprit would be ICF as
> it scans symbols for identical content and folds it... What if we instruct
> the linker to not fold with -OPT:NOICF?

Unfortunately, even if we don't use -OPT (REF and ICF), this still occur.  Actually, there is no way except to use 32-bit linker...

Comment 27

[Ryan VanderMeulen [:RyanVM]]

Not sure if there's really an issue here anymore or not, but I'm quite certain that nothing's going to happen here one way or another now that MSVC2013 seems to work OK.