1340395 - Linker busting between fatal errors "LNK1000: Internal error during LIB::Search" and "LNK1102: Out of Memory"

Status: RESOLVED FIXED

(Thunderbird :: Build Config, defect)

Description

[Edmund Wong (:ewong)]

Got this with a c-c win32 build:

    ..\..\modules\zlib\src\inflate.obj
    ..\..\modules\zlib\src\inftrees.obj
    ..\..\modules\zlib\src\trees.obj
    ..\..\modules\zlib\src\uncompr.obj
    ..\..\modules\zlib\src\zutil.obj
    StaticXULComponentsEnd\StaticXULComponentsEnd.obj


LINK : fatal error LNK1000: Internal error during LIB::Search

  Version 14.00.24213.1

  ExceptionCode            = C0000005
  ExceptionFlags           = 00000000
  ExceptionAddress         = 000007FEF217C387 (000007FEF2170000) "c:\builds\slave\c-c\build\vs2015u3\VC\redist\x64\Microsoft.VC140.CRT\VCRUNTIME140.dll"
  NumberParameters         = 00000002
  ExceptionInformation[ 0] = 0000000000000000
  ExceptionInformation[ 1] = 0000000405D82000

CONTEXT:
  Rax    = 0000000011738150  R8     = 00000000031CD270
  Rbx    = 0000000011738110  R9     = 0000000000000000
  Rcx    = 0000000000000298  R10    = 0000000405D81CD0
  Rdx    = 00000003F4649B80  R11    = 0000000000000608
  Rsp    = 000000000019D1A8  R12    = 0000000000000D79
  Rbp    = 00000000000005C8  E13    = 00000000031CD270
  Rsi    = 0000000405D82000  R14    = 0000000405D81CD0
  Rdi    = 0000000011738480  R15    = 0000000000000000
  Rip    = 000007FEF217C387  EFlags = 0000000000010207
  SegCs  = 0000000000000033  SegDs  = 000000000000002B
  SegSs  = 000000000000002B  SegEs  = 000000000000002B
  SegFs  = 0000000000000053  SegGs  = 000000000000002B
  Dr0    = 0000000000000000  Dr3    = 0000000000000000
  Dr1    = 0000000000000000  Dr6    = 0000000000000000
  Dr2    = 0000000000000000  Dr7    = 0000000000000000
c:/builds/slave/c-c/build/mozilla/config/rules.mk:787: recipe for target 'xul.dll' failed
mozmake.exe[4]: *** [xul.dll] Error 1000
mozmake.exe[4]: *** Deleting file 'xul.dll'

somewhat akin to bug 436279 but that's been resolved (if only because
it hasn't cropped up in a while).

Just filing this just in case something needs to be done.

Comment 1

[Edmund Wong (:ewong)]

This is on the loaner.  Win32 build.

Both nightly and normal build fails with either OOM or LNK1000

Comment 2

[Edmund Wong (:ewong)]

Attachment: log for nightly build.

Comment 3

[Edmund Wong (:ewong)]

Last good build:
https://hg.mozilla.org/mozilla-central/rev/a9ec72f82299

First bad build:
https://hg.mozilla.org/mozilla-central/rev/8c8b54b13be7

Comment 4

[Edmund Wong (:ewong)]

Improvement on the last good, first bad:

Last good:
https://hg.mozilla.org/mozilla-central/rev/e783bdf2cb50

First Bad:
https://hg.mozilla.org/mozilla-central/rev/e9b926463f9e

Comment 5

[Edmund Wong (:ewong)]

(In reply to Edmund Wong (:ewong) from comment #4)
> Improvement on the last good, first bad:
> 
> Last good:
> https://hg.mozilla.org/mozilla-central/rev/e783bdf2cb50
> 

Actually.. the last good is still https://hg.mozilla.org/mozilla-central/rev/a9ec72f82299

Comment 6

[Edmund Wong (:ewong)]

> First Bad:
> https://hg.mozilla.org/mozilla-central/rev/e9b926463f9e

https://hg.mozilla.org/mozilla-central/rev/2737f66ad6ac gives a 


    StaticXULComponentsEnd\StaticXULComponentsEnd.obj

js_static.lib(Unified_cpp_js_src25.obj) : fatal error LNK1318: Unexpected PDB er
ror; OK (0) ''
c:/builds/slave/c-c/build/mozilla/config/rules.mk:787: recipe for target 'xul.dl
l' failed
mozmake.exe[4]: *** [xul.dll] Error 1318
mozmake.exe[4]: Leaving directory 'c:/builds/slave/c-c/build/objdir/toolkit/libr
ary'
c:/builds/slave/c-c/build/mozilla/config/recurse.mk:71: recipe for target 'toolk
it/library/target' failed
mozmake.exe[3]: *** [toolkit/library/target] Error 2

so not sure if it's related.

Comment 7

[Ian Neal]

How much memory has the loaner got? How much free memory has it got? Has it been rebooted recently?

Comment 8

[Edmund Wong (:ewong)]

(In reply to Ian Neal from comment #7)
> How much memory has the loaner got? How much free memory has it got? Has it
> been rebooted recently?

15GB and yes.  It has been rebooted.  

w/ m-c = https://hg.mozilla.org/mozilla-central/rev/a9ec72f82299,

it builds.

w/ any cset post a9ec72f82299, it gives me either errors.  

Now, with tip on c-c and m-c,  it's taken more than 20 minutes to
link, which isn't right since I think it's taken more than 1/2 hr.
This is with a clobber build.

Comment 9

[Edmund Wong (:ewong)]

Attachment: 2nd type of error

Comment 10

[Edmund Wong (:ewong)]

(In reply to Edmund Wong (:ewong) from comment #9)
> Created attachment 8839043 [details]
> 2nd type of error

by 2nd type of error, I meant:

    ..\..\modules\zlib\src\gzwrite.obj
    ..\..\modules\zlib\src\infback.obj
    ..\..\modules\zlib\src\inffast.obj
    ..\..\modules\zlib\src\inflate.obj
    ..\..\modules\zlib\src\inftrees.obj
    ..\..\modules\zlib\src\trees.obj
    ..\..\modules\zlib\src\uncompr.obj
    ..\..\modules\zlib\src\zutil.obj
    StaticXULComponentsEnd\StaticXULComponentsEnd.obj

js_static.lib(Unified_cpp_js_src15.obj) : fatal error LNK1318: Unexpected PDB error; OK (0) ''

c:/builds/slave/c-c-ntly/build/mozilla/config/rules.mk:787: recipe for target 'xul.dll' failed
mozmake.exe[4]: *** [xul.dll] Error 1318
mozmake.exe[4]: Leaving directory 'c:/builds/slave/c-c-ntly/build/objdir/toolkit/library'

Comment 11

[Edmund Wong (:ewong)]

:glandium,  might you have any ideas? 

mozilla-central post-https://hg.mozilla.org/mozilla-central/rev/a9ec72f82299 
fails to build.

Comment 12

[Edmund Wong (:ewong)]

Having spoken with IanN, he suggested I backout https://hg.mozilla.org/mozilla-central/rev/b7a2f7ff5e87, 
which I did and it builds.

Comment 13

[Edmund Wong (:ewong)]

1) pull both c-c and m-c tip
2) python mozilla\mach clobber
3) build
   - busts with '2nd type of error' log

4) python mozilla\mach clobber
5) cd mozilla
6) hg backout -r b7a2f7ff5e87
7) cd ..
8) <build>
   - completes build

Comment 14

[Edmund Wong (:ewong)]

Now with m-c's tip, you need to backout both:

https://hg.mozilla.org/mozilla-central/rev/e7c397118fd2

and

https://hg.mozilla.org/mozilla-central/rev/b7a2f7ff5e87

Comment 15

[Jorg K (CEST = GMT+2)]

My last two clobber builds of TB on 2017-02-18 and today failed with the same error. I only have 8GB RAM and 800MB are dedicated to internal graphics. Before I had no trouble building (and BTW, I have three development machines with 8GB).

At first I followed my own advice given here:
https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Simple_Thunderbird_build:

If on Windows you get link errors like "LNK1102: out of memory" or "LNK1318: Unexpected PDB error; OK (0)", try deleting the largest .PDB files before rushing out the door to buy more RAM. Clobbering (see below) will also remove those files.

That didn't help. So then I deleted all 3587 PDB files I saw following
http://stackoverflow.com/questions/174013/linker-out-of-memory-lnk1102
and the linker went through, albeit with a lot of warnings about missing PDB files.

Since according to comment #8 even 16GB are not enough, I'd like to see this rectified. BTW, only few people build TB/SM in debug mode. Many TB developers are on Linux, Richard and FRG build non-debug. And Kent James will wake up when he tries to build next time.

Comment 16

[Edmund Wong (:ewong)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=80e03ed6ab84

Comment 17

[Edmund Wong (:ewong)]

Attachment: [mozconfigs] proposed workaround patch

This is a workaround that I found to work.  I haven't tried the debug build yet, but I have pushed to try: 
https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=c3a7bcf4de70599f240111d5bfef6a0566ce3347

Comment 18

[Jorg K (CEST = GMT+2)]

Thanks, I'll compile with the patch next time I do a full debug build.

Comment 19

[:aceman]

So is this a Visual Studio compiler bug?

Comment 20

[Jorg K (CEST = GMT+2)]

I don't think so. You tell the compiler to generate 3587 PDB files and then the linker chokes on those. This might work for small programs, but not for TB or SM; I haven't tried compiling FF lately.

Comment 21

[Jorg K (CEST = GMT+2)]

I added
  CFLAGS="$CFLAGS -FS"
  CXXFLAGS="$CXXFLAGS -FS"
  mk_add_options "export COMPILE_PDB_FLAG=-Fdgenerated.pdb"
to my local config and now I can build again.

I get 722 PDB files (instead of 3587), mostly called generated.pdb.

Question: Do we need the patch if the builds work on the servers? How much RAM do they have?

Comment 22

[Edmund Wong (:ewong)]

(In reply to Jorg K (GMT+1) from comment #21)
> I added
>   CFLAGS="$CFLAGS -FS"
>   CXXFLAGS="$CXXFLAGS -FS"
>   mk_add_options "export COMPILE_PDB_FLAG=-Fdgenerated.pdb"
> to my local config and now I can build again.
> 
> I get 722 PDB files (instead of 3587), mostly called generated.pdb.
>

They should be all called generated.pdb (iiuc).
 
> Question: Do we need the patch if the builds work on the servers? How much
> RAM do they have?

I'm not too sure how much they have but if they are similar to the loaner
I use, should be around 15GB.  

That said, I'm wondering if 'how' they are built makes a difference.
Looking at the tree and not seeing everyone complaining, it has got to
be how it's built.

Comment 24

[Jorg K (CEST = GMT+2)]

Comment on attachment 8840274 [details] [diff] [review]
[mozconfigs] proposed workaround patch

I'm undecided here. The proposed solution works locally, so r+. I still doesn't fix anyone compiling locally, see bug 1342617.

I'm not sure why you don't make more noise in bug 1322703. I'm also not sure why build bot compiles don't fail.

Comment 25

[Edmund Wong (:ewong)]

(In reply to Jorg K (GMT+1) from comment #24)
> Comment on attachment 8840274 [details] [diff] [review]
> [mozconfigs] proposed workaround patch
> 
> I'm undecided here. The proposed solution works locally, so r+. I still
> doesn't fix anyone compiling locally, see bug 1342617.

To be honest, I'm also unsure here also as to whether this fixes anything
or just covers an issue with local builds vs. automation builds. 

> 
> I'm not sure why you don't make more noise in bug 1322703. I'm also not sure
> why build bot compiles don't fail.

well, I've ni glandium..  waiting for a response.

Comment 26

[Edmund Wong (:ewong)]

https://hg.mozilla.org/comm-central/rev/67c398c134dfc7b092b73b8abf2c35f40eb37ef4
Bug 1340395 - Append -FS to CFLAGS and CXXFLAGS and use generated.pdb for all the linking. r=jorgk

Comment 27

[Patrick Cloke [:clokep]]

Comment on attachment 8840274 [details] [diff] [review]
[mozconfigs] proposed workaround patch

I don't think I quite have the understanding for this one. Sorry!

Comment 28

[Jorg K (CEST = GMT+2)]

Patrick, never mind. It landed already. Note that in bug 1322703 they're discussing whether they will back that out, in which case this bug here can be backed out as well.

Comment 29

[Jorg K (CEST = GMT+2)]

Backout:
https://hg.mozilla.org/comm-central/rev/c26182d4b4d9134ad0448178bf98b10d76654795

I backed this out since bug 1322703 also got backed out.