745093 - esr-10: Win PGO builds hitting 3GB virtual address space limit, failing with: "nsapprunner.cpp(3629 : fatal error C1001: An internal error has occurred in the compiler. LINK : fatal error LNK1000: Internal error during IMAGE::BuildImage"

Status: RESOLVED WORKSFORME

(Firefox Build System :: General, defect)

Description

[Phil Ringnalda (:philor)]

+++ This bug was initially created as a clone of Bug #709193 +++

You loved it as bug 709193, when we wound up closing trunk for several days and preffing off all sort of features and landing various intrusive patches that removed big chunks of code and pulled various things out of libxul; you're going to just flat out *adore* it now that we appear to be hitting the virtual address space limit on esr-10, where we can do none of those things.

https://tbpl.mozilla.org/php/getParsedLog.php?id=10863993&tree=Mozilla-Esr10 on the landing of https://hg.mozilla.org/releases/mozilla-esr10/rev/5aeaf6c14ade from bug 679759, which certainly doesn't strike me as likely to have actually triggered it directly.

Generating code
3903 of 159877 (  2.44%) profiled functions will be compiled for speed
e:\builds\moz2_slave\m-esr10-w32\build\toolkit\xre\nsapprunner.cpp(3629) : fatal error C1001: An internal error has occurred in the compiler.
(compiler file 'F:\SP\vctools\compiler\utc\src\P2\main.c[0x10C9CFA1:0x007035F8]', line 182)
 To work around this problem, try simplifying or changing the program near the locations listed above.
Please choose the Technical Support command on the Visual C++ 
 Help menu, or open the Technical Support help file for more information

LINK : fatal error LNK1000: Internal error during IMAGE::BuildImage

  Version 8.00.50727.762

  ExceptionCode            = C0000005
  ExceptionFlags           = 00000000
  ExceptionAddress         = 10C9CFA1 (10B00000) "d:\msvs8\VC\BIN\c2.dll"
  NumberParameters         = 00000002
  ExceptionInformation[ 0] = 00000000
  ExceptionInformation[ 1] = 007035F8

CONTEXT:
  Eax    = 693AB7D0  Esp    = 0012ED78
  Ebx    = 10E84F60  Ebp    = 00000000
  Ecx    = ADCA3F40  Esi    = 00000000
  Edx    = 007035F8  Edi    = 10D0FF98
  Eip    = 10C9CFA1  EFlags = 00010246
  SegCs  = 0000001B  SegDs  = 00000023
  SegSs  = 00000023  SegEs  = 00000023
  SegFs  = 0000003B  SegGs  = 00000000
  Dr0    = 00000000  Dr3    = 00000000
  Dr1    = 00000000  Dr6    = 00000000
  Dr2    = 00000000  Dr7    = 00000000
linker max virtual size: 3019960320

I retriggered the build, which won't really prove anything either way, and I closed the tree in anticipation of more failure.

Comment 1

[Mike Hommey [:glandium]]

> you're going to just flat out *adore* it now that we appear to be hitting
> the virtual address space limit on esr-10, where we can do none of those
> things.

We should be able to backport bug 709721 and bug 709914, and the subsequent fixups.

Comment 2

[(not currently active) Ted Mielczarek]

Yeah, I suspect those are the least-harmful patches we can land on ESR. We haven't come close to hitting the linker limit on trunk since that, so that should protect ESR for its entire lifetime.

Comment 3

[Phil Ringnalda (:philor)]

Amusingly, both my retrigger and the nightly were green, so we could reopen and just keep pushing with a sword dangling over our head.

Comment 4

[Alex Keybl [:akeybl]]

We've decided to re-open the tree, see if we run into more build failures, and if so back out bug 679759 speculatively. If that still doesn't resolve the issue, we'll need to pursue backporting bug 709721 and bug 709914 as Mike/Ted suggest.

Comment 5

[Alex Keybl [:akeybl]]

Please re-nominate if this occurs again on the ESR branch. Our hope is that this was a one-time problem and does not become a more regular occurrence.

Comment 6

[alex_mayorga]

Per https://bugzilla.mozilla.org/show_bug.cgi?id=814009#c50 this might have broken Win64 Nightly.

How does one "re-nominate" if that's indeed the case?

Comment 7

[alanjstr]

Win64 is not ESR, so it is not eligible.  It looks like they're trying to tackle Win64 in bug 829738

Comment 8

[Alex Keybl [:akeybl]]

(In reply to alex_mayorga from comment #6)
> Per https://bugzilla.mozilla.org/show_bug.cgi?id=814009#c50 this might have
> broken Win64 Nightly.
> 
> How does one "re-nominate" if that's indeed the case?

To answer this question, you'd just set tracking-firefox-esr10 to ?.

Comment 9

[Phil Ringnalda (:philor)]

But the answer would still be comment 7 - this one-time failure of Win32 on the esr10 branch has nothing to do with that.