403506 - Build environment gives heap allocation errors on Vista

Status: RESOLVED WORKSFORME

(Firefox Build System :: General, defect)

Description

[Marco Zehe (:MarcoZ)]

Under Windows Vista, the Mozilla build environment often has problems allocating heap memory. For example, I cannot use patch.exe at all because it cannot allocate heap memory through the cygwin/msys system.

I looked for the error message "can't  reserve memory at the cygwin base 0x71110000". I found this page: http://www.madwizard.org/extern/winavr-vista/. Under "Explanation", it says: "The problem with _spawnvp lies in an invalid parameter passed to the _access function of msvcrt.dll. This problem has always been there but only came to
light when newer msvcrt.dll versions such as the one in Vista validated its parameters."

The Mozilla build environment should be updated ASAP with a newer version to avoid build problems as more people move to Vista.

Comment 1

[:Gavin Sharp [email: [e-mail]]]

I remember vlad talking about this on IRC.

Comment 2

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

That page is talking about two separate problems -- the first one is one that's an issue with mingw's gcc (on win32 X_OK is no longer valid for directories or somesuch, I forget the details -- but the fix is to remove X_OK from the bitmask passed to access()).  That's not the problem that patch.exe is hitting.

The patch.exe issue is a mingw core issue; they made some progress there, but a real fix is significantly more complex  The main cause is Vista's address-space layout randomization (ASLR), which is used to load DLLs at different addresses on every bootup to prevent attacks that depend on knowing the address a DLL was loaded at.  Sometimes the address something's already occupying interferes with the location where mingw wants to place the cygwin heap.

The best workaround that I've found is, upon booting up, before running anything, to open a command prompt and run patch.exe.  If it works (and chances are that it will, because very few things have loaded at that point), then you're fine, and things will continue to work until you reboot.  If it doesn't work, reboot until it does.  It's painful if you restart your computer often, not so bad on a laptop that you suspend.

As far as I know, the mozilla build package has the MSYS dll that has the heap at 0x30000000, but judging by your message maybe that's not the case -- bsmedberg may want to update it.