Last Comment Bug 514495 - Build fixes for 10.6
: Build fixes for 10.6
Status: NEW
[current steps on wiki page in URL fi...
:
Product: Camino Graveyard
Classification: Graveyard
Component: General (show other bugs)
: unspecified
: All Mac OS X
: -- normal with 2 votes (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
Mentors:
http://wiki.caminobrowser.org/Develop...
Depends on: 641852
Blocks:
  Show dependency treegraph
 
Reported: 2009-09-03 12:12 PDT by Smokey Ardisson (offline for a while; not following bugs - do not email)
Modified: 2013-06-16 09:47 PDT (History)
9 users (show)
stuart.morgan+bugzilla: camino2.0-
alqahira: camino2.1-
See Also:
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
Failure excerpts from hendy's build log (17.57 KB, text/plain)
2009-09-03 12:12 PDT, Smokey Ardisson (offline for a while; not following bugs - do not email)
no flags Details
No Stripping Please, We're British (577 bytes, patch)
2010-04-23 16:15 PDT, Christopher Henderson
no flags Details | Diff | Splinter Review
Heeboo's unify fix, for safekeeping (697 bytes, patch)
2010-05-01 21:42 PDT, Smokey Ardisson (offline for a while; not following bugs - do not email)
no flags Details | Diff | Splinter Review
run-mozilla.sh hack patch (879 bytes, patch)
2010-05-15 21:35 PDT, Smokey Ardisson (offline for a while; not following bugs - do not email)
no flags Details | Diff | Splinter Review
No Stripping Please, We're British - v2 (766 bytes, patch)
2010-08-25 21:49 PDT, philippe (part-time)
no flags Details | Diff | Splinter Review
run-mozilla.sh hack patch - v2 (885 bytes, patch)
2010-10-01 19:59 PDT, philippe (part-time)
no flags Details | Diff | Splinter Review
On 64-bit native arch configs, set up cross-compilation in the mozconfig (985 bytes, patch)
2012-03-08 21:49 PST, Smokey Ardisson (offline for a while; not following bugs - do not email)
phiw2: feedback+
Details | Diff | Splinter Review

Description Smokey Ardisson (offline for a while; not following bugs - do not email) 2009-09-03 12:12:44 PDT
Created attachment 398435 [details]
Failure excerpts from hendy's build log

To build out of the box on 10.6, we'll need a few changes

0) Users need to ensure their MacPorts/etc deps have 32-bit binaries in them

1) We need mozconfig changes to force single-arch builds to be cross-compiles (but we need to make sure not to break Uni builds; this will require some playing around for out-of-the-box-for-every-build-type)

2) There are some warnings about "warning: this decimal constant is unsigned only in ISO C90" that crop up now that kill 5 files in the CaminoApp build due to -wall; we'll need to override -wall on these files, since the problem comes from a Core include in each case).
Comment 1 Smokey Ardisson (offline for a while; not following bugs - do not email) 2009-09-03 12:39:46 PDT
mozconfig bits:

CC="gcc-4.0 -arch i386"
CXX="g++-4.0 -arch i386"
ac_add_options --target=i386-apple-darwin8.0.0

HOST_CC="gcc-4.0"
HOST_CXX="g++-4.0"
RANLIB=ranlib
AR=ar
AS=$CC
LD=ld
STRIP="strip -x -S"
CROSS_COMPILE=1

Per mento in the Universal mozconfig http://mxr.mozilla.org/mozilla/source/build/macosx/universal/mozconfig, the RANLIB through STRIP lines are harmless in a native build, so we can unconditionally include them, as are the CC/CXX lines.  I think HOST_CC and HOST_CXX are also harmless (part of the cross-compile build system and not used in native builds).

That leaves the CROSS_COMPILE=1 that we'll need to case for 10.6, and the add_ac_options --target compiler bit that I don't know about (I kind of wonder if it's even needed on 10.6, but I'm not sure I want to make hendy distclean again to see).
Comment 2 Smokey Ardisson (offline for a while; not following bugs - do not email) 2009-09-03 18:27:17 PDT
So, -Wno-traditional is apparently valid for C/ObjC but not for ObjC++, according to Xcode on 10.5.  I get 2 failures and I'm out (the first two files, AppComponents.mm and SecurityDialogs.mm); the other three are .mm or .cpp, too, so back to the drawing board?
Comment 3 Smokey Ardisson (offline for a while; not following bugs - do not email) 2009-09-03 20:04:51 PDT
(In reply to comment #1)
> mozconfig bits:

> ac_add_options --target=i386-apple-darwin8.0.0

I still don't understand this one.  In a Universal build, we set these to i386-apple-darwin$DARWIN_VERSION for Intel / powerpc-apple-darwin$DARWIN_VERSION
for PowerPC, where $DARWIN_VERSION is `uname -r`.  In single-arch builds, it gets set to the same thing, depending on your arch.

mento mentions in bug 322578 comment 1:

> Configured this way, it is possible to test a cross
> build where the host and target are both the same architecture.  The key is
> making sure that the target's identifier doesn't match the host's exactly.  The
> difference between darwin8.3.0 vs. darwin8.0.0 is sufficient for this purpose
> and is otherwise inconsequential.

I guess because we're doing a cross build on 10.6 we need to set it there to confuse whatever was depending on it not being identical to the host?
Comment 4 Nomis101 2009-09-05 04:15:21 PDT
(In reply to comment #0)
> 0) Users need to ensure their MacPorts/etc deps have 32-bit binaries in them

You don't need 32-bit binaries, you can build with x86_64 binaries and the mozconfig from Bug 477945.
I've build Thunderbird 3.1a1pre just fine with the x86_64 binaries from MacPorts on Snow Leopard (in i386).
Comment 5 Smokey Ardisson (offline for a while; not following bugs - do not email) 2009-09-18 16:41:39 PDT
Just documenting here for the record, since apparently we've only ever discussed this part on irc: until we fix comment 0 point 2, the way to avoid the build dying there is to turn off -Werror aka GCC_TREAT_WARNINGS_AS_ERRORS

Easiest way to do that is to comment out or set to NO the GCC_TREAT_WARNINGS_AS_ERRORS = YES
lines in camino/config/Camino.xcconfig and camino/config/PrefPane.xcconfig (there may be no problem files in the prefPanes, but hendy's test builds only got as far as CaminoApp before dying from the errors).
Comment 6 Smokey Ardisson (offline for a while; not following bugs - do not email) 2009-09-23 23:20:55 PDT
Not really something we'd actually block on, I don't think, but it would be good if we got the "build-out-of-the-box-on-10.6" stuff working for 2.0; flagging so that it stays on the radar.
Comment 7 Smokey Ardisson (offline for a while; not following bugs - do not email) 2009-10-03 18:22:19 PDT
> 0) Users need to ensure their MacPorts/etc deps have 32-bit binaries in them

For reference, that's:

[4:48pm] krmathis: "sudo port install libIDL +universal"
Comment 8 Smokey Ardisson (offline for a while; not following bugs - do not email) 2009-10-20 21:00:12 PDT
When Stuart was building the other night, he ran into bug 513747 comment 13 (unsure how Kai managed to avoid it in his builds); the hack in bug 513747 comment 14 allowed him to get past signing the NSS chk files.
Comment 9 Stuart Morgan 2009-10-26 13:11:57 PDT
Comment 8 makes this extra messy :P 10.5 will have to be the newest officially supported config for 2.0, and we'll work on 10.6 for trunk.
Comment 10 Andrew Thompson 2009-11-25 07:20:58 PST
Can someone post step by step instructions on getting a build to work on 10.6. What's above is fragmented and hard to follow.

I did this: sudo port install libIDL +universal, but haven't made any mozconfig changes yet. What's next?

Apologies for spamming this bug... but a reply here would be much more valuable than IRC. I am happy to write up instructions, edit a wiki, etc. as needed.
Comment 11 Smokey Ardisson (offline for a while; not following bugs - do not email) 2009-11-25 19:55:41 PST
For a 10.6 build:

0) Ensure universal dependencies by
sudo port install libIDL +universal

1) In your own mozconfig, include all of 

CC="gcc-4.0 -arch i386"
CXX="g++-4.0 -arch i386"
ac_add_options --target=i386-apple-darwin8.0.0

HOST_CC="gcc-4.0"
HOST_CXX="g++-4.0"
RANLIB=ranlib
AR=ar
AS=$CC
LD=ld
STRIP="strip -x -S"
CROSS_COMPILE=1

in addition to whatever else you include.

3) Comment out (or set to NO) the
GCC_TREAT_WARNINGS_AS_ERRORS = YES
lines in camino/config/Camino.xcconfig and camino/config/PrefPane.xcconfig

4) If the build fails in "shlibsign" (part of the NSS build), apply the hack in bug 513747 comment 14 (not everyone seems to hit this, for whatever reason).

I've added a note to the whiteboard to point to this comment; ideally we'll get this in the tree and not need anything in the wiki at all (though the wiki does point to this bug for people wanting to build on 10.6).
Comment 12 Andrew Thompson 2009-11-30 20:09:13 PST
I made the changes above, but I am still failing early in the build

In file included from <root>/lizard/mozilla/nsprpub/config/nsinstall.c:56:
/Developer/SDKs/MacOSX10.4u.sdk/usr/include/stdarg.h:4:25: error: stdarg.h: No such file or directory
<root>/lizard/mozilla/nsprpub/config/nsinstall.c: In function ‘fail’:
<root>/lizard/mozilla/nsprpub/config/nsinstall.c:436: warning: implicit declaration of function ‘va_start’
<root>/lizard/mozilla/nsprpub/config/nsinstall.c:438: warning: implicit declaration of function ‘va_end’

At line 4 of that stdarg.h, there's a #include_next directive for stdarg.h, so I guess it should be searching for the next instance of the file. How do I debug this?
Comment 13 Stuart Morgan 2009-11-30 21:39:34 PST
Did you do a completely clean build? Trying to re-use a tree that was previously built pre-10.6 didn't work for me.
Comment 14 Andrew Thompson 2009-12-01 04:55:52 PST
Well, I did precisely one make -f client.mk in that tree before I did the +universal

sudo port install libIDL +universal

After that I did a 'sudo port update libIDL +universal' and told it to force the variant and it rebuilt everything libIDL depends on.

make -f client.mk clean, followed by another build didn't help.

Well, I suppose I could blow away the entire tree next...
Comment 15 Andrew Thompson 2009-12-01 21:01:02 PST
Deleting my entire tree and repulling from CVS has solved the problem. It now builds (and I didn't need to do step 4 above, FWIW)
Comment 16 Smokey Ardisson (offline for a while; not following bugs - do not email) 2009-12-01 21:19:56 PST
Maybe all you needed was a distclean, but at that point nuking and pulling a new tree is just as easy.

philippe noted tonight that he also didn't need step 4, and that when building with the 10.5 sdk, the errors that require step 3 didn't show up; go figure :P
Comment 17 Smokey Ardisson (offline for a while; not following bugs - do not email) 2010-03-17 16:35:37 PDT
Flagging this to keep it on the radar for 2.1, since bug 513747 is supposedly wanted-1.9.2+, which means we'd have a chance of at least getting that Gecko bug out of the way.

philippe told me that building 1.9.2 gets rid of the strange "warning: this decimal constant is unsigned only in ISO C90" errors, so if bug 513747 lands, we should be good to go with just fixing our default mozconfig to do the right thing out-of-the-box in 2.1-1.9.2 builds.
Comment 18 Smokey Ardisson (offline for a while; not following bugs - do not email) 2010-04-20 21:49:14 PDT
Apparently the very latest Xcode on 10.6 (3.2.2?) fails packaging Sparkle; Xcode has, entirely on its own, decided to do some sort of nib-stripping to every nib it packages, and SUStatus.nib fails the new Xcode-self-invented buildstep:

StripNIB SUStatus.nib
    cd /Users/phiw/camino-build/build-1.9.2-gcc42/camino/sparkle
    /Developer/usr/bin/ibtool --strip /Users/phiw/camino-build/build-1.9.2-gcc42/camino/sparkle/build/Release/Sparkle.framework/Versions/A/Resources/SUStatus.nib --output-format human-readable-text /Users/phiw/camino-build/build-1.9.2-gcc42/camino/sparkle/SUStatus.nib

/* com.apple.ibtool.errors */
/Users/phiw/camino-build/build-1.9.2-gcc42/camino/sparkle/SUStatus.nib: error: The document "SUStatus.nib" cannot be stripped because it isn't deployable.
The document "SUStatus.nib" cannot be stripped because it isn't deployable.

Although the build keeps on going to the bitter end (yay Xcode :/ ), it does eventually fail in a cascade of mess.
Comment 19 philippe (part-time) 2010-04-21 01:52:12 PDT
(In reply to comment #18)

Opening the offending NIB in IB (3.2.2) and saving has allowed 3 successful builds (once in a clean $OBJDIR, 2 times after a distclean).
Comment 20 Smokey Ardisson (offline for a while; not following bugs - do not email) 2010-04-21 21:51:13 PDT
A) Use of run-mozilla.sh seems to have expanded into something in js/ such that the hack mentioned in comment 8 is no longer sufficient :/

However, in Heeboo's testing, if you prepend /usr/lib: to the Darwin DYLD_LIBRARY_PATH in run-mozilla.sh, 1) it appears to work and 2) replaces the hack in sign.sh.

----

B) osacompile on 10.6 is very, very lame.  Although the manpage says that it still accepts (and only accepts) 'ppc' and 'i386' as arguments to -arch, osacompile seems to ignore that altogether and create a 3-way fat binary with: x86_64 i386 ppc7400

Note that's not 'ppc'; it's 'ppc7400'.  mento built support into unify for handling fat files (thinning them, and then unifying them); unfortunately, it assumes the arch(es) are 'ppc' and 'i386', not 'ppc7400' (or x86_64), so thinning fails.

You can 'fix' this by changing 'ppc' to 'ppc7400' in http://mxr.mozilla.org/mozilla1.9.2/source/build/macosx/universal/unify#811, but that causes other fat files (JEP) to fail, since they have arch ppc.

The required fix there is to make unify try 'ppc' and, if that fails, try 'ppc7400' before failing entirely on the thinning process.
Comment 21 Christopher Henderson 2010-04-23 16:15:24 PDT
Created attachment 441205 [details] [diff] [review]
No Stripping Please, We're British

I've found that disabling nib stripping in the Sparkle config file fixes the build issues I've had with it.
Comment 22 Smokey Ardisson (offline for a while; not following bugs - do not email) 2010-04-24 18:48:17 PDT
(In reply to comment #21)
> Created an attachment (id=441205) [details]
> No Stripping Please, We're British
> 
> I've found that disabling nib stripping in the Sparkle config file fixes the
> build issues I've had with it.

"Stripping a NIB reduces its size and makes it uneditable." <-- so before we start doing production builds on 10.6, we'll want that in all of our projects' xcconfigs (or at least all of them that have localizable nibs).
Comment 23 Smokey Ardisson (offline for a while; not following bugs - do not email) 2010-05-01 21:42:13 PDT
Created attachment 442969 [details] [diff] [review]
Heeboo's unify fix, for safekeeping

Before I forget, here's Heeboo's patch to fix unify to correctly thin ppc7400 binaries that osacompile produces on 10.6.

I'd really like to get someone on 10.6 to try some other things wrt osacompile, and, if we're not successful (or even if we are), file a rdar:// against osacompile (at the very least, Apple should correct the lying manpage).  Then we can work on getting Heeboo's patch in the tree if it's still needed.
Comment 24 Smokey Ardisson (offline for a while; not following bugs - do not email) 2010-05-15 21:25:01 PDT
Also, starting(?) with the crash-landing of lorentz into 1.9.2, you'll need to add 

ac_add_options --disable-crashreporter

to your mozconfig.  The net effect of that is to disable about:crashes, which can be reached via chrome://global/content/crashes.xhtml if you need to hack on about:crashes ;)
Comment 25 Smokey Ardisson (offline for a while; not following bugs - do not email) 2010-05-15 21:35:31 PDT
Created attachment 445579 [details] [diff] [review]
run-mozilla.sh hack patch

(In reply to comment #20)
> A) Use of run-mozilla.sh seems to have expanded into something in js/ such that
> the hack mentioned in comment 8 is no longer sufficient :/
> 
> However, in Heeboo's testing, if you prepend /usr/lib: to the Darwin
> DYLD_LIBRARY_PATH in run-mozilla.sh, 1) it appears to work and 2) replaces the
> hack in sign.sh.

This patch implements this; it's here in patch form so that we can just point people to a series of patches to apply.
Comment 26 Smokey Ardisson (offline for a while; not following bugs - do not email) 2010-05-15 22:12:22 PDT
OK, I've consolidated all of the current information on the wiki on http://wiki.caminobrowser.org/Development:Building:Mac_OS_X_10.6 (which is a supplement referenced by the 1.9.2 instructions, http://wiki.caminobrowser.org/Development:Building:Mozilla_1.9.2_Branch ).

Moving the current steps to the wiki allows us to keep them up-to-date and in a friendly fashion (and avoids making people parse this bug!).

We should track new/additional changes here, of course, but once we have a fix, we should update the wiki page (and the 1.9.2 instructions, if we have to add a new section to the 10.6 page, since the 1.9.2 instructions say "see 10.6#foo" in all the relevant places).
Comment 27 philippe (part-time) 2010-07-29 22:51:41 PDT
(In reply to comment #21)
> Created attachment 441205 [details] [diff] [review]
> No Stripping Please, We're British
> 
> I've found that disabling nib stripping in the Sparkle config file fixes the
> build issues I've had with it.

Xcode 3.2.3 has this possibly fixed. 2 builds so far without the SUStatus.nib failure.
Comment 28 philippe (part-time) 2010-08-25 21:47:09 PDT
(In reply to comment #27) 
> Xcode 3.2.3 has this possibly fixed. 2 builds so far without the SUStatus.nib
> failure.

Or not, I ran into this error today after reverting some patches out of my tree, followed by a distclean.
Comment 29 philippe (part-time) 2010-08-25 21:49:58 PDT
Created attachment 469342 [details] [diff] [review]
No Stripping Please, We're British - v2

Updated patch for the SUstatusNIB issue for recentish changes in Sparkle
Comment 30 philippe (part-time) 2010-10-01 19:59:21 PDT
Created attachment 480356 [details] [diff] [review]
run-mozilla.sh hack patch - v2

update patch to account for
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/51149d349784
Comment 31 Smokey Ardisson (offline for a while; not following bugs - do not email) 2011-03-22 18:34:20 PDT
Comment on attachment 469342 [details] [diff] [review]
No Stripping Please, We're British - v2

The Sparkle patch is no longer needed; we have a more comprehensive anti-stripping patch in the tree now that bug 641852 has landed.
Comment 32 Smokey Ardisson (offline for a while; not following bugs - do not email) 2011-03-22 18:52:11 PDT
So at this point, I don't think there's anything else we can do here, unless someone wants to write a harness or wrapper that we change our instructions to tell people to call after pulling the first time (instead of make -f client.mk); said harness/wrapper could apply the two remaining patches and munge the default mozconfig *and* handle merge changes to the three files.  (Or, if someone writes the wrapper, we could case the mozconfig like we used to do on 1.8, and land that.)

We are, at least, down from mozconfig and two required patches (and one universal-only) to mozconfig and one required patch (and one universal-only), so we made some progress in the 2.1 cycle :P
Comment 33 Smokey Ardisson (offline for a while; not following bugs - do not email) 2012-01-19 23:37:19 PST
(In reply to comment #17)
> lands, we should be good to go with just fixing our default mozconfig to do
> the right thing out-of-the-box in 2.1-1.9.2 builds.

mento cased the mozconfig before http://hg.mozilla.org/camino/annotate/fa1cf8bc7309/config/mozconfig using uname -p, but that won't work here, because uname -p keeps lying all the way into 10.7.

We could use uname -r on 10.7, but for 10.6 we still need to know whether the default/native architecture is i386 or x86_64, and the only way I think you can do that is compile a test binary and read back its architecture :-(
Comment 34 Smokey Ardisson (offline for a while; not following bugs - do not email) 2012-03-08 21:49:07 PST
Created attachment 604303 [details] [diff] [review]
On 64-bit native arch configs, set up cross-compilation in the mozconfig

By virtue of having two hours of waiting for Gecko to clone for 2.1.2rc tonight, I had some time on my hands, and because of a comment from DerekS, I decided to poke the sad .mozconfig situation.

This monstrosity of a case statement should do the right thing, with the following caveats:

1) It's designed specifically for 10.6 right now (but I think it'll work on 10.7 if you use the alternate [symlink] method of letting Gecko find the compiler).

2) I have no idea what it might do to real Universal builds on these machines.

This needs to be tested using a build that uses a fresh objdir so that nothing left over from a prior build's configure-and-mozconfig-parsing could contaminate the results.

I've verified (in a shell script) the ugly case statement's command returns the desired results on 10.5.8 Intel, 10.4.11 Intel, and 10.5.8 PPC, and philippe has checked on 10.6 and 10.7 on 64-bit-native-arch machines, so I'm pretty confident that the command itself will work on all supported buildhosts; I hope it will be the same inside the mozconfig.

This also disables the Gecko crash reporter on 10.6 and 10.7, which is unrelated to the native-arch mess.
Comment 35 Smokey Ardisson (offline for a while; not following bugs - do not email) 2012-03-08 21:56:48 PST
Oh, two other notes:

1) I'd like to replace "gcc" with "$CC" but I don't know if that will work (whether CC has been initially populated yet, and whether it does the right thing on 10.7 if it has been initially populated).

2) When mento did his case thing for the SDK, etc., before, he used the full path to uname; that's probably good practice (and is easy enough to do for the uname case statement), and assuming lipo always lives in /usr/bin, I guess we can do that for all the non-$CC binaries in the ugly-command case statement.
Comment 36 philippe (part-time) 2012-03-08 23:25:49 PST
Comment on attachment 604303 [details] [diff] [review]
On 64-bit native arch configs, set up cross-compilation in the mozconfig

With a plain vanilla .mozconfig, a fresh $OBJDIR, 10.6
** BUILD SUCCEEDED *

(I can't test on 10.7 atm)
Comment 37 Chris Lawson (gone) 2013-06-16 09:47:45 PDT
Comment on attachment 604303 [details] [diff] [review]
On 64-bit native arch configs, set up cross-compilation in the mozconfig

Review of attachment 604303 [details] [diff] [review]:
-----------------------------------------------------------------

Clearing review flag; I'm not really around any more.

Note You need to log in before you can comment on or make changes to this bug.