Last Comment Bug 733905 - switch OS X builds to clang
: switch OS X builds to clang
Status: RESOLVED FIXED
:
Product: Core
Classification: Components
Component: Build Config (show other bugs)
: Trunk
: x86 Mac OS X
: -- normal (vote)
: mozilla17
Assigned To: Rafael Ávila de Espíndola (:espindola) (not reading bugmail)
:
Mentors:
Depends on: 748208 762071 774462 779333 779907 787302
Blocks:
  Show dependency treegraph
 
Reported: 2012-03-07 14:10 PST by Rafael Ávila de Espíndola (:espindola) (not reading bugmail)
Modified: 2014-07-29 08:18 PDT (History)
18 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
wontfix
fixed


Attachments
switch OS X builds to clang (582 bytes, patch)
2012-03-07 15:52 PST, Rafael Ávila de Espíndola (:espindola) (not reading bugmail)
ehsan: review+
Details | Diff | Splinter Review
switch os x build (3.20 KB, patch)
2012-07-17 19:15 PDT, Rafael Ávila de Espíndola (:espindola) (not reading bugmail)
ehsan: review-
rail: feedback-
Details | Diff | Splinter Review
copying the files (4.83 KB, patch)
2012-07-18 06:17 PDT, Rafael Ávila de Espíndola (:espindola) (not reading bugmail)
no flags Details | Diff | Splinter Review
change only os x :-) (4.47 KB, patch)
2012-07-18 06:31 PDT, Rafael Ávila de Espíndola (:espindola) (not reading bugmail)
ehsan: review+
rail: feedback+
respindola: checkin+
Details | Diff | Splinter Review

Description Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-03-07 14:10:46 PST

    
Comment 1 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-03-07 15:52:00 PST
Created attachment 603901 [details] [diff] [review]
switch OS X builds to clang

I am still debugging a failure on linux 64, but I know it is related to the old linker we use on centos 5, so it should not block a clang move. A try push switching us to clang is at

https://tbpl.mozilla.org/?tree=Try&rev=af60b4debfaf

and a dummy push for comparison is at

https://tbpl.mozilla.org/?tree=Try&rev=b361228ce8f8

I am r? jp so he can proxy this as appropriate.
Comment 2 :Ehsan Akhgari 2012-03-07 16:47:18 PST
Comment on attachment 603901 [details] [diff] [review]
switch OS X builds to clang

Review of attachment 603901 [details] [diff] [review]:
-----------------------------------------------------------------

\o/
Comment 3 Jeff Muizelaar [:jrmuizel] 2012-03-07 16:55:19 PST
\o/
Comment 4 Gregory Szorc [:gps] 2012-03-07 16:59:32 PST
\o/

15% less CPU usage when compiling! https://groups.google.com/group/mozilla.dev.builds/browse_thread/thread/149725b1303f3814#
Comment 5 Justin Dolske [:Dolske] 2012-03-07 18:37:14 PST
\o/
Comment 6 JP Rosevear [:jpr] 2012-03-07 19:50:56 PST
Should we wait one week in order to go through a full 6 week release cycle before landing this?
Comment 7 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-03-08 07:48:55 PST
(In reply to JP Rosevear [:jpr] from comment #6)
> Should we wait one week in order to go through a full 6 week release cycle
> before landing this?

There were some network problems when running talos, so I pushed again 

https://tbpl.mozilla.org/?tree=Try&rev=a28ccea64488 (clang)
https://tbpl.mozilla.org/?tree=Try&rev=1f8988ce003c (gcc))

What is the best way to compare the talos results of two runs?

I think we should push as soon as possible. Waiting would force us to keep gcc-4.2 for another full release cycle, and since the switch to msvc 2010 it is the oldest compiler we have.

While it is possible that clang has a bug that we don't see on try, we know that gcc 4.2 has bugs that cause problems for using new APIs.  See bugs 682445, 681694 and 678607.

Also, if we do find a problem, reverting to gcc-4.2 is really easy (this patch changes 2 lines).

Last but not least, maintaining firefox building with an unsupported compiler is not trivial, as it is easy for people to check in non standard conformant code: http://clang.llvm.org/compatibility.html#cxx
Comment 8 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-03-08 08:39:08 PST
OK, the compare is at

http://perf.snarkfest.net/compare-talos/index.html?oldRevs=1f8988ce003c&newRev=a28ccea64488&submit=true

The "old rev" is gcc, the "new rev" is clang.
Comment 9 :Ehsan Akhgari 2012-03-08 10:29:43 PST
This is a better Talos compare: <http://perf.snarkfest.net/compare-talos/index.html?oldRevs=2f6368ca605e,3dcb40ebd487,8ef88a69f861,fca77ac7cdf9,78e56fd22f2a,75fcd465d506&newRev=a28ccea64488&tests=a11y,tdhtml,tdhtml_nochrome,a11y_paint,tdhtml_paint,tdhtml_nochrome_paint,tp4,tp4_memset,tp4_pbytes,tp4_rss,tp4_shutdown,tp4_xres,tp4m,tp4m_content_rss,tp4m_main_rss,tp4m_main_rss_nochrome,tp4m_nochrome,tp4m_shutdown,tp4m_shutdown_nochrome,tp5r,tp5r_memset,tp5r_pbytes,tp5r_rss,tp5r_shutdown,tp5r_xres,tp5r_paint,tp5r_memset_paint,tp5r_pbytes_paint,tp5r_%cpu_paint,tp5r_modlistbytes_paint,tp5r_rss_paint,tp5r_shutdown_paint,tp5r_xres_paint,dromaeo_basics,dromaeo_css,dromaeo_dom,dromaeo_jslib,dromaeo_sunspider,dromaeo_v8,tsspider,tsspider_nochrome,tsspider_paint,tsspider_nochrome_paint,v8,tgfx,tgfx_nochrome,tgfx_paint,tgfx_nochrome_paint,tscroll,tsvg,tsvg_opacity,tzoom,ts,ts_paint,ts_cold,ts_cold_generated_max,ts_cold_generated_max_shutdown,ts_cold_generated_med,ts_cold_generated_med_shutdown,ts_cold_generated_min,ts_cold_generated_min_shutdown,ts_cold_shutdown,ts_places_generated_max,ts_places_generated_max_shutdown,ts_places_generated_med,ts_places_generated_med_shutdown,ts_places_generated_min,ts_places_generated_min_shutdown,ts_shutdown,twinopen,tpaint&submit=true>

Here's what's happening.  dromaeo_css is regressed by about 4%.  dromaeo_dom by about 3%.  dromaeo_jslib by about 4%.  dromaeo_v8 by about 2%.  I'm not sure if I believe the rest of the numbers.

CCing Boris to see if he has ideas here.
Comment 10 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-03-09 07:46:21 PST
Do we have instructions on how to run these tests in a profiler?
Comment 11 Boris Zbarsky [:bz] 2012-03-09 09:44:17 PST
For dromaeo, you can run a particular subtest with shark hook by adding &shark to the url if you're using a --enable-shark build on 10.6 and shark is in programmatic mode....

The other thing worth doing is looking at the old and new numbers side-by-side to see whether it's an across-the-board regression or just a big slowdown on a few tests.
Comment 12 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-03-09 11:19:42 PST
(In reply to Boris Zbarsky (:bz) from comment #11)
> For dromaeo, you can run a particular subtest with shark hook by adding
> &shark to the url if you're using a --enable-shark build on 10.6 and shark
> is in programmatic mode....

I have done that back in firefox 4 with dromaeo.com. Is that the same version we run on talus? If not, where do I find the correct version?
 
> The other thing worth doing is looking at the old and new numbers
> side-by-side to see whether it's an across-the-board regression or just a
> big slowdown on a few tests.

In the data ehsan collected yui.html has a -8.8, so it is probably o good candidate. I will check if that reproduces with http://dromaeo.com/?yui.
Comment 13 Boris Zbarsky [:bz] 2012-03-09 12:45:11 PST
> Is that the same version we run on talus?

It's not quite identical, but close enough.  I'd start with debugging/profiling against dromaeo.com if the regression can be reproduced there.  If it can't, then you'd need to ask our infra/QA folks where to get the exact thing talos is running; I don't have that information...
Comment 14 :Ehsan Akhgari 2012-03-19 07:28:17 PDT
Rafael, did you get a chance to profile this?
Comment 15 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-03-20 07:54:47 PDT
(In reply to Ehsan Akhgari [:ehsan] from comment #14)
> Rafael, did you get a chance to profile this?

Yes, it found http://llvm.org/bugs/show_bug.cgi?id=12251

I will work on it as soon as I block on 735697 on other places work.
Comment 16 :Ehsan Akhgari 2012-04-23 19:56:54 PDT
I'm planing to rebuild clang based on revision 155417 and test the performance again.
Comment 17 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-04-24 05:52:24 PDT
(In reply to Ehsan Akhgari [:ehsan] from comment #16)
> I'm planing to rebuild clang based on revision 155417 and test the
> performance again.

Awesome!
Comment 18 :Ehsan Akhgari 2012-05-02 19:56:27 PDT
I pushed a new try build to test the new version of clang:

https://tbpl.mozilla.org/?tree=Try&rev=bf7ed18e78c9

I also wrote down the instructions for this test on the wiki: https://wiki.mozilla.org/Testing_Clang_Builds
Comment 19 :Ehsan Akhgari 2012-05-02 20:11:38 PDT
(In reply to Ehsan Akhgari [:ehsan] from comment #18)
> I pushed a new try build to test the new version of clang:
> 
> https://tbpl.mozilla.org/?tree=Try&rev=bf7ed18e78c9
> 
> I also wrote down the instructions for this test on the wiki:
> https://wiki.mozilla.org/Testing_Clang_Builds

Apparently my patch was not enough to make us use clang.  Rafael, what am I missing?
Comment 20 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-05-02 20:24:57 PDT
A port of what I have been using is at

https://tbpl.mozilla.org/?tree=Try&rev=344602b4d649

I am using "export CC=.." to avoid the libffi bug, but I think that breaks the universal build. Lets see.
Comment 21 Mozilla RelEng Bot 2012-05-02 20:30:22 PDT
Try run for bf7ed18e78c9 is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=bf7ed18e78c9
Results (out of 7 total builds):
    exception: 7
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/eakhgari@mozilla.com-bf7ed18e78c9
Comment 22 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-05-02 20:34:22 PDT
Now with talos:
https://tbpl.mozilla.org/?tree=Try&rev=112032a0b140
Comment 23 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-05-03 05:51:39 PDT
Now with -Werror disabled:
https://tbpl.mozilla.org/?tree=Try&rev=60ff764672dc
Comment 24 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-05-03 07:18:18 PDT
A dummy push for comparison is at

https://tbpl.mozilla.org/?tree=Try&rev=4674d021077d
Comment 25 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-05-03 11:38:14 PDT
It looks like we have to debug:

* mochitest-4 on linux32 debug and linux64 debug
* crashtest on linux32 debug
Comment 26 :Ehsan Akhgari 2012-05-04 11:23:03 PDT
Here's the performance comparison: http://bit.ly/Km8892

We have regressions on dromaeo_css and dromaeo_jslib.
Comment 27 :Ehsan Akhgari 2012-05-06 10:16:55 PDT
Here's the comparison between multiple runs on the try job and on the m-c base changeset for Dromaeo: http://bit.ly/IzHrHF

Looks like the regression is real.
Comment 28 :Ehsan Akhgari 2012-05-10 13:12:18 PDT
I did another try push based on the same revision with -O3, to see if it can buy back the lost performance above.  https://tbpl.mozilla.org/?tree=Try&rev=5b49fd91dc16
Comment 29 :Ehsan Akhgari 2012-05-10 14:19:13 PDT
(In reply to Ehsan Akhgari [:ehsan] from comment #28)
> I did another try push based on the same revision with -O3, to see if it can
> buy back the lost performance above. 
> https://tbpl.mozilla.org/?tree=Try&rev=5b49fd91dc16

I realized that I pushed this with -t none by mistake; here's a version with -t all: https://tbpl.mozilla.org/?tree=Try&rev=e9d0834b5d29
Comment 30 :Ehsan Akhgari 2012-05-11 16:30:36 PDT
(In reply to Ehsan Akhgari [:ehsan] from comment #29)
> (In reply to Ehsan Akhgari [:ehsan] from comment #28)
> > I did another try push based on the same revision with -O3, to see if it can
> > buy back the lost performance above. 
> > https://tbpl.mozilla.org/?tree=Try&rev=5b49fd91dc16
> 
> I realized that I pushed this with -t none by mistake; here's a version with
> -t all: https://tbpl.mozilla.org/?tree=Try&rev=e9d0834b5d29

The regressions are still there with a -O3 build, although there are a number of other improvements in other tests.
Comment 31 :Ehsan Akhgari 2012-05-11 16:30:48 PDT
(In reply to Ehsan Akhgari [:ehsan] from comment #30)
> (In reply to Ehsan Akhgari [:ehsan] from comment #29)
> > (In reply to Ehsan Akhgari [:ehsan] from comment #28)
> > > I did another try push based on the same revision with -O3, to see if it can
> > > buy back the lost performance above. 
> > > https://tbpl.mozilla.org/?tree=Try&rev=5b49fd91dc16
> > 
> > I realized that I pushed this with -t none by mistake; here's a version with
> > -t all: https://tbpl.mozilla.org/?tree=Try&rev=e9d0834b5d29
> 
> The regressions are still there with a -O3 build, although there are a
> number of other improvements in other tests.

http://bit.ly/K8LBdx
Comment 32 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-06-12 09:26:54 PDT
I have tried running dromaeo locally and it looks like we are really close:

http://dromaeo.com/?id=171519,171522

I will take a look at the "Convert pixels to canvas" benchmark.
Comment 33 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-07-05 04:02:16 PDT
I did a new performance comparison. It is at

http://bit.ly/LQqC0D

I intentionally ran the win opt tests multiple times to have an idea on how reliable they are. I wonder if I doing something wrong, but windows 7 shows regressions (like 5.16% in dromaeo_dom) without me changing anything in the windows build :-(

Ironically the linux builds show improvements in dromaeo_dom when building with clang and the os X one shows regression, which is also unexpected given that the linux one is building with a much newer gcc.
Comment 34 :Ehsan Akhgari 2012-07-05 11:30:24 PDT
This is the latest comparison: http://bit.ly/Pf6abL

This is now all pure win on Linux (non-PGO of course).  We still regress two tests (dromaeo_css and dromaeo_dom) on Mac, but we think that it's time to switch to clang now, and possibly fix these regressions in the future.

I'll write to dev-platform about this.
Comment 35 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-07-05 18:57:19 PDT
I am debugging the 64 bit regression in http://dromaeo.com/?dom-travers
Comment 36 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-07-17 19:15:39 PDT
Created attachment 643234 [details] [diff] [review]
switch os x build

This is a patch on top of m-i.

https://tbpl.mozilla.org/?tree=Try&pusher=respindola@mozilla.com

Since the current tooltool files are empty, I just made them a symlink to the clang file. Let me know if you would prefer some other organization.
Comment 37 :Ehsan Akhgari 2012-07-17 19:25:27 PDT
Comment on attachment 643234 [details] [diff] [review]
switch os x build

Symlinks are not a good idea, Windows will not be able to handle them correctly.  I would suggest copying them (hg cp).  (Yeah, it sucks, I know, but Windows sucks more...)
Comment 38 Rail Aliiev [:rail] 2012-07-17 22:06:26 PDT
Comment on attachment 643234 [details] [diff] [review]
switch os x build

I agree with Ehsan, the dumber file type you use the less troubles you have. Let's go with regular files instead symlinks.
Comment 39 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-07-18 06:17:33 PDT
Created attachment 643346 [details] [diff] [review]
copying the files
Comment 40 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-07-18 06:31:58 PDT
Created attachment 643354 [details] [diff] [review]
change only os x :-)
Comment 41 Rail Aliiev [:rail] 2012-07-18 06:48:53 PDT
Comment on attachment 643354 [details] [diff] [review]
change only os x :-)

lgtm!
Comment 42 :Ehsan Akhgari 2012-07-18 08:19:58 PDT
Comment on attachment 643354 [details] [diff] [review]
change only os x :-)

Review of attachment 643354 [details] [diff] [review]:
-----------------------------------------------------------------

\o/
Comment 43 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-07-18 08:24:11 PDT
Comment on attachment 643354 [details] [diff] [review]
change only os x :-)

Checked it in to inbound. Waiting for one build before checking it in to m-mc.
Comment 44 Ed Morley [:emorley] 2012-07-19 07:35:30 PDT
https://hg.mozilla.org/mozilla-central/rev/48127b02a061
Comment 45 Steven Michaud [:smichaud] (Retired) 2012-07-30 11:26:54 PDT
As best I can tell, this patch broke OS X builds on all versions of OS X -- even those that have clang installed and in the path.

Releng builds were fixed by bug 775509.  But that fix depends on tooltool, which isn't part of the "standard" tree.

Bug 775509 seems to be about moving tooltool to the "standard" tree.  But in the meantime I think we should have some other way to build using clang.  How about just having autoconf look for "clang" and "clang++"?  Or do older versions of clang not work properly?

(We can still have the bundled clang (built using tooltool) override whatever autoconf finds, if the bundled clang has actually been extracted.)
Comment 46 Steven Michaud [:smichaud] (Retired) 2012-07-30 11:29:41 PDT
> Bug 775509 seems to be about moving tooltool to the "standard" tree.

Oops, that's bug 768879.
Comment 47 :Ehsan Akhgari 2012-07-30 11:59:55 PDT
(In reply to comment #45)
> As best I can tell, this patch broke OS X builds on all versions of OS X --
> even those that have clang installed and in the path.

Do you set CC and CXX to clang/clang++ in your mozconfig?
Comment 48 Steven Michaud [:smichaud] (Retired) 2012-07-30 12:06:12 PDT
> Do you set CC and CXX to clang/clang++ in your mozconfig?

I've just tried that, but it doesn't work (at least not with the version of clang that comes with OS X 10.6 snd/or XCode 3.2.6).

But in any case you shouldn't have to do that.
Comment 49 :Ehsan Akhgari 2012-07-30 12:08:07 PDT
(In reply to comment #48)
> > Do you set CC and CXX to clang/clang++ in your mozconfig?
> 
> I've just tried that, but it doesn't work (at least not with the version of
> clang that comes with OS X 10.6 snd/or XCode 3.2.6).

I think those versions of clang are too old.  You need to have the open source 3.1 release at a minimum IIRC.

> But in any case you shouldn't have to do that.

Do what?
Comment 50 Steven Michaud [:smichaud] (Retired) 2012-07-30 12:12:21 PDT
(Following up comment #48)

So apparently old versions of clang *don't* work, and we'll have to wait until tooltool is moved to the standard tree.

Anyone know how to extract the bundled clang "by hand"?  I.e. to do what tooltool does, but "manually"?

> But in any case you shouldn't have to do that.

Do what?

You shouldn't have to set CC and CXX in your mozconfig file.  These values should be found "automatically" (as they always have been previously).  Though of course whatever you set in your mozconfig file should override the "automatic" values.
Comment 51 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-07-30 13:00:14 PDT
> Do what?
> 
> You shouldn't have to set CC and CXX in your mozconfig file.  These values
> should be found "automatically" (as they always have been previously). 
> Though of course whatever you set in your mozconfig file should override the
> "automatic" values.

The automatic value is to select in order
* gcc-4.2
* clang in your $PATH

which looks like a reasonable default. The mozconfigs we have in m-c are specif to how releng does its builds.

In summary, I don't think OS X is special in this case. If you have a working compiler in your PATH it will be used by default.

It *is* really bad that we don't have a publicly readable tooltool repo and I hope 768879 gets fixed, but before that you can upgrade xcode, build the open source clang, ask releng for a particular binary or, if it was me that did the upgrade you want, get it from
http://people.mozilla.org/~respindola/.
Comment 52 Steven Michaud [:smichaud] (Retired) 2012-07-30 13:29:30 PDT
> If you have a working compiler in your PATH it will be used by default.

That isn't true -- not in my tests.
Comment 53 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-07-30 13:34:08 PDT
(In reply to Steven Michaud from comment #52)
> > If you have a working compiler in your PATH it will be used by default.
> 
> That isn't true -- not in my tests.

Please check what is going wrong in config.log and report then. The autoconf macro responsible for selecting the compiler is MOZ_DEFAULT_COMPILER (and it was not modified in this bug).
Comment 54 Steven Michaud [:smichaud] (Retired) 2012-07-30 14:03:12 PDT
I'll shortly (tomorrow?) post a patch that restores the behavior you describe.

In the meantime, try yourself to build a universal build (with a simple mozconfig like I'll paste in below).  I think you'll find that it doesn't work on any version of OS X.

export CFLAGS="-g -gfull"
export CXXFLAGS="-g -gfull"
. $topsrcdir/browser/config/mozconfig
. $topsrcdir/build/macosx/universal/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-firefox-univ
mk_add_options MOZ_MAKE_FLAGS=-j4
mk_add_options AUTOCONF=autoconf213
ac_add_options --enable-application=browser
ac_add_options --disable-optimize
ac_add_options --disable-debug
ac_add_options --enable-tests
ac_add_options --disable-strip
ac_add_options --disable-install-strip
# For NSS symbols
export MOZ_DEBUG_SYMBOLS=1
ac_add_options --enable-debug-symbols="-g -gfull"
Comment 55 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-07-30 14:33:01 PDT
> . $topsrcdir/browser/config/mozconfig
> . $topsrcdir/build/macosx/universal/mozconfig

As I mentioned on comment 51 "The mozconfigs we have in m-c are specif to how releng does its builds." They set the CC and CXX to the values that releng uses and that correctly cuts off the compiler detection.

If you don't set CC or CXX configure will detect and use a compiler in your PATH.
Comment 56 Steven Michaud [:smichaud] (Retired) 2012-07-30 15:08:43 PDT
Rafael, please try to build with the mozconfig from comment #54, or with the one I'll paste in below.  I think you'll find that the builds fail on all versions of OS X.

export CFLAGS="-gdwarf-2"
export CXXFLAGS="-gdwarf-2"
. $topsrcdir/build/macosx/universal/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-firefox-nightly
mk_add_options MOZ_MAKE_FLAGS=-j4
mk_add_options AUTOCONF=autoconf213
ac_add_options --enable-application=browser
ac_add_options --enable-tests
ac_add_options --enable-codesighs
ac_add_options --disable-install-strip
ac_add_options --enable-js-diagnostics
ac_add_options --with-macbundlename-prefix=Firefox
# For NSS symbols
export MOZ_DEBUG_SYMBOLS=1
ac_add_options --enable-debug-symbols="-gdwarf-2"
# Needed to enable breakpad in application.ini
export MOZILLA_OFFICIAL=1
Comment 57 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-07-30 15:16:28 PDT
They *should* fail if you don't have the same setup as releng does. Your mozconfig includes  $topsrcdir/build/macosx/universal/mozconfig which includes build/macosx/common which sets CC and CXX to the setup that releng uses.

This was true before. The releng setup was to use /usr/bin/gcc-4.2. If you had a system without that, the build would fail. The current releng setup is to use clang from a directory in the source tree. If you don't have that it should fail.

Your build will also fail if you don't have ccache installed for example, since that is another tool that releng uses.
Comment 58 Steven Michaud [:smichaud] (Retired) 2012-07-30 15:37:20 PDT
So you're suggesting not using a mozconfig at all?

Do you use a mozconfig successfully?  If so please paste it in.
Comment 59 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-07-30 15:43:48 PDT
(In reply to Steven Michaud from comment #58)
> So you're suggesting not using a mozconfig at all?

You have two options
*) Don't use the .mozconfigs that we have in m-c
*) Set up your system to look like a releng bot.
 
> Do you use a mozconfig successfully?  If so please paste it in.

This one correctly selects the clang I have on my path (/usr/bin/clang):

ac_add_options --target=x86_64-apple-darwin10
ac_add_options --enable-application=browser

ac_add_options --enable-update-channel=${MOZ_UPDATE_CHANNEL}
ac_add_options --enable-update-packaging
ac_add_options --enable-codesighs
ac_add_options --disable-install-strip
ac_add_options --enable-signmar
ac_add_options --enable-profiling

# Nightlies only since this has a cost in performance
ac_add_options --enable-js-diagnostics

# Needed to enable breakpad in application.ini
export MOZILLA_OFFICIAL=1

export MOZ_TELEMETRY_REPORTING=1
mk_add_options MOZ_MAKE_FLAGS="-j12"

ac_add_options --with-macbundlename-prefix=Firefox

# Treat warnings as errors in directories with FAIL_ON_WARNINGS.
ac_add_options --enable-warnings-as-errors

# Package js shell.
export MOZ_PACKAGE_JSSHELL=1
Comment 60 Steven Michaud [:smichaud] (Retired) 2012-07-30 16:54:56 PDT
I find that I can build successfully, using either of my mozconfig files (from comment #54 or comment #56, if I remove the following line:

. $topsrcdir/build/macosx/universal/mozconfig

But if I do that I don't get a universal build.

So once again, Rafael, I challenge you to do a successful universal build in current trunk, without having an environment identical to that used by releng.
Comment 61 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-07-30 17:46:27 PDT
You "found" that? From comment 57:

 Your mozconfig includes  $topsrcdir/build/macosx/universal/mozconfig which includes build/macosx/common which sets CC and CXX to the setup that releng uses.

I can take you challenge tomorrow, but I think you can do it. Give it a try.
Comment 62 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-07-31 10:22:40 PDT
For universal you have (and had back in the gcc 4.2 days) to provide an explicit CC and CXX since our build system has no direct support for universal binaries and so has to be provided with a CC and CXX that produce binaries for the correct architecture.

So far this .mozconfig is working for me:

mk_add_options MOZ_BUILD_PROJECTS="i386 x86_64"
mk_add_options MOZ_UNIFY_BDATE=1
mk_add_options MOZ_POSTFLIGHT_ALL+=build/macosx/universal/flight.mk

CC=/usr/bin/clang
CXX=/usr/bin/clang++

ac_add_app_options i386 --target=i386-apple-darwin10
ac_add_app_options x86_64 --target=x86_64-apple-darwin10

if test -n "$MOZ_BUILD_APP" ; then
  TARGET_CPU=$MOZ_BUILD_APP
  HOST_CC=$CC
  HOST_CXX=$CXX
  NATIVE_CPU=`$topsrcdir/build/autoconf/config.guess | cut -f1 -d-`
  CC="$CC -arch $TARGET_CPU"
  CXX="$CXX -arch $TARGET_CPU"
  RANLIB=ranlib
  AR=ar
  AS=$CC
  LD=ld
  STRIP="strip -x -S"
  if test "$NATIVE_CPU" != "$TARGET_CPU" ; then
    CROSS_COMPILE=1
  fi
  UNIVERSAL_BINARY=1
  export CC CXX HOST_CC HOST_CXX RANLIB AR AS LD STRIP
fi

ac_add_options --enable-application=browser

ac_add_options --enable-update-channel=${MOZ_UPDATE_CHANNEL}
ac_add_options --enable-update-packaging
ac_add_options --enable-codesighs
ac_add_options --disable-install-strip
ac_add_options --enable-signmar
ac_add_options --enable-profiling
mk_add_options MOZ_MAKE_FLAGS="-j12"
Comment 63 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-07-31 10:50:59 PDT
We are still targeting 10.5 for 32 bits, so you will have to add

ac_add_options --enable-stdcxx-compat 

to that list.
Comment 64 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-07-31 11:55:12 PDT
And you also need an sdk:

ac_add_options --with-macos-sdk=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk

btw, note how this is an example where the mozconfig files in m-c would not work on my machine (xcode 4.4) even if we had decided to stay with gcc: The SDK is not installed on the same directory.
Comment 65 Steven Michaud [:smichaud] (Retired) 2012-07-31 12:26:26 PDT
Thanks, Rafael.

I'll try your mozconfig on several different versions of OS X and report back.
Comment 66 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2012-07-31 12:28:15 PDT
Please do not that at the very least you have to update

ac_add_options --with-macos-sdk=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk

to point to where you have the SDK on those systems. This path is the new default for xcode 4.4 (and 4.3 maybe?). Older versions install in /Developer.
Comment 67 Steven Michaud [:smichaud] (Retired) 2012-07-31 12:31:44 PDT
(Following up comment #65)

Also with and without the built-in clang (the one that comes with XCode, to see which platforms have a recent enough version of clang).

(In reply to comment #66)

For newer versions of XCode (those without installers), I've just been making /Developer/SDKs a soft link to the new location.

Note You need to log in before you can comment on or make changes to this bug.