Open Bug 419344 Opened 13 years ago Updated 3 years ago

enable profile-guided optimization on mac

Categories

(Firefox Build System :: General, defect, P3)

x86
macOS
defect

Tracking

(Not tracked)

People

(Reporter: ted, Unassigned)

References

Details

(Keywords: perf)

Promised joduinn I would file this, and forgot.  This is blocked on a few things, I don't have bug numbers handy, but I know it at least needs XCode 3.
Depends on: 419348
Depends on: 417045
Tweaking summary, as bug 417045 will give us new machines rather than upgrading existing ones.
Summary: enable profile-guided optimization on bm-xserve08 → enable profile-guided optimization on mac
Keywords: perf
People have been sounding much less keen on this. Should we close this out ?
Eh?  Who is less keen on turning on PGO and why?
Nobody is less keen.  I would like it to be a blocker for Firefox 3.  
I'm less keen. Apple's gcc kind of sucks, so building with profiling currently hangs. We'd have to narrow down a list of modules to enable it on, and I haven't had time to do that yet. We're running out of time in the beta cycle to do this, so unless someone else wants to figure it out, it's probably going to fall by the wayside.
Ted: Do you have any idea how many modules hang with the new gcc? 

RobSayer: from your comments in bug#417045, it seems you got this to work. Can you help? 

Note: we are upgrading macs from 10.4 to 10.5 *because* of this request to compile with PGO. If we are not going to compile with PGO on mac, I'd like to revisit the 10.4->10.5 upgrades work being done in bug#417045. One way or another, we need to decide quickly.

Note: There's also been reliability problems with Talos machines working fine on 10.4, but failing intermittently on 10.5. Dont know details, but Alice would know more.
Priority: -- → P1
> Rob: from your comments in bug#417045, it seems you got this to work. Can
> you help? 

I got it to work for SpiderMonkey only. 

> Note: we are upgrading macs from 10.4 to 10.5 *because* of this request to compile with PGO

It also fixed compliance bugs and seemed to generate faster code based on my testing, so it is worth doing even if we don't use PGO.
(In reply to comment #7)
> > Rob: from your comments in bug#417045, it seems you got this to work. Can
> > you help? 
> 
> I got it to work for SpiderMonkey only. 

Ah, interesting. Has *anyone* ever got compile with PGO on mac to work? Or is Ted "boldly going where noone has gone before"? 


> > Note: we are upgrading macs from 10.4 to 10.5 *because* of this request to compile with PGO
> It also fixed compliance bugs and seemed to generate faster code based on my
> testing, so it is worth doing even if we don't use PGO.
ok, good to know. Any ballpark of how much faster the code is with 10.5 yet without PGO?
Some have said at Mozillazine they have got PGO on PPC Mac's to compile.

http://forums.mozillazine.org/viewtopic.php?t=630298&postdays=0&postorder=asc&postsperpage=15&start=30

I have also seen improvements in using GCC 4.2 from Apple.  I can't get the current trunk to compile on 4.2 now though, so can't come up with some accurate results.
(In reply to comment #6)
> Ted: Do you have any idea how many modules hang with the new gcc? 

No, because that would require me to enable it on a case-by-case basis, which I haven't done yet.

(In reply to comment #8)
> Ah, interesting. Has *anyone* ever got compile with PGO on mac to work? Or is
> Ted "boldly going where noone has gone before"? 

AFAIK, nobody has done a full PGO build on Mac yet. Rob did builds with a PGO'ed Spidermonkey, but that's it.
Summary of quick call with damon, rsayer, and ted:

1) We're should enable PGO compiling for SpiderMonday *only*. We're going to compile all other modules without PGO.

2) We're staying with the gcc compiler on mac until we figure out bug#403224. 

3) bm-xserve16 is up and running with 10.5 on it. nthomas and ted are using it right now.
Component: Release Engineering → Release Engineering: Future
Priority: P1 → P3
QA Contact: build → release
fyi: bm-xserve16 is being recycled and used for moz2 builds. Once we solve the dep.bug, we can come back and try again.
Mass move of bugs from Release Engineering:Future -> Release Engineering. See
http://coop.deadsquid.com/2010/02/kiss-the-future-goodbye/ for more details.
Component: Release Engineering: Future → Release Engineering
Component: Release Engineering → Build Config
Product: mozilla.org → Core
QA Contact: release → build-config
Version: other → Trunk
Please get this working somewhere before we automate it.
Using clang 3.2 seems to correctly profile the application and put the GCDA's in the proper folders. 

Using 3.1 I did a coverage build, created a list of the GCNO locations, built firefox with profiling, and moved the incorrectly placed GCDA's in their correct place using the names/locations of the GCNO files.
Hi, I could not get a PGO build to work on OS X Yosemite, Instead I get the following error:

make[1]: Leaving directory '/Volumes/hd2/src/mozilla-release'
make -C /Volumes/hd2/src/mozilla-release/build_directory package MOZ_PGO_INSTRUMENTED=1 MOZ_INTERNAL_SIGNING_FORMAT= MOZ_EXTERNAL_SIGNING_FORMAT=
make[1]: Entering directory '/Volumes/hd2/src/mozilla-release/build_directory'
make[1]: *** No rule to make target 'package'.  Stop.
make[1]: Leaving directory '/Volumes/hd2/src/mozilla-release/build_directory'
client.mk:233: recipe for target 'profiledbuild' failed
make: *** [profiledbuild] Error 2
 
.mozconfig:
# Import the stock config for building the browser (Firefox)
. $topsrcdir/browser/config/mozconfig
. $topsrcdir/build/macosx/mozconfig.common
. $topsrcdir/build/macosx/universal/mozconfig



#mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/build_directory
mk_add_options MOZ_OBJDIR=/Volumes/hd2/src/mozilla-release/build_directory/
mk_add_options MOZ_MAKE_FLAGS="-j5 "
mk_add_options AUTOCONF=autoconf213


ac_add_options --enable-application=browser
ac_add_options --disable-debug
ac_add_options --disable-crashreporter
ac_add_options --disable-debug-symbols

ac_add_options --with-macos-sdk=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk
ac_add_options --with-unify-dist=../i386/dist

ac_add_options --enable-install-strip
ac_add_options --enable-strip
ac_add_options --enable-xterm-updates

ac_add_options --enable-llvm-hacks
ac_add_options --disable-jemalloc
ac_add_options --enable-static
ac_add_options --enable-static-libs
ac_add_options --disable-stdcxx-compat
ac_add_options --enable-media-plugins
ac_add_options --with-google-oauth-api-keyfile=/Users/dave/scripts/firefox_google_OAuth
mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) $(MOZ_OBJDIR)/_profile/pgo/profileserver.py'
mk_add_options PROFILE_GEN_SCRIPT_PAUSE='$(PYTHON) $(MOZ_OBJDIR)/_profile/pgo/stopmsg.py'
ac_add_options --enable-profile-guided-optimization
ac_add_options --disable-tests
You're doing a universal build, and the build system doesn't know how to do a universal PGO build.
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.