The default bug view has changed. See this FAQ.
Bug 559964 (gcc4.5)

switch to gcc-4.5 and enable PGO for linux builds

RESOLVED FIXED

Status

Release Engineering
Other
P4
normal
RESOLVED FIXED
7 years ago
3 years ago

People

(Reporter: (dormant account), Assigned: glandium)

Tracking

(Blocks: 1 bug)

Dependency tree / graph

Firefox Tracking Flags

(firefox6-, blocking2.0 -)

Details

(Whiteboard: [puppet][buildslaves] nominated in comment 163)

Attachments

(7 attachments, 29 obsolete attachments)

5.77 KB, patch
Details | Diff | Splinter Review
7.80 KB, patch
bear
: review+
Details | Diff | Splinter Review
2.32 KB, patch
bhearsum
: review+
Details | Diff | Splinter Review
1.71 KB, patch
Details | Diff | Splinter Review
12.88 KB, patch
catlee
: review+
Details | Diff | Splinter Review
21.83 KB, patch
catlee
: review+
Details | Diff | Splinter Review
3.26 KB, text/plain
ted
: review+
Details
(Reporter)

Description

7 years ago
PGO provides a huge(30%) startup improvement on Linux. It's the biggest easy win available right now.
A few questions:

- Any special instructions for compiling gcc 4.4?  We had some patches for gcc4.3 when we compiled it.

- Which branches should this be enabled for?

- Any known compatibility issues between 4.3 and 4.4?  ABI changes?  libstdc++ changes?-

- What are the impacts of users of the binaries?  Will they need the gcc 4.4 and associated libstdc++ libraries installed?
(Reporter)

Comment 2

7 years ago
(In reply to comment #1)
> A few questions:
> 
> - Any special instructions for compiling gcc 4.4?  We had some patches for
> gcc4.3 when we compiled it.

No such patches for 4.4.
> 
> - Which branches should this be enabled for?

trunk.

> 
> - Any known compatibility issues between 4.3 and 4.4?  ABI changes?  libstdc++
> changes?-

Libstdc++ is gcc-dependent. So users will have to be using a distributing with gcc4.4

> - What are the impacts of users of the binaries?  Will they need the gcc 4.4
> and associated libstdc++ libraries installed?

they need the 4.4 libstdc++
I don't think we actually *do* require a newer version of libstdc++: it's still libstdc++.so.6, and the ABI hasn't really changed since 4.2. If we don't run on Ubuntu 08.04 LTS, that would be a problem.
(Assignee)

Comment 4

7 years ago
The only change I'm aware of in 4.4 is the va_list ABI on ARM, but mozilla has no externally available API using it, I believe.

Please note, as I said in bug 418866, and since mozilla support for linux x86-64 is being added, that gcc 4.4 fails to generate working coverage builds (1st pass) on x86-64, which makes it impossible to build pgo. I haven't tried with 4.5 yet.
(Reporter)

Comment 5

7 years ago
(In reply to comment #3)
> I don't think we actually *do* require a newer version of libstdc++: it's still
> libstdc++.so.6, and the ABI hasn't really changed since 4.2. If we don't run on
> Ubuntu 08.04 LTS, that would be a problem.

You are right. I just confirmed 4.4-produced binaries run great on 4.2-based ubuntu LTS. So there are no abi concerns from me anymore. 

So how soon can we get a linux PGO tinderbox?
(In reply to comment #5)
> (In reply to comment #3)
> > I don't think we actually *do* require a newer version of libstdc++: it's still
> > libstdc++.so.6, and the ABI hasn't really changed since 4.2. If we don't run on
> > Ubuntu 08.04 LTS, that would be a problem.
> 
> You are right. I just confirmed 4.4-produced binaries run great on 4.2-based
> ubuntu LTS. So there are no abi concerns from me anymore. 
> 
> So how soon can we get a linux PGO tinderbox?

Will this replace the existing linux build, or be in addition to it?

If you want to change how the existing build is done, I think this merits some newsgroup discussion first.
(Reporter)

Comment 7

7 years ago
(In reply to comment #6)
> Will this replace the existing linux build, or be in addition to it?
> 
> If you want to change how the existing build is done, I think this merits some
> newsgroup discussion first.

I would like to replace the current build as it would be insufferably slow compared to a PGO one. I sent a notice to http://groups.google.com/group/mozilla.dev.platform/browse_thread/thread/4037844abc6ef1f5#

I don't see why anyone would disagree with a faster firefox, but lets proceed on this bug in parallel to the discussion. In the worst case we can settle for an additional build.
(Assignee)

Comment 8

7 years ago
(In reply to comment #7)
> I would like to replace the current build as it would be insufferably slow
> compared to a PGO one.

FWIW, I didn't get any really noticeable improvement with pgo builds, even on startup time, though i haven't formally verified the latter, and I have a SSD, so i cheat for startup. But that would suggest that there is no cpu-bound improvement in your faster startups, which in turn suggests there might be better ways to get these improvements (such as your reordering work).

But benchmarks such as sunspider didn't end up running significantly faster even when running it during the profiling phase. Is it worth doubling the build time ?
(Reporter)

Comment 9

7 years ago
(In reply to comment #8)
> (In reply to comment #7)
> > I would like to replace the current build as it would be insufferably slow
> > compared to a PGO one.
> 
> FWIW, I didn't get any really noticeable improvement with pgo builds, even on
> startup time, though i haven't formally verified the latter, and I have a SSD,
> so i cheat for startup. But that would suggest that there is no cpu-bound
> improvement in your faster startups, which in turn suggests there might be
> better ways to get these improvements (such as your reordering work).

Actually the good numbers produced by pgo in http://blog.mozilla.com/tglek/2010/04/12/squeezing-every-last-bit-of-performance-out-of-the-linux-toolchain/ have little to do with cpu time. -freorder-blocks-and-partition option changed the layout of the binary for the better. Any minor cpu-time benefits are just icing on my startup cake.
 
> 
> But benchmarks such as sunspider didn't end up running significantly faster
> even when running it during the profiling phase. Is it worth doubling the build
> time ?

For a 30-40% improvement in startup time, yes.
(Assignee)

Comment 10

7 years ago
(In reply to comment #9)
> For a 30-40% improvement in startup time, yes.

Even when you can get the same improvement with tools that don't require to double the build time ?
With most of our Linux builds happening on very fast hardware I don't think we should overly concern ourselves with doubling the build time. The last mozilla-central nightly on Linux took 15 minutes to compile. Even at 30 minutes to compile, Linux is not going to be our long-pole platform.
(Reporter)

Updated

7 years ago
Depends on: 561236
(Reporter)

Comment 12

7 years ago
After Benjamin grumbled about the 4.4 switch, I figured out what needs to be done to do PGO builds on x86/x86_64 with gcc 4.5 such that the resulting binaries run fine on older distributions.
So lets get GCC 4.5.0 installed and pgo turned on.
Summary: Install GCC 4.4 on linux VMs to enable PGO on Linux → Install GCC 4.5 on linux VMs to enable PGO on Linux
From Taras on IRC:

- Build with same switches as we did gcc 4.3.3: https://wiki.mozilla.org/ReferencePlatforms/Linux-CentOS-5.0#GCC_4.3.3

- No need to keep gcc headers around; just need binaries and libs (whatever 'make install' creates)
You really want to switch our builds to a compiler that was released not quite two weeks ago? Brave!
(Reporter)

Comment 15

7 years ago
(In reply to comment #14)
> You really want to switch our builds to a compiler that was released not quite
> two weeks ago? Brave!

Yup. This way we avoid another compiler change in the near future. This way we have a much better chance of having any fresh issues fixed quickly by gcc guys if they arise.
Assignee: nobody → rail
Status: NEW → ASSIGNED
Priority: -- → P2
Hardware: x86 → All
Created attachment 443099 [details]
compile-gcc-4.5.0.sh

Attaching the current GCC 4.5.0 compile script which prepares Puppet friendly tarball.

GCC 4.5 requires new version of gmp, mpfr and mpc libraries. To avoid any possible conflicts with te current toolchain I installed all of tools with --prefix=/tools/gcc-4.5.0 prefix and add wrappers for gcc/g++ with injected LD_LIBRARY_PATH. Any ideas how to avoid this hack without touching /etc/ld.so.conf and setting LD_LIBRARY_PATH?

Normal build (using wrappers) runs fine, but without the new libstdc++.so.6.0.14 linked to libstdc++.so.6.0 fails with:

./firefox-bin: /usr/lib32/libstdc++.so.6: version `GLIBCXX_3.4.14' not found (required by ./libxul.so)

Taras, is there any ETA for updated profiledbuild target for linux? IIRC it hasn't landed yet.
(Reporter)

Comment 17

7 years ago
(In reply to comment #16)
> Created an attachment (id=443099) [details]
> compile-gcc-4.5.0.sh
> 
> Attaching the current GCC 4.5.0 compile script which prepares Puppet friendly
> tarball.
> 
> GCC 4.5 requires new version of gmp, mpfr and mpc libraries. To avoid any
> possible conflicts with te current toolchain I installed all of tools with
> --prefix=/tools/gcc-4.5.0 prefix and add wrappers for gcc/g++ with injected
> LD_LIBRARY_PATH. Any ideas how to avoid this hack without touching
> /etc/ld.so.conf and setting LD_LIBRARY_PATH?
> 
> Normal build (using wrappers) runs fine, but without the new
> libstdc++.so.6.0.14 linked to libstdc++.so.6.0 fails with:
> 
> ./firefox-bin: /usr/lib32/libstdc++.so.6: version `GLIBCXX_3.4.14' not found
> (required by ./libxul.so)

Yeah, that's because of the new libstd++ dependency. Bug 561236 aims to address this. I need to investigate alternative workarounds. Will try to wrap that up this week.

> 
> Taras, is there any ETA for updated profiledbuild target for linux? IIRC it
> hasn't landed yet.

Ted, waiting for your review on this in bug 418866.
Created attachment 443309 [details]
compile-gcc-4.5.0.sh

Create tarballs ready for Puppet.
The binaries can be downloaded from http://people.mozilla.org/~raliiev/puppet/
Attachment #443099 - Attachment is obsolete: true
Attachment #443309 - Flags: review?(catlee)
Created attachment 443310 [details] [diff] [review]
GCC 4.5.0 Puppet manifests

Install gcc450 as a secondary compiler for Centos platform. No symlink required.
Attachment #443310 - Flags: review?(bhearsum)
Created attachment 443311 [details] [diff] [review]
Combined patch for profiledbuild

Putting the combined patch from bugs mentioned by Taras just for the history. :)

I "tested" (just ran it :) ) the profiled build on i386 arch only (will try on amd64 these days). Worked fine on Ubuntu 8.04 (i386) and 10.04 (amd64 with lib32) with default libstdc++. Visually starts faster.

I used default nightly .mozconfig with the following changes:

CC="ccache /tools/gcc-4.5.0/bin/gcc"
CXX="ccache /tools/gcc-4.5.0/bin/g++"
mk_add_options  PROFILE_GEN_SCRIPT='$(PYTHON) @TOPSRCDIR@/_profile/pgo/profileserver.py'
amd64 build failed with the following error:

/tools/gcc-4.5.0/bin/g++ -o xptcstubs_x86_64_linux.o -c -I../../../../../../dist/stl_wrappers -I../../../../../../dist/system_wrappers -include ../../../../../../config/gcc_hidden.h -DMOZILLA_INTERNAL_API -DOSTYPE=\"Linux2.6.18-53.1.14\" -DOSARCH=Linux -DEXPORT_XPTC_API -D_IMPL_NS_COM -I./../.. -I./../../../../xptinfo/src  -I. -I. -I../../../../../../dist/include -I../../../../../../dist/include/nsprpub  -I/builds/slave/mozilla-central-linux64-nightly/build/dist/include/nspr -I/builds/slave/mozilla-central-linux64-nightly/build/dist/include/nss       -fPIC  -fno-rtti -fno-exceptions -Wall -Wpointer-arith -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wcast-align -Wno-invalid-offsetof -Wno-variadic-macros -Werror=return-type -Wno-long-long -pedantic -gdwarf-2 -fno-strict-aliasing -fshort-wchar -pthread -pipe  -DNDEBUG -DTRIMMED -gdwarf-2 -fprofile-use -Wcoverage-mismatch -freorder-blocks-and-partition -fprofile-correction  -Os -freorder-blocks -fomit-frame-pointer    -DMOZILLA_CLIENT -include ../../../../../../mozilla-config.h -Wp,-MD,.deps/xptcstubs_x86_64_linux.pp xptcstubs_x86_64_linux.cpp
{standard input}: Assembler messages:
{standard input}:7765: Error: can't resolve `.text.unlikely' {.text.unlikely section} - `.LCOLDB0' {.text section}
{standard input}:7927: Error: can't resolve `.text.unlikely' {.text.unlikely section} - `.LCOLDB0' {.text section}

Updated

7 years ago
Attachment #443309 - Flags: review?(catlee) → review+
GCC 4.5.0 tarballs for linux and linux64 platforms are in /N/centos5/dist.
(Reporter)

Comment 23

7 years ago
(In reply to comment #16)
> GCC 4.5 requires new version of gmp, mpfr and mpc libraries. To avoid any
> possible conflicts with te current toolchain I installed all of tools with
> --prefix=/tools/gcc-4.5.0 prefix and add wrappers for gcc/g++ with injected
> LD_LIBRARY_PATH. Any ideas how to avoid this hack without touching
> /etc/ld.so.conf and setting LD_LIBRARY_PATH?

No. Another unfun alternative is to make a mozg++ wrapper script with
LD_LIBRARY_PATH=... /path/to/g++ $@

> 
> Normal build (using wrappers) runs fine, but without the new
> libstdc++.so.6.0.14 linked to libstdc++.so.6.0 fails with:
> 
> ./firefox-bin: /usr/lib32/libstdc++.so.6: version `GLIBCXX_3.4.14' not found
> (required by ./libxul.so)

Apply the gcc patch in bug 561236 to fix this.


(In reply to comment #21)
> {standard input}: Assembler messages:
> {standard input}:7765: Error: can't resolve `.text.unlikely' {.text.unlikely
> section} - `.LCOLDB0' {.text section}
> {standard input}:7927: Error: can't resolve `.text.unlikely' {.text.unlikely
> section} - `.LCOLDB0' {.text section}

That's weird. I don't have a 64bit centos vm here. This *might* be due to having an ancient linker. Newer binutils might fix this.
(Reporter)

Comment 24

7 years ago
Adding LDFLAGS -static-libstdc++ to LDFLAGS seems to work fine(as an alternative to patching gcc). Infact, I don't even see any significant increase in size.
Created attachment 443596 [details]
compile-gcc-4.5.0.sh

(In reply to comment #24)
> Adding LDFLAGS -static-libstdc++ to LDFLAGS seems to work fine(as an
> alternative to patching gcc). Infact, I don't even see any significant increase
> in size.

Worked fine for i386. Increase in size is 417380 (11702238 vs 12119618).

I removed all vector/list changes from the previous "Combined patch for profiledbuild" (next one) for this build.

Snippet from .mozcofig:
export LDFLAGS="-static-libstdc++"

(In reply to comment #23)
> (In reply to comment #16)
> > GCC 4.5 requires new version of gmp, mpfr and mpc libraries. To avoid any
> > possible conflicts with te current toolchain I installed all of tools with
> > --prefix=/tools/gcc-4.5.0 prefix and add wrappers for gcc/g++ with injected
> > LD_LIBRARY_PATH. Any ideas how to avoid this hack without touching
> > /etc/ld.so.conf and setting LD_LIBRARY_PATH?

Setting LDFLAGS and BOOT_LDFLAGS to "-L/tools/gcc-4.5.0/lib -Wl,-rpath,/tools/gcc-4.5.0/lib" fixed the problem. No need to set LD_LIBRARY_PATH now. Tarballs for linux and linux64 put on /N/ and deployed using Pupet in staging.

> That's weird. I don't have a 64bit centos vm here. This *might* be due to
> having an ancient linker. Newer binutils might fix this.

i386 is more or less ready, let's try to hack this one. :)
Attachment #443309 - Attachment is obsolete: true
Created attachment 443599 [details] [diff] [review]
Combined patch for profiledbuild
Attachment #443311 - Attachment is obsolete: true
Created attachment 443721 [details] [diff] [review]
buildbot-configs changes

Use GCC 4.5.5 as default compiler for mozilla-central linux (32bit only) builds.
Adjust PROFILE_GEN_SCRIPT for nightly builds and add it for the release builds.

profiledbuild is not turned on by default, will be enabled later by tuning config.py.
Attachment #443721 - Flags: review?(bhearsum)
Created attachment 443736 [details] [diff] [review]
buildbot-configs changes

Added LDFLAGS for statical linking
Attachment #443721 - Attachment is obsolete: true
Attachment #443736 - Flags: review?(bhearsum)
Attachment #443721 - Flags: review?(bhearsum)
Comment on attachment 443736 [details] [diff] [review]
buildbot-configs changes

I think we should convert the addonsmgr, places, electrolysis, and tracemonkey repos over too, since they're derived from mozilla-central.

Beware of the symlinks!
Attachment #443310 - Flags: review?(bhearsum) → review+
Created attachment 443967 [details] [diff] [review]
buildbot-configs changes

s,@TOPSRCDIR@/build,@MOZ_OBJDIR@,
Attachment #443736 - Attachment is obsolete: true
Attachment #443967 - Flags: review?(bhearsum)
Attachment #443736 - Flags: review?(bhearsum)
Depends on: 564381
(Reporter)

Comment 31

7 years ago
Found a small problem with above.

Need to add PIC flags BOOT_CFLAGS=-fPIC and BOOT_LDFLAGS=-fPIC in order to produce useful binary with -static-libstdc++.

Otherwise libstdc++ causes libxul to contain text relocations which piss off selinux and prelink.
Comment on attachment 443599 [details] [diff] [review]
Combined patch for profiledbuild

landed to mozilla-central
Attachment #443599 - Attachment is obsolete: true
Summary: Install GCC 4.5 on linux VMs to enable PGO on Linux → Install GCC 4.5 on linux buildslaes to enable PGO on Linux
Whiteboard: [puppet][buildslaves]
Summary: Install GCC 4.5 on linux buildslaes to enable PGO on Linux → Install GCC 4.5 on linux buildslaves to enable PGO on Linux
Created attachment 445094 [details] [diff] [review]
compile-gcc-4.5.0.sh

GCC 4.5.0 build script with updated flags.

Taras, could you take a look and check if the parameters and flags are OK.

The builds using this compiler pass all unittests (profiledbuild and debug build) in staging. If there is no stopper I'd like to deploy the current compiler build to the build slaves and enable it and profiledbuild for mozilla-central.
Attachment #443596 - Attachment is obsolete: true
Attachment #445094 - Flags: review?(tglek)
(Reporter)

Comment 34

7 years ago
Comment on attachment 445094 [details] [diff] [review]
compile-gcc-4.5.0.sh

>mkdir -p $pkg-$ver/gcc-obj-release
>export CFLAGS="-fPIC"
>export BOOT_CFLAGS="$CFLAGS"
>export LDFLAGS="-L$PREFIX/lib -Wl,-rpath,$PREFIX/lib -fPIC"
>export BOOT_LDFLAGS="$LDFLAGS"
>cd $pkg-$ver/gcc-obj-release
>../configure --prefix=$PREFIX --enable-__cxa_atexit --enable-languages=c,c++ \
>    --with-gmp=$PREFIX \
>    --with-mpfr=$PREFIX \
>    --with-mpc=$PREFIX
>make BOOT_LDFLAGS="$BOOT_LDFLAGS" LDFLAGS="$LDFLAGS" \
>    CFLAGS="$CFLAGS" BOOT_CFLAGS="$BOOT_CFLAGS" bootstrap
>make BOOT_LDFLAGS="$BOOT_LDFLAGS" LDFLAGS="$LDFLAGS" bootstrap

Don't need to pass these arguments to make, but it probably doesn't hurt. 

Looks good other than that.
Attachment #445094 - Flags: review?(tglek) → review+
Created attachment 445161 [details] [diff] [review]
compile-gcc-4.5.0.sh

Bump version (moz1) to make puppet happy.

Keeping r+ by Taras
Attachment #445094 - Attachment is obsolete: true
Attachment #445161 - Flags: review+
Comment on attachment 443310 [details] [diff] [review]
GCC 4.5.0 Puppet manifests

http://hg.mozilla.org/build/puppet-manifests/rev/367074f7877a
64bit part ignored
Attachment #443310 - Flags: checked-in+
Created attachment 445167 [details] [diff] [review]
GCC 4.5.0_moz1 Puppet manifests

Bump version, obsolete previous build, maintain gcc-4.5 symlink
Attachment #445167 - Flags: review?(bhearsum)
(Reporter)

Comment 38

7 years ago
Comment on attachment 445161 [details] [diff] [review]
compile-gcc-4.5.0.sh


>mkdir -p $pkg-$ver/gcc-obj-release
>export CFLAGS="-fPIC"
>export BOOT_CFLAGS="$CFLAGS"
>export LDFLAGS="-L$PREFIX/lib -Wl,-rpath,$PREFIX/lib -fPIC"
>export BOOT_LDFLAGS="$LDFLAGS"
>cd $pkg-$ver/gcc-obj-release
>../configure --prefix=$PREFIX \
>    --enable-__cxa_atexit --enable-languages=c,c++ \
>    --with-gmp=$PREFIX \
>    --with-mpfr=$PREFIX \
>    --with-mpc=$PREFIX
>make BOOT_LDFLAGS="$BOOT_LDFLAGS" LDFLAGS="$LDFLAGS" \
>    CFLAGS="$CFLAGS" BOOT_CFLAGS="$BOOT_CFLAGS" bootstrap

Instead of passing -fPIC above, the correct way is to add --with-pic to configure options. This fixes the full gcc build here.
(Reporter)

Comment 39

7 years ago
(In reply to comment #38)

> 
> Instead of passing -fPIC above, the correct way is to add --with-pic to
> configure options. This fixes the full gcc build here.

Scratch that. --with-pic produces a weird libstdc++. export CXXFLAGS=-fPIC is the way to go.
Blocks: 415563
(Reporter)

Comment 40

7 years ago
4.5 fixes a build failure with 4.3 in  bug 415563

Comment 41

7 years ago
taras: do you have the details of the gcc bug in bug 415563
that is fixed in gcc 4.5?  I'd like to know under what conditions
that bug affects us, and whether the fix has been backported
to a gcc 4.3.x or 4.4.x release.  This info will enable me
to implement a workaround in NSPR.  Thanks.
(Reporter)

Comment 42

7 years ago
(In reply to comment #41)
> taras: do you have the details of the gcc bug in bug 415563
> that is fixed in gcc 4.5?  I'd like to know under what conditions
> that bug affects us, and whether the fix has been backported
> to a gcc 4.3.x or 4.4.x release.  This info will enable me
> to implement a workaround in NSPR.  Thanks.

No, all I did was try two different gcc versions(4.3.3 and 4.5). The stackoverflow link that blassey posted in bug 415563 indicates that -march=i486 on relevant files is a workaround.
Taras, sorry for the delay, I'm a bit busy by other tasks right now. :(

Would you mind if I ask you to try compiling gcc and test it on one of the our builders? I could ask IT to clone one.
(Reporter)

Comment 44

7 years ago
(In reply to comment #43)
> Taras, sorry for the delay, I'm a bit busy by other tasks right now. :(
> 
> Would you mind if I ask you to try compiling gcc and test it on one of the our
> builders? I could ask IT to clone one.

I've been testing in the buildvm, CXXFLAGS=-fPIC is all we need to get this going. Just need to confirm/deploy in production now
Build fails with CXXFLAGS=-fPIC set (for GCC). Tried to use the current version of binutils (2.20) but the result is the same.

/tools/gcc-4.5/bin/g++ -o xptcinvoke_gcc_x86_unix.o -c -I../../../../../../dist/stl_wrappers -I../../../../../../dist/system_wrappers -include /builds/slave/linux_build/build/config/gcc_hidden.h -DMOZILLA_INTERNAL_API -DOSTYPE=\"Linux2.6.18-53.1.19\" -DOSARCH=Linux -DEXPORT_XPTC_API -D_IMPL_NS_COM -I/builds/slave/linux_build/build/xpcom/reflect/xptcall/src/md/unix/../.. -I/builds/slave/linux_build/build/xpcom/reflect/xptcall/src/md/unix/../../../../xptinfo/src  -I/builds/slave/linux_build/build/xpcom/reflect/xptcall/src/md/unix -I. -I../../../../../../dist/include -I../../../../../../dist/include/nsprpub  -I/builds/slave/linux_build/build/obj-firefox/dist/include/nspr -I/builds/slave/linux_build/build/obj-firefox/dist/include/nss       -fPIC  -fno-rtti -fno-exceptions -Wall -Wpointer-arith -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wcast-align -Wno-invalid-offsetof -Wno-variadic-macros -Werror=return-type -Wno-long-long -pedantic -gdwarf-2 -fno-strict-aliasing -fshort-wchar -pthread -pipe  -DNDEBUG -DTRIMMED -gdwarf-2 -fprofile-use -fprofile-correction -Wcoverage-mismatch -freorder-blocks-and-partition -Os -freorder-blocks -fomit-frame-pointer    -DMOZILLA_CLIENT -include ../../../../../../mozilla-config.h -Wp,-MD,.deps/xptcinvoke_gcc_x86_unix.pp /builds/slave/linux_build/build/xpcom/reflect/xptcall/src/md/unix/xptcinvoke_gcc_x86_unix.cpp
{standard input}: Assembler messages:
{standard input}:200: Error: can't resolve `.text.unlikely' {.text.unlikely section} - `.LCOLDB0' {.text section}
make[10]: *** [xptcinvoke_gcc_x86_unix.o] Error 1
(Reporter)

Updated

7 years ago
Depends on: 569137
(Reporter)

Comment 46

7 years ago
Sorry forgot about that, I had a workaround patch in my queue for this. I filed bug 569137 to fix this.
That's not a binutils problem, it's something to do with how gcc layouts the sections. I'll file a gcc bug eventually.
Blocks: 552567
Created attachment 448687 [details] [diff] [review]
GCC 4.5.0 spec (including GMP, MPFR, MPC libraries)

A bit ugly spec:
* I included GMP, MPFR and MPC libraries into the package, so I need to install them to the target (/tools/gcc...) before gcc build. The libraries could be packaged separately, but we aren't going to use them for other packages.
* %install just copies files from --prefix to rpm's "right" directory to make rpmbuild happy

A binary produced by this package got OK from Taras.
Attachment #445161 - Attachment is obsolete: true
Attachment #448687 - Flags: review?(bhearsum)
Created attachment 448692 [details] [diff] [review]
GCC 4.5.0 Puppet manifests

The manifest also removes the previous buggy gcc-4.5.0 version.
Attachment #443310 - Attachment is obsolete: true
Attachment #445167 - Attachment is obsolete: true
Attachment #448692 - Flags: review?(bhearsum)
Attachment #445167 - Flags: review?(bhearsum)
Attachment #448692 - Flags: review?(bhearsum) → review?(bear)
Comment on attachment 448692 [details] [diff] [review]
GCC 4.5.0 Puppet manifests

Bear, can you review this?
Comment on attachment 448687 [details] [diff] [review]
GCC 4.5.0 spec (including GMP, MPFR, MPC libraries)

Taras tells me the build instructions look good.

Looks OK to me, too.
Attachment #448687 - Flags: review?(bhearsum) → review+
Comment on attachment 443967 [details] [diff] [review]
buildbot-configs changes

I think you need to use /tools/gcc-4.5 now, based on my reading of the spec and puppet manifests.
Attachment #443967 - Flags: review?(bhearsum) → review-

Updated

7 years ago
Attachment #448692 - Flags: review?(bear) → review+

Comment 52

7 years ago
Is this ready for testing on staging - do you have the rpm ready?

Updated

7 years ago
No longer blocks: 552567
Created attachment 449699 [details] [diff] [review]
GCC 4.5.0 spec (including GMP, MPFR, MPC libraries)

* Fixed --prefix (/tools/gcc-4.5) to eliminate Puppet symlink manipulation.
* Used 0moz1 release instead of moz1
Attachment #448687 - Attachment is obsolete: true
Attachment #449699 - Flags: review?(bear)
Created attachment 449703 [details] [diff] [review]
GCC 4.5.0 Puppet manifests

Use install_rpm
Attachment #448692 - Attachment is obsolete: true
Attachment #449703 - Flags: review?(bhearsum)

Updated

7 years ago
Attachment #449699 - Flags: review?(bear) → review+

Comment 55

7 years ago
Comment on attachment 449699 [details] [diff] [review]
GCC 4.5.0 spec (including GMP, MPFR, MPC libraries)

just realized your asking me to review the spec but I was seeing the puppet changes
Attachment #449699 - Flags: review+ → review-
Created attachment 449710 [details]
GCC 4.5.0 spec (including GMP, MPFR, MPC libraries)

Oops. Here is the spec.
Attachment #449699 - Attachment is obsolete: true
Attachment #449710 - Flags: review?(bear)

Updated

7 years ago
Attachment #449710 - Flags: review?(bear) → review+
Attachment #449703 - Flags: review?(bhearsum) → review+
Comment on attachment 449703 [details] [diff] [review]
GCC 4.5.0 Puppet manifests

Last NFS remnant is being removed here, yay!
Comment on attachment 449703 [details] [diff] [review]
GCC 4.5.0 Puppet manifests

changeset:   177:d7b96acef7ad
Attachment #449703 - Flags: checked-in+
Comment on attachment 449710 [details]
GCC 4.5.0 spec (including GMP, MPFR, MPC libraries)

changeset:   3:026e7ec61437
Attachment #449710 - Flags: checked-in+
I watched a 32 and 64-bit slave sync up successfully.

Updated

7 years ago
Blocks: 570994
Created attachment 450609 [details] [diff] [review]
buildbot-configs changes

Taras, could you take a look at these mozconfigs for linux and linux64?

Some notes.

* I synced linux and linux64 mozconfigs, they are the same (next step will be removing linux64 configs and symlinking them to linux? :) ).

* As a result there are some changes:
** debug build for linux64 got --enable-libxul
** release build for linux64 got --enable-tests
** Dropped "export ac_cv_have_visibility_class_bug=yes" (per comments set for gcc-4.2.2)
Attachment #443967 - Attachment is obsolete: true
Attachment #450609 - Flags: review?(tglek)
FYI:

(In reply to comment #61)
> ** debug build for linux64 got --enable-libxul
> ** release build for linux64 got --enable-tests

These are both default settings. The --enable-libxul thing changed when we merged the electrolysis code, because it doesn't build without libxul, so we changed debug builds to default to libxul. --enable-tests has been the default for a long time.
(Reporter)

Updated

7 years ago
Attachment #450609 - Flags: review?(tglek) → review?(ted.mielczarek)
(Reporter)

Comment 63

7 years ago
Comment on attachment 450609 [details] [diff] [review]
buildbot-configs changes

I think Ted is the domain expert here.
Comment on attachment 450609 [details] [diff] [review]
buildbot-configs changes

>diff --git a/mozilla2-staging/linux/mozilla-central/codecoverage/mozconfig b/mozilla2-staging/linux/mozilla-central/codecoverage/mozconfig
>--- a/mozilla2-staging/linux/mozilla-central/codecoverage/mozconfig
>+++ b/mozilla2-staging/linux/mozilla-central/codecoverage/mozconfig
>@@ -1,20 +1,18 @@
> mk_add_options MOZ_MAKE_FLAGS="-j3"
>+mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py'

You don't really need this in the debug and codecoverage mozconfigs, since we're not going to do PGO builds there, but it also won't hurt anything by being there.

Looks fine otherwise. I should really file a bug about cleaning these up a bit. Since we got --enable-debug-symbols working correctly on trunk, we don't need the export CFLAGS="-gdwarf-2" bits, or the export MOZ_DEBUG_SYMBOLS=1 bits. Plus, as I mentioned before, --enable-tests is the default.
Attachment #450609 - Flags: review?(ted.mielczarek) → review+
Created attachment 451648 [details] [diff] [review]
buildbot-configs changes

(In reply to comment #64)
> (From update of attachment 450609 [details] [diff] [review])
> >diff --git a/mozilla2-staging/linux/mozilla-central/codecoverage/mozconfig b/mozilla2-staging/linux/mozilla-central/codecoverage/mozconfig
> >--- a/mozilla2-staging/linux/mozilla-central/codecoverage/mozconfig
> >+++ b/mozilla2-staging/linux/mozilla-central/codecoverage/mozconfig
> >@@ -1,20 +1,18 @@
> > mk_add_options MOZ_MAKE_FLAGS="-j3"
> >+mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py'
> 
> You don't really need this in the debug and codecoverage mozconfigs, since
> we're not going to do PGO builds there, but it also won't hurt anything by
> being there.

Removed it. Let's keep the mosconfigs clean. :) 

Keeping r+.
Attachment #450609 - Attachment is obsolete: true
Attachment #451648 - Flags: review+
Blocks: 571571
Created attachment 453331 [details] [diff] [review]
buildbot-configs changes

Applied to mozilla-2.0 too. Keeping r+.
Attachment #451648 - Attachment is obsolete: true
Attachment #453331 - Flags: review+
Comment on attachment 453331 [details] [diff] [review]
buildbot-configs changes

http://hg.mozilla.org/build/buildbot-configs/rev/d67416d28d95
Attachment #453331 - Flags: checked-in+
No longer blocks: 571571

Updated

7 years ago
Attachment #453331 - Flags: checked-in+

Updated

7 years ago
Attachment #453331 - Flags: checked-in-

Updated

7 years ago
Blocks: 575620
(Reporter)

Updated

7 years ago
Depends on: 576399
(Reporter)

Comment 68

7 years ago
Just landed a 4.5 workaround that should avoid the regressions we saw. When is the next time we can try a 4.5 switch?

Comment 69

7 years ago
I have been building Firefox (32-bit) on Linux using PGO successfully since Firefox version 3.0.

I have no problem with my most recent build of Firefox 3.6.6.

I have just had a preliminary attempt to build Firefox 4.0b1 and run into a minor problem:

Details:
source: pulled 'tar' downloaded from 'release candidate' folder on 'ftp site';
OS: Slackware Linux 12.1;
GCC version 4.2.3

Observation:
The PGO build process now proceed using '-fprofile-arcs/-fbranch-probabilities. instead of the correct process for GCC 4.2.3 which is '-fprofile-generate/fprofile-use'.

This has occurred because of the patch applied to 'configure.in' in BUG 418866 resulting in addition of GCC options (-fprofile-correction specifically) which is not compatible with GCC 4.2.3, therefore resulting in the process falling back to the option meant for GCC 3.3

See excerpt from 'configure.in':
dnl ========================================================
dnl Profile guided optimization
dnl ========================================================
dnl Test for profiling options
dnl Under gcc 3.3, use -fprofile-arcs/-fbranch-probabilities
dnl Under gcc 3.4+, use -fprofile-generate/-fprofile-use

I appreciate that future Mozilla builds will use GCC 4.5.x but if no amendment is made to 'configure'in' to allow for correct processing under GCC 4.2.3 then private builders will have problems as that GCC compiler will be treated like GCC 3.3.

Can anything be done to patch 'configure.in' to allow GCC 3.4+ to 4.2.3 to work as previously?.
(Reporter)

Comment 70

7 years ago
Created attachment 457606 [details] [diff] [review]
gcc patch

We'd like this patch applied to help static analysis builds
Created attachment 457806 [details] [diff] [review]
GCC 4.5.0 spec (including plugin_finish_decl.diff)

on mv-moz2-linux-ix-slave01:
Wrote: /home/cltbld/rpmbuild/SRPMS/gcc45-4.5.0-0moz2.src.rpm
Wrote: /home/cltbld/rpmbuild/RPMS/i386/gcc45-4.5.0-0moz2.i386.rpm

moz2-linux64-slave07:
Wrote: /usr/src/redhat/RPMS/x86_64/gcc45-4.5.0-0moz2.x86_64.rpm
Attachment #449710 - Attachment is obsolete: true
Attachment #457806 - Flags: review?(bear)
Created attachment 457809 [details] [diff] [review]
GCC 4.5.0 Puppet manifests

Bump RPM version
Attachment #449703 - Attachment is obsolete: true
Attachment #457809 - Flags: review?(bhearsum)

Updated

7 years ago
Attachment #457806 - Flags: review?(bear) → review+
Attachment #457809 - Flags: review?(bhearsum) → review+
Comment on attachment 457806 [details] [diff] [review]
GCC 4.5.0 spec (including plugin_finish_decl.diff)

http://hg.mozilla.org/build/rpm-sources/rev/c8fe426b3bdb
Attachment #457806 - Flags: checked-in+
Comment on attachment 457809 [details] [diff] [review]
GCC 4.5.0 Puppet manifests

http://hg.mozilla.org/build/puppet-manifests/rev/9b1b8bdd384a
Attachment #457809 - Flags: checked-in+
Updated RPM package is being deployed and seen on some production and try machines. It should be available on all machines after the weekend.
(Reporter)

Comment 76

7 years ago
Is there an eta for setting 4.5 as the default compiler yet?
Have you done new builds with it yet?  It should be available on try now.  What are the performance impacts of it?
(Reporter)

Comment 78

7 years ago
(In reply to comment #77)
> Have you done new builds with it yet?  It should be available on try now.  What
> are the performance impacts of it?

I landed a workaround in bug 576399 which seems to bring perf back to 4.3 levels(according to try). Hard to say for sure until we try it.
Ok, let's try it early next week, assuming that works with the beta schedule.
(Reporter)

Comment 80

7 years ago
(In reply to comment #79)
> Ok, let's try it early next week, assuming that works with the beta schedule.

So my plan is to try my workaround. Hope that it works like it did on try server.

If it turns out to not work, I'd like to switch our linux compilation to -O2, take the regression on startup(win on everything else), then turn on PGO and hopefully win on startup too.

It will all be simple if 4.5 works as expected(with 576399 workaround) and no hard decisions on accepting regressions have to be taken.
Created attachment 460452 [details] [diff] [review]
buildbot-configs changes

Refreshed mozconfigs.
Attachment #453331 - Attachment is obsolete: true
Attachment #460452 - Flags: review?(catlee)
(Reporter)

Comment 82

7 years ago
4.5.1rc came out last week. Lets wait a few days for 4.5.1 final.

Updated

7 years ago
Attachment #460452 - Flags: review?(catlee) → review+

Comment 83

7 years ago
bug 582593 indicates that gcc 4.5 is broken :)
(Reporter)

Comment 84

7 years ago
(In reply to comment #82)
> 4.5.1rc came out last week. Lets wait a few days for 4.5.1 final.

Ok, lets roll this out http://ftp.gnu.org/pub/gnu/gcc/gcc-4.5.1
Blocks: 584472
(In reply to comment #84)
> (In reply to comment #82)
> > 4.5.1rc came out last week. Lets wait a few days for 4.5.1 final.
> 
> Ok, lets roll this out http://ftp.gnu.org/pub/gnu/gcc/gcc-4.5.1

Taras, do you still need the patch (attachment 457606 [details] [diff] [review])?
(Reporter)

Comment 86

7 years ago
(In reply to comment #85)

> Taras, do you still need the patch (attachment 457606 [details] [diff] [review])?

Yes please.
Created attachment 463080 [details] [diff] [review]
GCC 4.5.1 spec
Attachment #457806 - Attachment is obsolete: true
Attachment #463080 - Flags: review?(bear)
Created attachment 463082 [details] [diff] [review]
GCC 4.5.1 Puppet manifests
Attachment #457809 - Attachment is obsolete: true
Attachment #463082 - Flags: review?(bhearsum)

Updated

7 years ago
Attachment #463080 - Flags: review?(bear) → review+
Taras, please compile GCC 4.5.1 + patches and do some testing with it before we deploy it. We're happy to do compiler updates with good reason, but rolling them out to the entire build pool is not the right way to do initial testing.
(Reporter)

Comment 90

7 years ago
(In reply to comment #89)
> Taras, please compile GCC 4.5.1 + patches and do some testing with it before we
> deploy it. We're happy to do compiler updates with good reason, but rolling
> them out to the entire build pool is not the right way to do initial testing.

I've run tests on 4.5.1 now, works as well as 4.3 for me.
Thanks Taras
Attachment #463082 - Flags: review?(bhearsum) → review+

Updated

7 years ago
Blocks: 585235
Comment on attachment 463080 [details] [diff] [review]
GCC 4.5.1 spec

http://hg.mozilla.org/build/rpm-sources/rev/ae595518a672
Attachment #463080 - Flags: checked-in+
Blocks: 585605
No longer blocks: 585605
Comment on attachment 463082 [details] [diff] [review]
GCC 4.5.1 Puppet manifests

http://hg.mozilla.org/build/puppet-manifests/rev/4f06048ffe74

mpt-production-puppet, mv-production-puppet and staging-puppet servers are
updated as well. GCC 4.5.1 will be available on all slaves in day or two.
Attachment #463082 - Flags: checked-in+
(Reporter)

Comment 94

7 years ago
This would be nice to have for firefox4 for the overall speedup and to facilitate jar reordering in bug 559961.
blocking2.0: --- → ?
Created attachment 468316 [details] [diff] [review]
buildbot-configs changes
Attachment #460452 - Attachment is obsolete: true
Attachment #468316 - Flags: review?(catlee)

Updated

7 years ago
Attachment #468316 - Flags: review?(catlee) → review+
Comment on attachment 468316 [details] [diff] [review]
buildbot-configs changes

http://hg.mozilla.org/build/buildbot-configs/rev/74860b24b577
Attachment #468316 - Flags: checked-in+
Comment on attachment 468316 [details] [diff] [review]
buildbot-configs changes

Backed out due to failed "make test":
http://hg.mozilla.org/build/buildbot-configs/rev/9e22ddd81fa1
Attachment #468316 - Flags: checked-in+ → checked-in-
(Reporter)

Comment 98

7 years ago
Rail, are there instructions for turning your spec files to rpms? Alternatively, are there some src rpms for the 4.5.0/4.5.1 specs above?
(In reply to comment #98)
> Rail, are there instructions for turning your spec files to rpms?

Very short instructions are here: https://wiki.mozilla.org/ReleaseEngineering:Puppet:Usage#Building_RPMs

> Alternatively, are there some src rpms for the 4.5.0/4.5.1 specs above?

The latest one is here:
http://people.mozilla.org/~raliiev/gcc/gcc45-4.5.1-0moz1.src.rpm

If you want I can regenerate src.rpm for 4.5.0.
(Reporter)

Updated

7 years ago
Depends on: 594611
(Assignee)

Updated

7 years ago
Depends on: 590683
(Reporter)

Comment 100

7 years ago
Rail, please add the patch from http://gcc.gnu.org/bugzilla/show_bug.cgi?id=45623#c24 to our gcc 4.5.1. This appears to conclude our 4.5.1 issues on my machine. I would like to ensure that the same is true on try and switch to 4.5 asap. It's blocking bug 590181(otherwise we have to change how we build 4.3, but i'd rather not waste time on that if we can avoid it) and other perf fixes.
Blocks: 590181
Blocks: 414641
(In reply to comment #9)
> Actually the good numbers produced by pgo in
> http://blog.mozilla.com/tglek/2010/04/12/squeezing-every-last-bit-of-performance-out-of-the-linux-toolchain/
> have little to do with cpu time. -freorder-blocks-and-partition option changed
> the layout of the binary for the better. Any minor cpu-time benefits are just
> icing on my startup cake.

This was PGO plus -Os, right?  I wonder if PGO would help more with -O3 (patch in bug 590181).
(Reporter)

Comment 102

7 years ago
(In reply to comment #101)
> (In reply to comment #9)
> > Actually the good numbers produced by pgo in
> > http://blog.mozilla.com/tglek/2010/04/12/squeezing-every-last-bit-of-performance-out-of-the-linux-toolchain/
> > have little to do with cpu time. -freorder-blocks-and-partition option changed
> > the layout of the binary for the better. Any minor cpu-time benefits are just
> > icing on my startup cake.
> 
> This was PGO plus -Os, right?  I wonder if PGO would help more with -O3 (patch
> in bug 590181).

-O3 would hurt startup due to more bloated code causing more pagefaults.


Rail, http://gcc.gnu.org/viewcvs/branches/gcc-4_5-branch/gcc/tree-ssa-structalias.c?r1=164430&r2=164429&view=patch&pathrev=164430 is the official gcc fix. It supersedes the patch I linked in comment 100.
(In reply to comment #102)
> > This was PGO plus -Os, right?  I wonder if PGO would help more with -O3 (patch
> > in bug 590181).
> 
> -O3 would hurt startup due to more bloated code causing more pagefaults.

Right, this was the behavior we saw in bug 590181; -O3 was a Ts_cold regression, but sped everything else up.  But in that bug, we said we wanted -O3 despite the cold startup regression; my understanding is that we still intend to land it, once it's unblocked.

My suspicion is that PGO plus -O3 might give a bigger gain over -O3 than PGO plus -Os gives over -Os.  (Consider that adding -freorder-blocks to -Os is moving you closer to -O3.)  It's also possible that PGO plus -O3 might not hurt Ts_cold as much as non-PGO -O3, since perhaps PGO's optimizations reduce the cost of -O3's code bloat.

So my question is: Can we try this and collect data, perhaps in a separate bug?
Created attachment 477257 [details] [diff] [review]
GCC 4.5.1 spec

Updated spec file + the official patch
Attachment #463080 - Attachment is obsolete: true
Attachment #477257 - Flags: review?(bear)
Created attachment 477259 [details] [diff] [review]
GCC 4.5.1 Puppet manifests
Attachment #463082 - Attachment is obsolete: true
Attachment #477259 - Flags: review?(bhearsum)

Updated

7 years ago
Attachment #477257 - Flags: review?(bear) → review+
Attachment #477259 - Flags: review?(bhearsum) → review+
Comment on attachment 477257 [details] [diff] [review]
GCC 4.5.1 spec

http://hg.mozilla.org/build/rpm-sources/rev/7846aabb1b84
Attachment #477257 - Flags: checked-in+
Comment on attachment 477259 [details] [diff] [review]
GCC 4.5.1 Puppet manifests

http://hg.mozilla.org/build/puppet-manifests/rev/578a04a78b5a
Attachment #477259 - Flags: checked-in+
The package with the specified patch is being deployed and will be available on the build hosts (including try machines) within a couple of days. Happy testing! :)
(Reporter)

Comment 109

7 years ago
Ok, try confirms that gcc4.5 fixes did their job. Lets make 4.5 the default compiler.

Don't worry about a perf hit, once 4.5 sticks. Bug 590181 will make up for any -Os perf regressions in 4.5.
(In reply to comment #109)
> Don't worry about a perf hit, once 4.5 sticks. Bug 590181 will make up for any
> -Os perf regressions in 4.5.
But we should probably post about the fact that a regression will happen to tree management and platform to make sure people are aware of what is going on.

Comment 111

7 years ago
And this should also include a reply to the automated Talos postings when it sees the regressions, otherwise people will get jumpy.
(Reporter)

Comment 112

7 years ago
So 4.5 passed try except for opengl failures. I got
the following failing in an optimized build 
opengl
4998/80/270


and these in debug:
opengl
4998/125/280
I hope this is linux gl being broken and not.

Comment 113

7 years ago
Is that over and above the existing RGL failures? Linux/Linux64 are both pretty pooched on RGL on the standard build.
(Reporter)

Comment 114

7 years ago
(In reply to comment #112)

> I hope this is linux gl being broken

Joe says "ignore", this means gcc 4.5 got a clean bill of health on try.
So now we need an eta from Build&Release.

Updated

7 years ago
Blocks: 599530
As per today's development meeting, this is not going to happen for Beta 7.

Taras, can you talk with beltzner to schedule having this possibly included in Beta 8?  If you want to plead your case, come to the next Tuesday development meeting.

Also, Talos numbers are needed before this can land.
(Reporter)

Comment 116

7 years ago
jlebar, can you post your -O3 numbers with gcc 4.5?
Yes, I'll kick off a run on try.
(In reply to comment #117)
> Yes, I'll kick off a run on try.
We should get a few runs and then compare to a few good runs of parent changesets on mozilla-central with https://services.forerunnerdesigns.com/compare-talos/
Created attachment 480587 [details] [diff] [review]
buildbot-configs changes (mozconfigs)
Attachment #468316 - Attachment is obsolete: true
Attachment #480587 - Flags: review?(catlee)
Does this bug actually cover enabling PGO on our Linux builds, or just getting everything ready on the build slave side? If the former, we'll need to block on bug 602245.
(In reply to comment #120)
> Does this bug actually cover enabling PGO on our Linux builds, or just getting
> everything ready on the build slave side? If the former, we'll need to block on
> bug 602245.

AFAIK, we want to do both but in 2 steps: 1) use gcc-4.5.x for non profiled builds, wait some days and see if there are unexpected failures; 2) enable PGO
(Reporter)

Comment 122

7 years ago
So now that b7 has branched when can we deploy 4.5?
(Reporter)

Updated

7 years ago
Blocks: 603368
The b7 branch and trunk build with the same mozconfig. Can't do it at least until b7 ships.
Also, still missing talos numbers from comment 115.
For Talos results, see bug 590181 comment 68 and comment 69.  No ts_cold regression, which is a bit strange.  Its absence may be worth looking into.

Updated

7 years ago
Attachment #480587 - Flags: review?(catlee) → review+

Updated

7 years ago
Priority: P2 → P4

Comment 126

7 years ago
catlee, howcome this got bumped from P2 to P4?

I think we want this ASAP. Ideally, it'd be ready to go as soon as we ship b7. There is other stuff depending on this that we really want to get done.
(In reply to comment #126)
> catlee, howcome this got bumped from P2 to P4?
> 
> I think we want this ASAP. Ideally, it'd be ready to go as soon as we ship b7.
> There is other stuff depending on this that we really want to get done.

Because there's nothing actionable for us to do right now.  Everything is ready to go once we get the go-ahead.

Comment 128

7 years ago
Aha -- awesome. Thanks. :)
Bug 578880 seems to depend on this -- marking this a blocker.
blocking2.0: ? → betaN+
Blocks: 578880

Comment 130

7 years ago
We've shipped b7. Are we good to go?
(In reply to comment #130)
> We've shipped b7. Are we good to go?

Hasn't quite shipped yet.  Build 1 is still in QA.

Comment 132

7 years ago
Er, sorry. Meant tagged. :/
(132 comments, really?)

So...since we've got a b7 relbranch, and any future b7 fixes will go on the relbranch, I think we're ok to make the switch on trunk.

Taras, want to bring this up at Tuesday's platform meeting to schedule the switch-over?
Flags: needs-treeclosure?
(Reporter)

Updated

7 years ago
Depends on: 609543

Updated

6 years ago
Flags: needs-treeclosure?
Is there an ETA for this?  This is currently blocking blocker work for Firefox 4 on Linux, not just turning on PGO.
This needs bug 609543 to be fixed, at least, before we can try again.
Is gcc 4.5 already deployed to the build machines?  That is, can a mozconfig switch over to test whether bug 609543 is really being caused by gcc 4.5?
(In reply to comment #136)
> Is gcc 4.5 already deployed to the build machines?  That is, can a mozconfig
> switch over to test whether bug 609543 is really being caused by gcc 4.5?

Yes, it's been deployed for ages.  You can do a try build with a custom mozconfig to select gcc 4.5.

Comment 138

6 years ago
OK, bug 609543 is fixed. It sounds like jlebar has found another couple failures though: https://bugzilla.mozilla.org/show_bug.cgi?id=590181#c107. Do those need to block deployment here?
blocking2.0: betaN+ → -

Updated

6 years ago
No longer depends on: 609543

Updated

6 years ago
No longer blocks: 578880
Blocks: 578880

Updated

6 years ago
Depends on: 609543
(Assignee)

Comment 139

6 years ago
Comment on attachment 480587 [details] [diff] [review]
buildbot-configs changes (mozconfigs)

>+export LDFLAGS="-static-libstdc++"

You will also need to set HOST_LDFLAGS to the same value. (but that'll need bug 628303 to land first)
This is currently a problem on seamonkey, which switched to 4.5 with this patch, since elfhack (bug 606145) landed.
gcc-4.5 is already installed on the linux machines; this bug is now tracking enabling it by default for building firefox.
Summary: Install GCC 4.5 on linux buildslaves to enable PGO on Linux → switch to gcc-4.5 for linux builds
(Assignee)

Comment 141

6 years ago
Created attachment 518130 [details] [diff] [review]
PoC to allow to run gcc 4.5 builds against older libstdc++

I figured a hackish way to avoid g++ 4.5 linking against libstdc++ symbols that are only available in libstdc++.so.6 from gcc 4.5. I went even further and got rid of symbols from gcc 4.2, making the resulting binaries work with the system-provided libstdc++.so.6 on buildbots, allowing to get rid of these LD_LIBRARY_PATHs:
http://hg.mozilla.org/build/buildbot-configs/file/ccf475f8cf9d/mozilla/config.py#l1301

It also avoids requiring to use -static-libstdc++, so that could taken off attachment 480587 [details] [diff] [review].

I'll try to build with g++ 4.6 to see if there will be any further symbols to hack around in the future with the current codebase. (Future changes to the codebase may require other symbols, so the use of STL should still be discouraged)

Try builds available here:
http://ftp.mozilla.org/pub/mozilla.org/firefox/tryserver-builds/mh@glandium.org-69d6499b5c8a/

As that was only for a test, I tucked the hack in a random file and didn't protect it in some #ifdef. Integration details should be figured out before switching to gcc 4.5, obviously.
When bug 641842 is fixed, we should try rolling this out only on the build-system branch.
(Assignee)

Updated

6 years ago
Depends on: 643690
(Assignee)

Comment 143

6 years ago
Let's make both switches (gcc 4.5 + PGO) at the same time. I've heavily tested PGO on try and with gcc 4.5, it works great. We'll need to land bug 590181 first to get the best out of it (and I verified this doesn't make problems with gcc 4.3, except for a big size bump, that PGO greatly removes)
No longer blocks: 590181
Depends on: 641842, 602245
Summary: switch to gcc-4.5 for linux builds → switch to gcc-4.5 and enable PGO for linux builds
(Assignee)

Updated

6 years ago
Depends on: 590181
(Assignee)

Comment 144

6 years ago
So, we're almost set. All I need is a r+ on bug 643690, which should be pretty quick, and we're ready to go.

mozconfig-wise, we need to change CC and CXX, we don't need to add -static-libstdc++ to LDFLAGS, and we need to add the following to enable PGO:
  mk_add_options MOZ_PGO=1
and refresh PROFILE_GEN_SCRIPT where it makes sense.

We need to land bug 643690 before or during the switch, and bug 590181 preferably after or during.
How and when can we plan this switch?
(Assignee)

Comment 145

6 years ago
(In reply to comment #144)
> So, we're almost set. All I need is a r+ on bug 643690, which should be pretty
> quick, and we're ready to go.

... and 44 seconds later, the r+ was there. So we're ready :)
(Assignee)

Updated

6 years ago
Blocks: 650304
(Assignee)

Comment 146

6 years ago
And now that I landed bug 650467, we'll also need:
  ac_add_options --enable-stdcxx-compat
(Assignee)

Comment 147

6 years ago
Created attachment 526668 [details] [diff] [review]
buildbot-config changes

This is a refreshed patch as per comment 144 and comment 146.

I'm however unsure we want to enable PGO on unittest and debug builds.

Note that these changes will need to propagate to branches as the current mozilla-central propagates to aurora, beta and release.

At some point, some branches like tracemonkey and build-system will also need to switch. I'm however wondering how to handle try...
Attachment #480587 - Attachment is obsolete: true
Attachment #526668 - Flags: review?(catlee)
Comment on attachment 526668 [details] [diff] [review]
buildbot-config changes

Debug builds probably shouldn't use PGO. We don't on Windows at least.
No need to update the unittest mozconfigs, they're not used any more.

At the least try should be updated at the same time. Other branches should probably all happen at the same time as well. Maybe a post to dev.planning warning folks this is happening would be appropriate?
Attachment #526668 - Flags: review?(catlee) → review-
I think it would be best to give this a test run on build-system first. If we break anything, then it won't affect work on m-c.
(Assignee)

Comment 150

6 years ago
The problem with try is that if something before the landing of bug 643690 is pushed, it won't work.
(Assignee)

Comment 151

6 years ago
I had a nasty idea for try:

if hg log -r 98094f3afb93... -l1 | grep -q ^$; then
CC=/tools/gcc-4.5/bin/gcc
(...)
else
CC=/tools/gcc-4.3.3/installed/bin/gcc
(...)
fi

there's maybe a better way to do this, but I don't know mercurial well enough. It checks that revision 98094f3afb93 is an ancestor of the current revision.
(Assignee)

Comment 152

6 years ago
(In reply to comment #149)
> I think it would be best to give this a test run on build-system first. If we
> break anything, then it won't affect work on m-c.

Note that it doesn't look like build-system has a separate mozconfig at the moment.
(Assignee)

Comment 153

6 years ago
Created attachment 528560 [details] [diff] [review]
buildbot-config preparation

There appears to be a whole lot of stuff that are symbolic links to mozconfig-central configurations (e.g. mozilla-1.9.1, electrolysis...), we need to break these symbolic links first.
Attachment #526668 - Attachment is obsolete: true
Attachment #528560 - Flags: review?(catlee)
(Assignee)

Comment 154

6 years ago
Created attachment 528561 [details] [diff] [review]
buildbot-config changes

These are the actual changes to the buildbot configs.
Attachment #528561 - Flags: review?(catlee)
(Assignee)

Comment 155

6 years ago
(In reply to comment #154)
> Created attachment 528561 [details] [diff] [review]
> buildbot-config changes
> 
> These are the actual changes to the buildbot configs.

Note, I was wondering if we also need to change the "generic" ones.
(Assignee)

Comment 156

6 years ago
Created attachment 528579 [details]
Mercurial hook

This is a mercurial hook that checks whether something that is pushed to try contains changeset 3a38a70b0e12 as an ancestor, which is necessary for builds to work properly with gcc 4.5. It does so only if the try push is targetted at linux, linux64 or all platforms.
When it detects a push that doesn't contain that changeset, it issues a warning, currently only "Warning message" ; this obviously needs appropriate wording. For instance, do we want it to be verbose and give the workaround, or simply link to some documentation?
Attachment #528579 - Attachment mime type: text/x-python → text/plain

Updated

6 years ago
Attachment #528560 - Flags: review?(catlee) → review+

Updated

6 years ago
Attachment #528561 - Flags: review?(catlee) → review+
(Assignee)

Comment 157

6 years ago
Created attachment 528841 [details]
Mercurial hook

I will update the wiki accordingly (if I can)
Attachment #528579 - Attachment is obsolete: true
Attachment #528841 - Flags: review?(ted.mielczarek)
Attachment #528841 - Attachment mime type: text/x-python → text/plain
Comment on attachment 528841 [details]
Mercurial hook

Seems reasonable enough. You'll need to file a Server Ops bug to get the hook deployed into the try repo. You can commit it to the hghooks repo to have a place to put it: http://hg.mozilla.org/hgcustom/hghooks . You should ask someone in IT to verify that argparse will work there first, I'm not particularly sure about that.
Attachment #528841 - Flags: review?(ted.mielczarek) → review+
(Assignee)

Comment 159

6 years ago
Landed mozconfigs changes:
http://hg.mozilla.org/build/buildbot-configs/rev/938ef97e76f8
http://hg.mozilla.org/build/buildbot-configs/rev/222b10545b48
(Assignee)

Comment 160

6 years ago
Landed try repo hook:
http://hg.mozilla.org/hgcustom/hghooks/rev/2b48ff549ea9
(Assignee)

Updated

6 years ago
Depends on: 653437
(Assignee)

Comment 161

6 years ago
It looks like it worked well (once the mozconfig were actually used on the
buildbots)
Assignee: rail → mh+mozilla
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
This might have broken Linux crash reports.  See bug 654595.
(Assignee)

Updated

6 years ago
Depends on: 655020
(Assignee)

Updated

6 years ago
Depends on: 652459
No longer depends on: 655020
Depends on: 655020
(Assignee)

Updated

6 years ago
Depends on: 655121
(Assignee)

Updated

6 years ago
Depends on: 655124
(Assignee)

Updated

6 years ago
Alias: gcc4.5

Updated

6 years ago
No longer depends on: 655124

Updated

6 years ago
Depends on: 656500

Updated

6 years ago
Depends on: 658167
We forgot to track it for Firefox 6.
tracking-firefox6: --- → ?

Updated

6 years ago
Whiteboard: [puppet][buildslaves] → [puppet][buildslaves] nominated in comment 163

Updated

6 years ago
tracking-firefox6: ? → +
Just to clarify, we're tracking this for 6 just to watch what happens here and possibly backing out this change for 6? Or is there additional work that we're still waiting on here beyond possibly backing things out?
Is there any configuration that needs to migrate from mozilla-central to mozilla-aurora to mozilla-beta to mozilla-release in order for this to apply to releases?
(In reply to comment #165)
> Is there any configuration that needs to migrate from mozilla-central to
> mozilla-aurora to mozilla-beta to mozilla-release in order for this to apply
> to releases?

Looks like the beta buildbot-config [1] will need to be updated to match the aurora/m-c config [2].

[1] http://hg.mozilla.org/build/buildbot-configs/file/e2da95567950/mozilla2/linux/mozilla-beta/release/mozconfig

[2] http://hg.mozilla.org/build/buildbot-configs/file/e2da95567950/mozilla2/linux/mozilla-aurora/release/mozconfig
(Assignee)

Comment 167

6 years ago
The same thing as bug 658167 will need to happen on mozilla-beta and then on mozilla-release. That's why bug 558180 would really be helpful.

Updated

6 years ago
tracking-firefox6: + → -
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.