559964 - (gcc4.5) switch to gcc-4.5 and enable PGO for linux builds

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P4)

Description

[(dormant account)]

PGO provides a huge(30%) startup improvement on Linux. It's the biggest easy win available right now.

Comment 1

[Chris AtLee [:catlee]]

A few questions:

- Any special instructions for compiling gcc 4.4?  We had some patches for gcc4.3 when we compiled it.

- Which branches should this be enabled for?

- Any known compatibility issues between 4.3 and 4.4?  ABI changes?  libstdc++ changes?-

- What are the impacts of users of the binaries?  Will they need the gcc 4.4 and associated libstdc++ libraries installed?

Comment 2

[(dormant account)]

(In reply to comment #1)
> A few questions:
> 
> - Any special instructions for compiling gcc 4.4?  We had some patches for
> gcc4.3 when we compiled it.

No such patches for 4.4.
> 
> - Which branches should this be enabled for?

trunk.

> 
> - Any known compatibility issues between 4.3 and 4.4?  ABI changes?  libstdc++
> changes?-

Libstdc++ is gcc-dependent. So users will have to be using a distributing with gcc4.4

> - What are the impacts of users of the binaries?  Will they need the gcc 4.4
> and associated libstdc++ libraries installed?

they need the 4.4 libstdc++

Comment 3

[Benjamin Smedberg]

I don't think we actually *do* require a newer version of libstdc++: it's still libstdc++.so.6, and the ABI hasn't really changed since 4.2. If we don't run on Ubuntu 08.04 LTS, that would be a problem.

Comment 4

[Mike Hommey [:glandium]]

The only change I'm aware of in 4.4 is the va_list ABI on ARM, but mozilla has no externally available API using it, I believe.

Please note, as I said in bug 418866, and since mozilla support for linux x86-64 is being added, that gcc 4.4 fails to generate working coverage builds (1st pass) on x86-64, which makes it impossible to build pgo. I haven't tried with 4.5 yet.

Comment 5

[(dormant account)]

(In reply to comment #3)
> I don't think we actually *do* require a newer version of libstdc++: it's still
> libstdc++.so.6, and the ABI hasn't really changed since 4.2. If we don't run on
> Ubuntu 08.04 LTS, that would be a problem.

You are right. I just confirmed 4.4-produced binaries run great on 4.2-based ubuntu LTS. So there are no abi concerns from me anymore. 

So how soon can we get a linux PGO tinderbox?

Comment 6

[Chris AtLee [:catlee]]

(In reply to comment #5)
> (In reply to comment #3)
> > I don't think we actually *do* require a newer version of libstdc++: it's still
> > libstdc++.so.6, and the ABI hasn't really changed since 4.2. If we don't run on
> > Ubuntu 08.04 LTS, that would be a problem.
> 
> You are right. I just confirmed 4.4-produced binaries run great on 4.2-based
> ubuntu LTS. So there are no abi concerns from me anymore. 
> 
> So how soon can we get a linux PGO tinderbox?

Will this replace the existing linux build, or be in addition to it?

If you want to change how the existing build is done, I think this merits some newsgroup discussion first.

Comment 7

[(dormant account)]

(In reply to comment #6)
> Will this replace the existing linux build, or be in addition to it?
> 
> If you want to change how the existing build is done, I think this merits some
> newsgroup discussion first.

I would like to replace the current build as it would be insufferably slow compared to a PGO one. I sent a notice to http://groups.google.com/group/mozilla.dev.platform/browse_thread/thread/4037844abc6ef1f5#

I don't see why anyone would disagree with a faster firefox, but lets proceed on this bug in parallel to the discussion. In the worst case we can settle for an additional build.

Comment 8

[Mike Hommey [:glandium]]

(In reply to comment #7)
> I would like to replace the current build as it would be insufferably slow
> compared to a PGO one.

FWIW, I didn't get any really noticeable improvement with pgo builds, even on startup time, though i haven't formally verified the latter, and I have a SSD, so i cheat for startup. But that would suggest that there is no cpu-bound improvement in your faster startups, which in turn suggests there might be better ways to get these improvements (such as your reordering work).

But benchmarks such as sunspider didn't end up running significantly faster even when running it during the profiling phase. Is it worth doubling the build time ?

Comment 9

[(dormant account)]

(In reply to comment #8)
> (In reply to comment #7)
> > I would like to replace the current build as it would be insufferably slow
> > compared to a PGO one.
> 
> FWIW, I didn't get any really noticeable improvement with pgo builds, even on
> startup time, though i haven't formally verified the latter, and I have a SSD,
> so i cheat for startup. But that would suggest that there is no cpu-bound
> improvement in your faster startups, which in turn suggests there might be
> better ways to get these improvements (such as your reordering work).

Actually the good numbers produced by pgo in http://blog.mozilla.com/tglek/2010/04/12/squeezing-every-last-bit-of-performance-out-of-the-linux-toolchain/ have little to do with cpu time. -freorder-blocks-and-partition option changed the layout of the binary for the better. Any minor cpu-time benefits are just icing on my startup cake.
 
> 
> But benchmarks such as sunspider didn't end up running significantly faster
> even when running it during the profiling phase. Is it worth doubling the build
> time ?

For a 30-40% improvement in startup time, yes.

Comment 10

[Mike Hommey [:glandium]]

(In reply to comment #9)
> For a 30-40% improvement in startup time, yes.

Even when you can get the same improvement with tools that don't require to double the build time ?

Comment 11

[[e-mail] (:bhearsum)]

With most of our Linux builds happening on very fast hardware I don't think we should overly concern ourselves with doubling the build time. The last mozilla-central nightly on Linux took 15 minutes to compile. Even at 30 minutes to compile, Linux is not going to be our long-pole platform.

Comment 12

[(dormant account)]

After Benjamin grumbled about the 4.4 switch, I figured out what needs to be done to do PGO builds on x86/x86_64 with gcc 4.5 such that the resulting binaries run fine on older distributions.
So lets get GCC 4.5.0 installed and pgo turned on.

Comment 13

[Chris AtLee [:catlee]]

From Taras on IRC:

- Build with same switches as we did gcc 4.3.3: https://wiki.mozilla.org/ReferencePlatforms/Linux-CentOS-5.0#GCC_4.3.3

- No need to keep gcc headers around; just need binaries and libs (whatever 'make install' creates)

Comment 14

[(not currently active) Ted Mielczarek]

You really want to switch our builds to a compiler that was released not quite two weeks ago? Brave!

Comment 15

[(dormant account)]

(In reply to comment #14)
> You really want to switch our builds to a compiler that was released not quite
> two weeks ago? Brave!

Yup. This way we avoid another compiler change in the near future. This way we have a much better chance of having any fresh issues fixed quickly by gcc guys if they arise.

Comment 16

[Rail Aliiev [:rail]]

Attachment: compile-gcc-4.5.0.sh

Attaching the current GCC 4.5.0 compile script which prepares Puppet friendly tarball.

GCC 4.5 requires new version of gmp, mpfr and mpc libraries. To avoid any possible conflicts with te current toolchain I installed all of tools with --prefix=/tools/gcc-4.5.0 prefix and add wrappers for gcc/g++ with injected LD_LIBRARY_PATH. Any ideas how to avoid this hack without touching /etc/ld.so.conf and setting LD_LIBRARY_PATH?

Normal build (using wrappers) runs fine, but without the new libstdc++.so.6.0.14 linked to libstdc++.so.6.0 fails with:

./firefox-bin: /usr/lib32/libstdc++.so.6: version `GLIBCXX_3.4.14' not found (required by ./libxul.so)

Taras, is there any ETA for updated profiledbuild target for linux? IIRC it hasn't landed yet.

Comment 17

[(dormant account)]

(In reply to comment #16)
> Created an attachment (id=443099) [details]
> compile-gcc-4.5.0.sh
> 
> Attaching the current GCC 4.5.0 compile script which prepares Puppet friendly
> tarball.
> 
> GCC 4.5 requires new version of gmp, mpfr and mpc libraries. To avoid any
> possible conflicts with te current toolchain I installed all of tools with
> --prefix=/tools/gcc-4.5.0 prefix and add wrappers for gcc/g++ with injected
> LD_LIBRARY_PATH. Any ideas how to avoid this hack without touching
> /etc/ld.so.conf and setting LD_LIBRARY_PATH?
> 
> Normal build (using wrappers) runs fine, but without the new
> libstdc++.so.6.0.14 linked to libstdc++.so.6.0 fails with:
> 
> ./firefox-bin: /usr/lib32/libstdc++.so.6: version `GLIBCXX_3.4.14' not found
> (required by ./libxul.so)

Yeah, that's because of the new libstd++ dependency. Bug 561236 aims to address this. I need to investigate alternative workarounds. Will try to wrap that up this week.

> 
> Taras, is there any ETA for updated profiledbuild target for linux? IIRC it
> hasn't landed yet.

Ted, waiting for your review on this in bug 418866.

Comment 18

[Rail Aliiev [:rail]]

Attachment: compile-gcc-4.5.0.sh

Create tarballs ready for Puppet.
The binaries can be downloaded from http://people.mozilla.org/~raliiev/puppet/

Comment 19

[Rail Aliiev [:rail]]

Attachment: GCC 4.5.0 Puppet manifests

Install gcc450 as a secondary compiler for Centos platform. No symlink required.

Comment 20

[Rail Aliiev [:rail]]

Attachment: Combined patch for profiledbuild

Putting the combined patch from bugs mentioned by Taras just for the history. :)

I "tested" (just ran it :) ) the profiled build on i386 arch only (will try on amd64 these days). Worked fine on Ubuntu 8.04 (i386) and 10.04 (amd64 with lib32) with default libstdc++. Visually starts faster.

I used default nightly .mozconfig with the following changes:

CC="ccache /tools/gcc-4.5.0/bin/gcc"
CXX="ccache /tools/gcc-4.5.0/bin/g++"
mk_add_options  PROFILE_GEN_SCRIPT='$(PYTHON) @TOPSRCDIR@/_profile/pgo/profileserver.py'

Comment 21

[Rail Aliiev [:rail]]

amd64 build failed with the following error:

/tools/gcc-4.5.0/bin/g++ -o xptcstubs_x86_64_linux.o -c -I../../../../../../dist/stl_wrappers -I../../../../../../dist/system_wrappers -include ../../../../../../config/gcc_hidden.h -DMOZILLA_INTERNAL_API -DOSTYPE=\"Linux2.6.18-53.1.14\" -DOSARCH=Linux -DEXPORT_XPTC_API -D_IMPL_NS_COM -I./../.. -I./../../../../xptinfo/src  -I. -I. -I../../../../../../dist/include -I../../../../../../dist/include/nsprpub  -I/builds/slave/mozilla-central-linux64-nightly/build/dist/include/nspr -I/builds/slave/mozilla-central-linux64-nightly/build/dist/include/nss       -fPIC  -fno-rtti -fno-exceptions -Wall -Wpointer-arith -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wcast-align -Wno-invalid-offsetof -Wno-variadic-macros -Werror=return-type -Wno-long-long -pedantic -gdwarf-2 -fno-strict-aliasing -fshort-wchar -pthread -pipe  -DNDEBUG -DTRIMMED -gdwarf-2 -fprofile-use -Wcoverage-mismatch -freorder-blocks-and-partition -fprofile-correction  -Os -freorder-blocks -fomit-frame-pointer    -DMOZILLA_CLIENT -include ../../../../../../mozilla-config.h -Wp,-MD,.deps/xptcstubs_x86_64_linux.pp xptcstubs_x86_64_linux.cpp
{standard input}: Assembler messages:
{standard input}:7765: Error: can't resolve `.text.unlikely' {.text.unlikely section} - `.LCOLDB0' {.text section}
{standard input}:7927: Error: can't resolve `.text.unlikely' {.text.unlikely section} - `.LCOLDB0' {.text section}

Comment 22

[Rail Aliiev [:rail]]

GCC 4.5.0 tarballs for linux and linux64 platforms are in /N/centos5/dist.

Comment 23

[(dormant account)]

(In reply to comment #16)
> GCC 4.5 requires new version of gmp, mpfr and mpc libraries. To avoid any
> possible conflicts with te current toolchain I installed all of tools with
> --prefix=/tools/gcc-4.5.0 prefix and add wrappers for gcc/g++ with injected
> LD_LIBRARY_PATH. Any ideas how to avoid this hack without touching
> /etc/ld.so.conf and setting LD_LIBRARY_PATH?

No. Another unfun alternative is to make a mozg++ wrapper script with
LD_LIBRARY_PATH=... /path/to/g++ $@

> 
> Normal build (using wrappers) runs fine, but without the new
> libstdc++.so.6.0.14 linked to libstdc++.so.6.0 fails with:
> 
> ./firefox-bin: /usr/lib32/libstdc++.so.6: version `GLIBCXX_3.4.14' not found
> (required by ./libxul.so)

Apply the gcc patch in bug 561236 to fix this.


(In reply to comment #21)
> {standard input}: Assembler messages:
> {standard input}:7765: Error: can't resolve `.text.unlikely' {.text.unlikely
> section} - `.LCOLDB0' {.text section}
> {standard input}:7927: Error: can't resolve `.text.unlikely' {.text.unlikely
> section} - `.LCOLDB0' {.text section}

That's weird. I don't have a 64bit centos vm here. This *might* be due to having an ancient linker. Newer binutils might fix this.

Comment 24

[(dormant account)]

Adding LDFLAGS -static-libstdc++ to LDFLAGS seems to work fine(as an alternative to patching gcc). Infact, I don't even see any significant increase in size.

Comment 25

[Rail Aliiev [:rail]]

Attachment: compile-gcc-4.5.0.sh

(In reply to comment #24)
> Adding LDFLAGS -static-libstdc++ to LDFLAGS seems to work fine(as an
> alternative to patching gcc). Infact, I don't even see any significant increase
> in size.

Worked fine for i386. Increase in size is 417380 (11702238 vs 12119618).

I removed all vector/list changes from the previous "Combined patch for profiledbuild" (next one) for this build.

Snippet from .mozcofig:
export LDFLAGS="-static-libstdc++"

(In reply to comment #23)
> (In reply to comment #16)
> > GCC 4.5 requires new version of gmp, mpfr and mpc libraries. To avoid any
> > possible conflicts with te current toolchain I installed all of tools with
> > --prefix=/tools/gcc-4.5.0 prefix and add wrappers for gcc/g++ with injected
> > LD_LIBRARY_PATH. Any ideas how to avoid this hack without touching
> > /etc/ld.so.conf and setting LD_LIBRARY_PATH?

Setting LDFLAGS and BOOT_LDFLAGS to "-L/tools/gcc-4.5.0/lib -Wl,-rpath,/tools/gcc-4.5.0/lib" fixed the problem. No need to set LD_LIBRARY_PATH now. Tarballs for linux and linux64 put on /N/ and deployed using Pupet in staging.

> That's weird. I don't have a 64bit centos vm here. This *might* be due to
> having an ancient linker. Newer binutils might fix this.

i386 is more or less ready, let's try to hack this one. :)

Comment 26

[Rail Aliiev [:rail]]

Attachment: Combined patch for profiledbuild

Comment 27

[Rail Aliiev [:rail]]

Attachment: buildbot-configs changes

Use GCC 4.5.5 as default compiler for mozilla-central linux (32bit only) builds.
Adjust PROFILE_GEN_SCRIPT for nightly builds and add it for the release builds.

profiledbuild is not turned on by default, will be enabled later by tuning config.py.

Comment 28

[Rail Aliiev [:rail]]

Attachment: buildbot-configs changes

Added LDFLAGS for statical linking

Comment 29

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 443736 [details] [diff] [review]
buildbot-configs changes

I think we should convert the addonsmgr, places, electrolysis, and tracemonkey repos over too, since they're derived from mozilla-central.

Beware of the symlinks!

Comment 30

[Rail Aliiev [:rail]]

Attachment: buildbot-configs changes

s,@TOPSRCDIR@/build,@MOZ_OBJDIR@,

Comment 31

[(dormant account)]

Found a small problem with above.

Need to add PIC flags BOOT_CFLAGS=-fPIC and BOOT_LDFLAGS=-fPIC in order to produce useful binary with -static-libstdc++.

Otherwise libstdc++ causes libxul to contain text relocations which piss off selinux and prelink.

Comment 32

[Rail Aliiev [:rail]]

Comment on attachment 443599 [details] [diff] [review]
Combined patch for profiledbuild

landed to mozilla-central

Comment 33

[Rail Aliiev [:rail]]

Attachment: compile-gcc-4.5.0.sh

GCC 4.5.0 build script with updated flags.

Taras, could you take a look and check if the parameters and flags are OK.

The builds using this compiler pass all unittests (profiledbuild and debug build) in staging. If there is no stopper I'd like to deploy the current compiler build to the build slaves and enable it and profiledbuild for mozilla-central.

Comment 34

[(dormant account)]

Comment on attachment 445094 [details] [diff] [review]
compile-gcc-4.5.0.sh

>mkdir -p $pkg-$ver/gcc-obj-release
>export CFLAGS="-fPIC"
>export BOOT_CFLAGS="$CFLAGS"
>export LDFLAGS="-L$PREFIX/lib -Wl,-rpath,$PREFIX/lib -fPIC"
>export BOOT_LDFLAGS="$LDFLAGS"
>cd $pkg-$ver/gcc-obj-release
>../configure --prefix=$PREFIX --enable-__cxa_atexit --enable-languages=c,c++ \
>    --with-gmp=$PREFIX \
>    --with-mpfr=$PREFIX \
>    --with-mpc=$PREFIX
>make BOOT_LDFLAGS="$BOOT_LDFLAGS" LDFLAGS="$LDFLAGS" \
>    CFLAGS="$CFLAGS" BOOT_CFLAGS="$BOOT_CFLAGS" bootstrap
>make BOOT_LDFLAGS="$BOOT_LDFLAGS" LDFLAGS="$LDFLAGS" bootstrap

Don't need to pass these arguments to make, but it probably doesn't hurt. 

Looks good other than that.

Comment 35

[Rail Aliiev [:rail]]

Attachment: compile-gcc-4.5.0.sh

Bump version (moz1) to make puppet happy.

Keeping r+ by Taras

Comment 36

[Rail Aliiev [:rail]]

Comment on attachment 443310 [details] [diff] [review]
GCC 4.5.0 Puppet manifests

http://hg.mozilla.org/build/puppet-manifests/rev/367074f7877a
64bit part ignored

Comment 37

[Rail Aliiev [:rail]]

Attachment: GCC 4.5.0_moz1 Puppet manifests

Bump version, obsolete previous build, maintain gcc-4.5 symlink

Comment 38

[(dormant account)]

Comment on attachment 445161 [details] [diff] [review]
compile-gcc-4.5.0.sh


>mkdir -p $pkg-$ver/gcc-obj-release
>export CFLAGS="-fPIC"
>export BOOT_CFLAGS="$CFLAGS"
>export LDFLAGS="-L$PREFIX/lib -Wl,-rpath,$PREFIX/lib -fPIC"
>export BOOT_LDFLAGS="$LDFLAGS"
>cd $pkg-$ver/gcc-obj-release
>../configure --prefix=$PREFIX \
>    --enable-__cxa_atexit --enable-languages=c,c++ \
>    --with-gmp=$PREFIX \
>    --with-mpfr=$PREFIX \
>    --with-mpc=$PREFIX
>make BOOT_LDFLAGS="$BOOT_LDFLAGS" LDFLAGS="$LDFLAGS" \
>    CFLAGS="$CFLAGS" BOOT_CFLAGS="$BOOT_CFLAGS" bootstrap

Instead of passing -fPIC above, the correct way is to add --with-pic to configure options. This fixes the full gcc build here.

Comment 39

[(dormant account)]

(In reply to comment #38)

> 
> Instead of passing -fPIC above, the correct way is to add --with-pic to
> configure options. This fixes the full gcc build here.

Scratch that. --with-pic produces a weird libstdc++. export CXXFLAGS=-fPIC is the way to go.

Comment 40

[(dormant account)]

4.5 fixes a build failure with 4.3 in  bug 415563

Comment 41

[Wan-Teh Chang]

taras: do you have the details of the gcc bug in bug 415563
that is fixed in gcc 4.5?  I'd like to know under what conditions
that bug affects us, and whether the fix has been backported
to a gcc 4.3.x or 4.4.x release.  This info will enable me
to implement a workaround in NSPR.  Thanks.

Comment 42

[(dormant account)]

(In reply to comment #41)
> taras: do you have the details of the gcc bug in bug 415563
> that is fixed in gcc 4.5?  I'd like to know under what conditions
> that bug affects us, and whether the fix has been backported
> to a gcc 4.3.x or 4.4.x release.  This info will enable me
> to implement a workaround in NSPR.  Thanks.

No, all I did was try two different gcc versions(4.3.3 and 4.5). The stackoverflow link that blassey posted in bug 415563 indicates that -march=i486 on relevant files is a workaround.

Comment 43

[Rail Aliiev [:rail]]

Taras, sorry for the delay, I'm a bit busy by other tasks right now. :(

Would you mind if I ask you to try compiling gcc and test it on one of the our builders? I could ask IT to clone one.

Comment 44

[(dormant account)]

(In reply to comment #43)
> Taras, sorry for the delay, I'm a bit busy by other tasks right now. :(
> 
> Would you mind if I ask you to try compiling gcc and test it on one of the our
> builders? I could ask IT to clone one.

I've been testing in the buildvm, CXXFLAGS=-fPIC is all we need to get this going. Just need to confirm/deploy in production now

Comment 45

[Rail Aliiev [:rail]]

Build fails with CXXFLAGS=-fPIC set (for GCC). Tried to use the current version of binutils (2.20) but the result is the same.

/tools/gcc-4.5/bin/g++ -o xptcinvoke_gcc_x86_unix.o -c -I../../../../../../dist/stl_wrappers -I../../../../../../dist/system_wrappers -include /builds/slave/linux_build/build/config/gcc_hidden.h -DMOZILLA_INTERNAL_API -DOSTYPE=\"Linux2.6.18-53.1.19\" -DOSARCH=Linux -DEXPORT_XPTC_API -D_IMPL_NS_COM -I/builds/slave/linux_build/build/xpcom/reflect/xptcall/src/md/unix/../.. -I/builds/slave/linux_build/build/xpcom/reflect/xptcall/src/md/unix/../../../../xptinfo/src  -I/builds/slave/linux_build/build/xpcom/reflect/xptcall/src/md/unix -I. -I../../../../../../dist/include -I../../../../../../dist/include/nsprpub  -I/builds/slave/linux_build/build/obj-firefox/dist/include/nspr -I/builds/slave/linux_build/build/obj-firefox/dist/include/nss       -fPIC  -fno-rtti -fno-exceptions -Wall -Wpointer-arith -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wcast-align -Wno-invalid-offsetof -Wno-variadic-macros -Werror=return-type -Wno-long-long -pedantic -gdwarf-2 -fno-strict-aliasing -fshort-wchar -pthread -pipe  -DNDEBUG -DTRIMMED -gdwarf-2 -fprofile-use -fprofile-correction -Wcoverage-mismatch -freorder-blocks-and-partition -Os -freorder-blocks -fomit-frame-pointer    -DMOZILLA_CLIENT -include ../../../../../../mozilla-config.h -Wp,-MD,.deps/xptcinvoke_gcc_x86_unix.pp /builds/slave/linux_build/build/xpcom/reflect/xptcall/src/md/unix/xptcinvoke_gcc_x86_unix.cpp
{standard input}: Assembler messages:
{standard input}:200: Error: can't resolve `.text.unlikely' {.text.unlikely section} - `.LCOLDB0' {.text section}
make[10]: *** [xptcinvoke_gcc_x86_unix.o] Error 1

Comment 46

[(dormant account)]

Sorry forgot about that, I had a workaround patch in my queue for this. I filed bug 569137 to fix this.
That's not a binutils problem, it's something to do with how gcc layouts the sections. I'll file a gcc bug eventually.

Comment 47

[Rail Aliiev [:rail]]

Attachment: GCC 4.5.0 spec (including GMP, MPFR, MPC libraries)

A bit ugly spec:
* I included GMP, MPFR and MPC libraries into the package, so I need to install them to the target (/tools/gcc...) before gcc build. The libraries could be packaged separately, but we aren't going to use them for other packages.
* %install just copies files from --prefix to rpm's "right" directory to make rpmbuild happy

A binary produced by this package got OK from Taras.

Comment 48

[Rail Aliiev [:rail]]

Attachment: GCC 4.5.0 Puppet manifests

The manifest also removes the previous buggy gcc-4.5.0 version.

Comment 49

[[e-mail] (:bhearsum)]

Comment on attachment 448692 [details] [diff] [review]
GCC 4.5.0 Puppet manifests

Bear, can you review this?

Comment 50

[[e-mail] (:bhearsum)]

Comment on attachment 448687 [details] [diff] [review]
GCC 4.5.0 spec (including GMP, MPFR, MPC libraries)

Taras tells me the build instructions look good.

Looks OK to me, too.

Comment 51

[[e-mail] (:bhearsum)]

Comment on attachment 443967 [details] [diff] [review]
buildbot-configs changes

I think you need to use /tools/gcc-4.5 now, based on my reading of the spec and puppet manifests.

Comment 52

[Mike Taylor [:bear]]

Is this ready for testing on staging - do you have the rpm ready?

Comment 53

[Rail Aliiev [:rail]]

Attachment: GCC 4.5.0 spec (including GMP, MPFR, MPC libraries)

* Fixed --prefix (/tools/gcc-4.5) to eliminate Puppet symlink manipulation.
* Used 0moz1 release instead of moz1

Comment 54

[Rail Aliiev [:rail]]

Attachment: GCC 4.5.0 Puppet manifests

Use install_rpm

Comment 55

[Mike Taylor [:bear]]

Comment on attachment 449699 [details] [diff] [review]
GCC 4.5.0 spec (including GMP, MPFR, MPC libraries)

just realized your asking me to review the spec but I was seeing the puppet changes

Comment 56

[Rail Aliiev [:rail]]

Attachment: GCC 4.5.0 spec (including GMP, MPFR, MPC libraries)

Oops. Here is the spec.

Comment 57

[[e-mail] (:bhearsum)]

Comment on attachment 449703 [details] [diff] [review]
GCC 4.5.0 Puppet manifests

Last NFS remnant is being removed here, yay!

Comment 58

[[e-mail] (:bhearsum)]

Comment on attachment 449703 [details] [diff] [review]
GCC 4.5.0 Puppet manifests

changeset:   177:d7b96acef7ad

Comment 59

[[e-mail] (:bhearsum)]

Comment on attachment 449710 [details]
GCC 4.5.0 spec (including GMP, MPFR, MPC libraries)

changeset:   3:026e7ec61437

Comment 60

[[e-mail] (:bhearsum)]

I watched a 32 and 64-bit slave sync up successfully.

Comment 61

[Rail Aliiev [:rail]]

Attachment: buildbot-configs changes

Taras, could you take a look at these mozconfigs for linux and linux64?

Some notes.

* I synced linux and linux64 mozconfigs, they are the same (next step will be removing linux64 configs and symlinking them to linux? :) ).

* As a result there are some changes:
** debug build for linux64 got --enable-libxul
** release build for linux64 got --enable-tests
** Dropped "export ac_cv_have_visibility_class_bug=yes" (per comments set for gcc-4.2.2)

Comment 62

[(not currently active) Ted Mielczarek]

FYI:

(In reply to comment #61)
> ** debug build for linux64 got --enable-libxul
> ** release build for linux64 got --enable-tests

These are both default settings. The --enable-libxul thing changed when we merged the electrolysis code, because it doesn't build without libxul, so we changed debug builds to default to libxul. --enable-tests has been the default for a long time.

Comment 63

[(dormant account)]

Comment on attachment 450609 [details] [diff] [review]
buildbot-configs changes

I think Ted is the domain expert here.

Comment 64

[(not currently active) Ted Mielczarek]

Comment on attachment 450609 [details] [diff] [review]
buildbot-configs changes

>diff --git a/mozilla2-staging/linux/mozilla-central/codecoverage/mozconfig b/mozilla2-staging/linux/mozilla-central/codecoverage/mozconfig
>--- a/mozilla2-staging/linux/mozilla-central/codecoverage/mozconfig
>+++ b/mozilla2-staging/linux/mozilla-central/codecoverage/mozconfig
>@@ -1,20 +1,18 @@
> mk_add_options MOZ_MAKE_FLAGS="-j3"
>+mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py'

You don't really need this in the debug and codecoverage mozconfigs, since we're not going to do PGO builds there, but it also won't hurt anything by being there.

Looks fine otherwise. I should really file a bug about cleaning these up a bit. Since we got --enable-debug-symbols working correctly on trunk, we don't need the export CFLAGS="-gdwarf-2" bits, or the export MOZ_DEBUG_SYMBOLS=1 bits. Plus, as I mentioned before, --enable-tests is the default.

Comment 65

[Rail Aliiev [:rail]]

Attachment: buildbot-configs changes

(In reply to comment #64)
> (From update of attachment 450609 [details] [diff] [review])
> >diff --git a/mozilla2-staging/linux/mozilla-central/codecoverage/mozconfig b/mozilla2-staging/linux/mozilla-central/codecoverage/mozconfig
> >--- a/mozilla2-staging/linux/mozilla-central/codecoverage/mozconfig
> >+++ b/mozilla2-staging/linux/mozilla-central/codecoverage/mozconfig
> >@@ -1,20 +1,18 @@
> > mk_add_options MOZ_MAKE_FLAGS="-j3"
> >+mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py'
> 
> You don't really need this in the debug and codecoverage mozconfigs, since
> we're not going to do PGO builds there, but it also won't hurt anything by
> being there.

Removed it. Let's keep the mosconfigs clean. :) 

Keeping r+.

Comment 66

[Rail Aliiev [:rail]]

Attachment: buildbot-configs changes

Applied to mozilla-2.0 too. Keeping r+.

Comment 67

[Rail Aliiev [:rail]]

Comment on attachment 453331 [details] [diff] [review]
buildbot-configs changes

http://hg.mozilla.org/build/buildbot-configs/rev/d67416d28d95

Comment 68

[(dormant account)]

Just landed a 4.5 workaround that should avoid the regressions we saw. When is the next time we can try a 4.5 switch?

Comment 69

[Bill Lancashire]

I have been building Firefox (32-bit) on Linux using PGO successfully since Firefox version 3.0.

I have no problem with my most recent build of Firefox 3.6.6.

I have just had a preliminary attempt to build Firefox 4.0b1 and run into a minor problem:

Details:
source: pulled 'tar' downloaded from 'release candidate' folder on 'ftp site';
OS: Slackware Linux 12.1;
GCC version 4.2.3

Observation:
The PGO build process now proceed using '-fprofile-arcs/-fbranch-probabilities. instead of the correct process for GCC 4.2.3 which is '-fprofile-generate/fprofile-use'.

This has occurred because of the patch applied to 'configure.in' in BUG 418866 resulting in addition of GCC options (-fprofile-correction specifically) which is not compatible with GCC 4.2.3, therefore resulting in the process falling back to the option meant for GCC 3.3

See excerpt from 'configure.in':
dnl ========================================================
dnl Profile guided optimization
dnl ========================================================
dnl Test for profiling options
dnl Under gcc 3.3, use -fprofile-arcs/-fbranch-probabilities
dnl Under gcc 3.4+, use -fprofile-generate/-fprofile-use

I appreciate that future Mozilla builds will use GCC 4.5.x but if no amendment is made to 'configure'in' to allow for correct processing under GCC 4.2.3 then private builders will have problems as that GCC compiler will be treated like GCC 3.3.

Can anything be done to patch 'configure.in' to allow GCC 3.4+ to 4.2.3 to work as previously?.

Comment 70

[(dormant account)]

Attachment: gcc patch

We'd like this patch applied to help static analysis builds

Comment 71

[Rail Aliiev [:rail]]

Attachment: GCC 4.5.0 spec (including plugin_finish_decl.diff)

on mv-moz2-linux-ix-slave01:
Wrote: /home/cltbld/rpmbuild/SRPMS/gcc45-4.5.0-0moz2.src.rpm
Wrote: /home/cltbld/rpmbuild/RPMS/i386/gcc45-4.5.0-0moz2.i386.rpm

moz2-linux64-slave07:
Wrote: /usr/src/redhat/RPMS/x86_64/gcc45-4.5.0-0moz2.x86_64.rpm

Comment 72

[Rail Aliiev [:rail]]

Attachment: GCC 4.5.0 Puppet manifests

Bump RPM version

Comment 73

[Rail Aliiev [:rail]]

Comment on attachment 457806 [details] [diff] [review]
GCC 4.5.0 spec (including plugin_finish_decl.diff)

http://hg.mozilla.org/build/rpm-sources/rev/c8fe426b3bdb

Comment 74

[Rail Aliiev [:rail]]

Comment on attachment 457809 [details] [diff] [review]
GCC 4.5.0 Puppet manifests

http://hg.mozilla.org/build/puppet-manifests/rev/9b1b8bdd384a

Comment 75

[Rail Aliiev [:rail]]

Updated RPM package is being deployed and seen on some production and try machines. It should be available on all machines after the weekend.

Comment 76

[(dormant account)]

Is there an eta for setting 4.5 as the default compiler yet?

Comment 77

[Chris AtLee [:catlee]]

Have you done new builds with it yet?  It should be available on try now.  What are the performance impacts of it?

Comment 78

[(dormant account)]

(In reply to comment #77)
> Have you done new builds with it yet?  It should be available on try now.  What
> are the performance impacts of it?

I landed a workaround in bug 576399 which seems to bring perf back to 4.3 levels(according to try). Hard to say for sure until we try it.

Comment 79

[Chris AtLee [:catlee]]

Ok, let's try it early next week, assuming that works with the beta schedule.

Comment 80

[(dormant account)]

(In reply to comment #79)
> Ok, let's try it early next week, assuming that works with the beta schedule.

So my plan is to try my workaround. Hope that it works like it did on try server.

If it turns out to not work, I'd like to switch our linux compilation to -O2, take the regression on startup(win on everything else), then turn on PGO and hopefully win on startup too.

It will all be simple if 4.5 works as expected(with 576399 workaround) and no hard decisions on accepting regressions have to be taken.

Comment 81

[Rail Aliiev [:rail]]

Attachment: buildbot-configs changes

Refreshed mozconfigs.

Comment 82

[(dormant account)]

4.5.1rc came out last week. Lets wait a few days for 4.5.1 final.

Comment 83

[timeless]

bug 582593 indicates that gcc 4.5 is broken :)

Comment 84

[(dormant account)]

(In reply to comment #82)
> 4.5.1rc came out last week. Lets wait a few days for 4.5.1 final.

Ok, lets roll this out http://ftp.gnu.org/pub/gnu/gcc/gcc-4.5.1

Comment 85

[Rail Aliiev [:rail]]

(In reply to comment #84)
> (In reply to comment #82)
> > 4.5.1rc came out last week. Lets wait a few days for 4.5.1 final.
> 
> Ok, lets roll this out http://ftp.gnu.org/pub/gnu/gcc/gcc-4.5.1

Taras, do you still need the patch (attachment 457606 [details] [diff] [review])?

Comment 86

[(dormant account)]

(In reply to comment #85)

> Taras, do you still need the patch (attachment 457606 [details] [diff] [review])?

Yes please.

Comment 87

[Rail Aliiev [:rail]]

Attachment: GCC 4.5.1 spec

Comment 88

[Rail Aliiev [:rail]]

Attachment: GCC 4.5.1 Puppet manifests

Comment 89

[[e-mail] (:bhearsum)]

Taras, please compile GCC 4.5.1 + patches and do some testing with it before we deploy it. We're happy to do compiler updates with good reason, but rolling them out to the entire build pool is not the right way to do initial testing.

Comment 90

[(dormant account)]

(In reply to comment #89)
> Taras, please compile GCC 4.5.1 + patches and do some testing with it before we
> deploy it. We're happy to do compiler updates with good reason, but rolling
> them out to the entire build pool is not the right way to do initial testing.

I've run tests on 4.5.1 now, works as well as 4.3 for me.

Comment 91

[[e-mail] (:bhearsum)]

Thanks Taras

Comment 92

[Rail Aliiev [:rail]]

Comment on attachment 463080 [details] [diff] [review]
GCC 4.5.1 spec

http://hg.mozilla.org/build/rpm-sources/rev/ae595518a672

Comment 93

[Rail Aliiev [:rail]]

Comment on attachment 463082 [details] [diff] [review]
GCC 4.5.1 Puppet manifests

http://hg.mozilla.org/build/puppet-manifests/rev/4f06048ffe74

mpt-production-puppet, mv-production-puppet and staging-puppet servers are
updated as well. GCC 4.5.1 will be available on all slaves in day or two.

Comment 94

[(dormant account)]

This would be nice to have for firefox4 for the overall speedup and to facilitate jar reordering in bug 559961.

Comment 95

[Rail Aliiev [:rail]]

Attachment: buildbot-configs changes

Comment 96

[Rail Aliiev [:rail]]

Comment on attachment 468316 [details] [diff] [review]
buildbot-configs changes

http://hg.mozilla.org/build/buildbot-configs/rev/74860b24b577

Comment 97

[Rail Aliiev [:rail]]

Comment on attachment 468316 [details] [diff] [review]
buildbot-configs changes

Backed out due to failed "make test":
http://hg.mozilla.org/build/buildbot-configs/rev/9e22ddd81fa1

Comment 98

[(dormant account)]

Rail, are there instructions for turning your spec files to rpms? Alternatively, are there some src rpms for the 4.5.0/4.5.1 specs above?

Comment 99

[Rail Aliiev [:rail]]

(In reply to comment #98)
> Rail, are there instructions for turning your spec files to rpms?

Very short instructions are here: https://wiki.mozilla.org/ReleaseEngineering:Puppet:Usage#Building_RPMs

> Alternatively, are there some src rpms for the 4.5.0/4.5.1 specs above?

The latest one is here:
http://people.mozilla.org/~raliiev/gcc/gcc45-4.5.1-0moz1.src.rpm

If you want I can regenerate src.rpm for 4.5.0.

Comment 100

[(dormant account)]

Rail, please add the patch from http://gcc.gnu.org/bugzilla/show_bug.cgi?id=45623#c24 to our gcc 4.5.1. This appears to conclude our 4.5.1 issues on my machine. I would like to ensure that the same is true on try and switch to 4.5 asap. It's blocking bug 590181(otherwise we have to change how we build 4.3, but i'd rather not waste time on that if we can avoid it) and other perf fixes.

Comment 101

[Justin Lebar (not reading bugmail)]

(In reply to comment #9)
> Actually the good numbers produced by pgo in
> http://blog.mozilla.com/tglek/2010/04/12/squeezing-every-last-bit-of-performance-out-of-the-linux-toolchain/
> have little to do with cpu time. -freorder-blocks-and-partition option changed
> the layout of the binary for the better. Any minor cpu-time benefits are just
> icing on my startup cake.

This was PGO plus -Os, right?  I wonder if PGO would help more with -O3 (patch in bug 590181).

Comment 102

[(dormant account)]

(In reply to comment #101)
> (In reply to comment #9)
> > Actually the good numbers produced by pgo in
> > http://blog.mozilla.com/tglek/2010/04/12/squeezing-every-last-bit-of-performance-out-of-the-linux-toolchain/
> > have little to do with cpu time. -freorder-blocks-and-partition option changed
> > the layout of the binary for the better. Any minor cpu-time benefits are just
> > icing on my startup cake.
> 
> This was PGO plus -Os, right?  I wonder if PGO would help more with -O3 (patch
> in bug 590181).

-O3 would hurt startup due to more bloated code causing more pagefaults.


Rail, http://gcc.gnu.org/viewcvs/branches/gcc-4_5-branch/gcc/tree-ssa-structalias.c?r1=164430&r2=164429&view=patch&pathrev=164430 is the official gcc fix. It supersedes the patch I linked in comment 100.

Comment 103

[Justin Lebar (not reading bugmail)]

(In reply to comment #102)
> > This was PGO plus -Os, right?  I wonder if PGO would help more with -O3 (patch
> > in bug 590181).
> 
> -O3 would hurt startup due to more bloated code causing more pagefaults.

Right, this was the behavior we saw in bug 590181; -O3 was a Ts_cold regression, but sped everything else up.  But in that bug, we said we wanted -O3 despite the cold startup regression; my understanding is that we still intend to land it, once it's unblocked.

My suspicion is that PGO plus -O3 might give a bigger gain over -O3 than PGO plus -Os gives over -Os.  (Consider that adding -freorder-blocks to -Os is moving you closer to -O3.)  It's also possible that PGO plus -O3 might not hurt Ts_cold as much as non-PGO -O3, since perhaps PGO's optimizations reduce the cost of -O3's code bloat.

So my question is: Can we try this and collect data, perhaps in a separate bug?

Comment 104

[Rail Aliiev [:rail]]

Attachment: GCC 4.5.1 spec

Updated spec file + the official patch

Comment 105

[Rail Aliiev [:rail]]

Attachment: GCC 4.5.1 Puppet manifests

Comment 106

[Rail Aliiev [:rail]]

Comment on attachment 477257 [details] [diff] [review]
GCC 4.5.1 spec

http://hg.mozilla.org/build/rpm-sources/rev/7846aabb1b84

Comment 107

[Rail Aliiev [:rail]]

Comment on attachment 477259 [details] [diff] [review]
GCC 4.5.1 Puppet manifests

http://hg.mozilla.org/build/puppet-manifests/rev/578a04a78b5a

Comment 108

[Rail Aliiev [:rail]]

The package with the specified patch is being deployed and will be available on the build hosts (including try machines) within a couple of days. Happy testing! :)

Comment 109

[(dormant account)]

Ok, try confirms that gcc4.5 fixes did their job. Lets make 4.5 the default compiler.

Don't worry about a perf hit, once 4.5 sticks. Bug 590181 will make up for any -Os perf regressions in 4.5.

Comment 110

[Shawn Wilsher :sdwilsh]

(In reply to comment #109)
> Don't worry about a perf hit, once 4.5 sticks. Bug 590181 will make up for any
> -Os perf regressions in 4.5.
But we should probably post about the fact that a regression will happen to tree management and platform to make sure people are aware of what is going on.

Comment 111

[[e-mail]]

And this should also include a reply to the automated Talos postings when it sees the regressions, otherwise people will get jumpy.

Comment 112

[(dormant account)]

So 4.5 passed try except for opengl failures. I got
the following failing in an optimized build 
opengl
4998/80/270


and these in debug:
opengl
4998/125/280
I hope this is linux gl being broken and not.

Comment 113

[[e-mail]]

Is that over and above the existing RGL failures? Linux/Linux64 are both pretty pooched on RGL on the standard build.

Comment 114

[(dormant account)]

(In reply to comment #112)

> I hope this is linux gl being broken

Joe says "ignore", this means gcc 4.5 got a clean bill of health on try.
So now we need an eta from Build&Release.

Comment 115

[Chris AtLee [:catlee]]

As per today's development meeting, this is not going to happen for Beta 7.

Taras, can you talk with beltzner to schedule having this possibly included in Beta 8?  If you want to plead your case, come to the next Tuesday development meeting.

Also, Talos numbers are needed before this can land.

Comment 116

[(dormant account)]

jlebar, can you post your -O3 numbers with gcc 4.5?

Comment 117

[Justin Lebar (not reading bugmail)]

Yes, I'll kick off a run on try.

Comment 118

[Shawn Wilsher :sdwilsh]

(In reply to comment #117)
> Yes, I'll kick off a run on try.
We should get a few runs and then compare to a few good runs of parent changesets on mozilla-central with https://services.forerunnerdesigns.com/compare-talos/

Comment 119

[Rail Aliiev [:rail]]

Attachment: buildbot-configs changes (mozconfigs)

Comment 120

[(not currently active) Ted Mielczarek]

Does this bug actually cover enabling PGO on our Linux builds, or just getting everything ready on the build slave side? If the former, we'll need to block on bug 602245.

Comment 121

[Rail Aliiev [:rail]]

(In reply to comment #120)
> Does this bug actually cover enabling PGO on our Linux builds, or just getting
> everything ready on the build slave side? If the former, we'll need to block on
> bug 602245.

AFAIK, we want to do both but in 2 steps: 1) use gcc-4.5.x for non profiled builds, wait some days and see if there are unexpected failures; 2) enable PGO

Comment 122

[(dormant account)]

So now that b7 has branched when can we deploy 4.5?

Comment 123

[[e-mail] (:bhearsum)]

The b7 branch and trunk build with the same mozconfig. Can't do it at least until b7 ships.

Comment 124

[Shawn Wilsher :sdwilsh]

Also, still missing talos numbers from comment 115.

Comment 125

[Justin Lebar (not reading bugmail)]

For Talos results, see bug 590181 comment 68 and comment 69.  No ts_cold regression, which is a bit strange.  Its absence may be worth looking into.

Comment 126

[[e-mail]]

catlee, howcome this got bumped from P2 to P4?

I think we want this ASAP. Ideally, it'd be ready to go as soon as we ship b7. There is other stuff depending on this that we really want to get done.

Comment 127

[Chris AtLee [:catlee]]

(In reply to comment #126)
> catlee, howcome this got bumped from P2 to P4?
> 
> I think we want this ASAP. Ideally, it'd be ready to go as soon as we ship b7.
> There is other stuff depending on this that we really want to get done.

Because there's nothing actionable for us to do right now.  Everything is ready to go once we get the go-ahead.

Comment 128

[[e-mail]]

Aha -- awesome. Thanks. :)

Comment 129

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Bug 578880 seems to depend on this -- marking this a blocker.

Comment 130

[[e-mail]]

We've shipped b7. Are we good to go?

Comment 131

[Chris AtLee [:catlee]]

(In reply to comment #130)
> We've shipped b7. Are we good to go?

Hasn't quite shipped yet.  Build 1 is still in QA.

Comment 132

[[e-mail]]

Er, sorry. Meant tagged. :/

Comment 133

[Chris AtLee [:catlee]]

(132 comments, really?)

So...since we've got a b7 relbranch, and any future b7 fixes will go on the relbranch, I think we're ok to make the switch on trunk.

Taras, want to bring this up at Tuesday's platform meeting to schedule the switch-over?

Comment 134

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Is there an ETA for this?  This is currently blocking blocker work for Firefox 4 on Linux, not just turning on PGO.

Comment 135

[(not currently active) Ted Mielczarek]

This needs bug 609543 to be fixed, at least, before we can try again.

Comment 136

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Is gcc 4.5 already deployed to the build machines?  That is, can a mozconfig switch over to test whether bug 609543 is really being caused by gcc 4.5?

Comment 137

[Chris AtLee [:catlee]]

(In reply to comment #136)
> Is gcc 4.5 already deployed to the build machines?  That is, can a mozconfig
> switch over to test whether bug 609543 is really being caused by gcc 4.5?

Yes, it's been deployed for ages.  You can do a try build with a custom mozconfig to select gcc 4.5.

Comment 138

[[e-mail]]

OK, bug 609543 is fixed. It sounds like jlebar has found another couple failures though: https://bugzilla.mozilla.org/show_bug.cgi?id=590181#c107. Do those need to block deployment here?

Comment 139

[Mike Hommey [:glandium]]

Comment on attachment 480587 [details] [diff] [review]
buildbot-configs changes (mozconfigs)

>+export LDFLAGS="-static-libstdc++"

You will also need to set HOST_LDFLAGS to the same value. (but that'll need bug 628303 to land first)
This is currently a problem on seamonkey, which switched to 4.5 with this patch, since elfhack (bug 606145) landed.

Comment 140

[Chris AtLee [:catlee]]

gcc-4.5 is already installed on the linux machines; this bug is now tracking enabling it by default for building firefox.

Comment 141

[Mike Hommey [:glandium]]

Attachment: PoC to allow to run gcc 4.5 builds against older libstdc++

I figured a hackish way to avoid g++ 4.5 linking against libstdc++ symbols that are only available in libstdc++.so.6 from gcc 4.5. I went even further and got rid of symbols from gcc 4.2, making the resulting binaries work with the system-provided libstdc++.so.6 on buildbots, allowing to get rid of these LD_LIBRARY_PATHs:
http://hg.mozilla.org/build/buildbot-configs/file/ccf475f8cf9d/mozilla/config.py#l1301

It also avoids requiring to use -static-libstdc++, so that could taken off attachment 480587 [details] [diff] [review].

I'll try to build with g++ 4.6 to see if there will be any further symbols to hack around in the future with the current codebase. (Future changes to the codebase may require other symbols, so the use of STL should still be discouraged)

Try builds available here:
http://ftp.mozilla.org/pub/mozilla.org/firefox/tryserver-builds/[e-mail]-69d6499b5c8a/

As that was only for a test, I tucked the hack in a random file and didn't protect it in some #ifdef. Integration details should be figured out before switching to gcc 4.5, obviously.

Comment 142

[(not currently active) Ted Mielczarek]

When bug 641842 is fixed, we should try rolling this out only on the build-system branch.

Comment 143

[Mike Hommey [:glandium]]

Let's make both switches (gcc 4.5 + PGO) at the same time. I've heavily tested PGO on try and with gcc 4.5, it works great. We'll need to land bug 590181 first to get the best out of it (and I verified this doesn't make problems with gcc 4.3, except for a big size bump, that PGO greatly removes)

Comment 144

[Mike Hommey [:glandium]]

So, we're almost set. All I need is a r+ on bug 643690, which should be pretty quick, and we're ready to go.

mozconfig-wise, we need to change CC and CXX, we don't need to add -static-libstdc++ to LDFLAGS, and we need to add the following to enable PGO:
  mk_add_options MOZ_PGO=1
and refresh PROFILE_GEN_SCRIPT where it makes sense.

We need to land bug 643690 before or during the switch, and bug 590181 preferably after or during.
How and when can we plan this switch?

Comment 145

[Mike Hommey [:glandium]]

(In reply to comment #144)
> So, we're almost set. All I need is a r+ on bug 643690, which should be pretty
> quick, and we're ready to go.

... and 44 seconds later, the r+ was there. So we're ready :)

Comment 146

[Mike Hommey [:glandium]]

And now that I landed bug 650467, we'll also need:
  ac_add_options --enable-stdcxx-compat

Comment 147

[Mike Hommey [:glandium]]

Attachment: buildbot-config changes

This is a refreshed patch as per comment 144 and comment 146.

I'm however unsure we want to enable PGO on unittest and debug builds.

Note that these changes will need to propagate to branches as the current mozilla-central propagates to aurora, beta and release.

At some point, some branches like tracemonkey and build-system will also need to switch. I'm however wondering how to handle try...

Comment 148

[Chris AtLee [:catlee]]

Comment on attachment 526668 [details] [diff] [review]
buildbot-config changes

Debug builds probably shouldn't use PGO. We don't on Windows at least.
No need to update the unittest mozconfigs, they're not used any more.

At the least try should be updated at the same time. Other branches should probably all happen at the same time as well. Maybe a post to dev.planning warning folks this is happening would be appropriate?

Comment 149

[(not currently active) Ted Mielczarek]

I think it would be best to give this a test run on build-system first. If we break anything, then it won't affect work on m-c.

Comment 150

[Mike Hommey [:glandium]]

The problem with try is that if something before the landing of bug 643690 is pushed, it won't work.

Comment 151

[Mike Hommey [:glandium]]

I had a nasty idea for try:

if hg log -r 98094f3afb93... -l1 | grep -q ^$; then
CC=/tools/gcc-4.5/bin/gcc
(...)
else
CC=/tools/gcc-4.3.3/installed/bin/gcc
(...)
fi

there's maybe a better way to do this, but I don't know mercurial well enough. It checks that revision 98094f3afb93 is an ancestor of the current revision.

Comment 152

[Mike Hommey [:glandium]]

(In reply to comment #149)
> I think it would be best to give this a test run on build-system first. If we
> break anything, then it won't affect work on m-c.

Note that it doesn't look like build-system has a separate mozconfig at the moment.

Comment 153

[Mike Hommey [:glandium]]

Attachment: buildbot-config preparation

There appears to be a whole lot of stuff that are symbolic links to mozconfig-central configurations (e.g. mozilla-1.9.1, electrolysis...), we need to break these symbolic links first.

Comment 154

[Mike Hommey [:glandium]]

Attachment: buildbot-config changes

These are the actual changes to the buildbot configs.

Comment 155

[Mike Hommey [:glandium]]

(In reply to comment #154)
> Created attachment 528561 [details] [diff] [review]
> buildbot-config changes
> 
> These are the actual changes to the buildbot configs.

Note, I was wondering if we also need to change the "generic" ones.

Comment 156

[Mike Hommey [:glandium]]

Attachment: Mercurial hook

This is a mercurial hook that checks whether something that is pushed to try contains changeset 3a38a70b0e12 as an ancestor, which is necessary for builds to work properly with gcc 4.5. It does so only if the try push is targetted at linux, linux64 or all platforms.
When it detects a push that doesn't contain that changeset, it issues a warning, currently only "Warning message" ; this obviously needs appropriate wording. For instance, do we want it to be verbose and give the workaround, or simply link to some documentation?

Comment 157

[Mike Hommey [:glandium]]

Attachment: Mercurial hook

I will update the wiki accordingly (if I can)

Comment 158

[(not currently active) Ted Mielczarek]

Comment on attachment 528841 [details]
Mercurial hook

Seems reasonable enough. You'll need to file a Server Ops bug to get the hook deployed into the try repo. You can commit it to the hghooks repo to have a place to put it: http://hg.mozilla.org/hgcustom/hghooks . You should ask someone in IT to verify that argparse will work there first, I'm not particularly sure about that.

Comment 159

[Mike Hommey [:glandium]]

Landed mozconfigs changes:
http://hg.mozilla.org/build/buildbot-configs/rev/938ef97e76f8
http://hg.mozilla.org/build/buildbot-configs/rev/222b10545b48

Comment 160

[Mike Hommey [:glandium]]

Landed try repo hook:
http://hg.mozilla.org/hgcustom/hghooks/rev/2b48ff549ea9

Comment 161

[Mike Hommey [:glandium]]

It looks like it worked well (once the mozconfig were actually used on the
buildbots)

Comment 162

[Daniel Holbert [:dholbert]]

This might have broken Linux crash reports.  See bug 654595.

Comment 163

[Armen [:armenzg]]

We forgot to track it for Firefox 6.

Comment 164

[Johnny Stenback (:jst)]

Just to clarify, we're tracking this for 6 just to watch what happens here and possibly backing out this change for 6? Or is there additional work that we're still waiting on here beyond possibly backing things out?

Comment 165

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Is there any configuration that needs to migrate from mozilla-central to mozilla-aurora to mozilla-beta to mozilla-release in order for this to apply to releases?

Comment 166

[Justin Lebar (not reading bugmail)]

(In reply to comment #165)
> Is there any configuration that needs to migrate from mozilla-central to
> mozilla-aurora to mozilla-beta to mozilla-release in order for this to apply
> to releases?

Looks like the beta buildbot-config [1] will need to be updated to match the aurora/m-c config [2].

[1] http://hg.mozilla.org/build/buildbot-configs/file/e2da95567950/mozilla2/linux/mozilla-beta/release/mozconfig

[2] http://hg.mozilla.org/build/buildbot-configs/file/e2da95567950/mozilla2/linux/mozilla-aurora/release/mozconfig

Comment 167

[Mike Hommey [:glandium]]

The same thing as bug 658167 will need to happen on mozilla-beta and then on mozilla-release. That's why bug 558180 would really be helpful.