852745 - Build clang with gcc 4.7 headers

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Joshua Cranmer [:jcranmer]]

Once gcc 4.7 is rolled out to all linux platforms, we need to produce new binaries to get clang with them. The instructions for rolling at a new clang is as follows, according to espindola:
11:44 <espindola> but then, what you need is
11:44 <espindola> build the .tar.gz
11:44 <espindola> you can do it by just running the script in m-c
11:45 <espindola> on OS X
11:45 <espindola> and on linux
11:45 <espindola> the script will also output a tooltool manifest
11:46 <espindola> ask someone in releng (I suggest rail) to copy the .tar.gz 
                  files (which the script renames to their sha512 sum) to the 
                  tooltool http server 
11:46 <espindola> and send a try job with the modified tooltool manifests
11:46 <jcranmer> on which machines should the build-clang.py run?
11:47 <espindola> for OS X I think any 10.7 or newer should do
11:47 <espindola> for linux
11:47 <espindola> use a reference setup
11:47 <espindola> build reference that is
11:47 <espindola> https://wiki.mozilla.org/ReferencePlatforms
11:47 <espindola> except that it is out of date
11:47 <espindola> we use centos 6 now :-(
11:48 <espindola> I think all that I had to do was create a VM, install centos 
                  6 in it
11:48 <espindola> and get rail to give me the gcc packages that they uses
11:48 <espindola> use
11:48 <espindola> so that clang can be build to use those headers

The patch to make clang build with the 4.7 headers will be uploaded shortly...

Comment 1

[Joshua Cranmer [:jcranmer]]

Attachment: Fix clang

It appears that rail has been reviewed in the past for these kinds of fixes, so asking review for him.

The CLANG_IS_PRODUCTION change is necessary to get clang plugins working, which is necessary for DXR and the plugin stuff I'm working on (they currently get by using older versions of clang).

Comment 2

[Rail Aliiev [:rail]]

Comment on attachment 726942 [details] [diff] [review]
Fix clang

Actually, we need to fix the mismatch between the package name and the prefix (moz1 vs moz4).

Comment 3

[Rail Aliiev [:rail]]

Comment on attachment 726942 [details] [diff] [review]
Fix clang

Actually it matches the package to be deployed.

Comment 4

[John Hopkins (:jhopkins)]

I built clang but did not end up with a clang.tar.bz2 file.

I did have to slightly modify build-clang.py as follows to get it to run, but I don't believe that impacted the functionality:

perl -pi -e 's/, item_sort_key=key_sort//' build-clang.py
perl -pi -e 's/simplejson/json/g' build-clang.py

Here are the full steps I ran on bld-centos6-hp-003:

mock_mozilla -v -r mozilla-centos6-x86_64 --init
mock_mozilla -v -r mozilla-centos6-x86_64 --install gcc472_0moz1 mercurial svn mozilla-python27
mock_mozilla -r mozilla-centos6-x86_64 --shell
rm -rf /builds/slave/moz-toolchain
rm -rf /builds/slave/clang-build
mkdir -p /builds/slave/clang-build
cd /builds/slave/clang-build
hg clone http://hg.mozilla.org/mozilla-central
cd mozilla-central
curl 'https://bug852745.bugzilla.mozilla.org/attachment.cgi?id=726942' > clang.diff
hg patch --no-commit clang.diff
export CC=/tools/gcc-4.7.2-0moz1/bin/gcc
export CXX=/tools/gcc-4.7.2-0moz1/bin/g++
cd build/unix/build-clang
perl -pi -e 's/, item_sort_key=key_sort//' build-clang.py
perl -pi -e 's/simplejson/json/g' build-clang.py
time /tools/python27/bin/python2.7 build-clang.py

Rail, any ideas?

Comment 5

[Rail Aliiev [:rail]]

I have never built clang using that script. From what I see subprocess.call eats the output. I think, if you can get the output you can see what goes wrong.

BTW, did the script raise any exception? What was the exit code?

Also, export CC, CXX shouldn't affect the result because the scripts uses hardcoded paths.

Comment 6

[John Hopkins (:jhopkins)]

It turns out that clang.tar.bz2 gets renamed to a sha512 digest so the archive was in fact built but then renamed to e14ccefd965372a57c540647b2b99e21a4aa82f81a8b9a9e18dac7cba4c3436181bef0dfab8c51bcb5c343f504a693fdcfbe7d609f10291b5dd65ab059979d29

Comment 7

[John Hopkins (:jhopkins)]

Built and uploaded to tooltool.  Manifest file is here:

 http://runtime-binaries.pvt.build.mozilla.org/tooltool/sha512/cf0802c788b26d6f29ada54b986956417b0619a5850c9974ca14e89fc0f2bf5c5c38af7a50cf2d68faa809b1024190eda0154751c4c377657a6802ba50a30cc2

Comment 8

[Christian Holler (:decoder)]

Reopening this, since we still need to update the manifest in the tree to use the new Clang. I'm doing a try push right now to see if it unbreaks the ASan builds.

Comment 9

[John Hopkins (:jhopkins)]

Attachment: linux32 clang.manifest

clang linux32 build also uploaded to tooltool. Attaching the manifest here.

Comment 10

[Joshua Cranmer [:jcranmer]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/d2a4ceb1ee6c

Comment 11

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/d2a4ceb1ee6c