693605 - Upgrade clang version on slaves

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[Rail Aliiev [:rail]]

Rafael said me that we need to upgrade clang version on slaves.

Rafael, could you provide version/revision to be used?

Comment 1

[Rafael Ávila de Espíndola (:espindola) (not reading bugmail)]

Yes, I just started looking at 693323 and will give you a revision number once I have a patch for that one.

Comment 2

[Rafael Ávila de Espíndola (:espindola) (not reading bugmail)]

I just finished both a OS X and a linux build with clang revision 141703.

Comment 3

[Rail Aliiev [:rail]]

(In reply to Rafael Ávila de Espíndola (:espindola) from comment #2)
> I just finished both a OS X and a linux build with clang revision 141703.

Just to be clear. Should I upgrade clang to rev 141703?

Comment 4

[Rafael Ávila de Espíndola (:espindola) (not reading bugmail)]

Attachment: update clang

Yes, that should be a good revision. We can also use the opportunity to drop the hack we had for bug 671711.

Comment 5

[Rail Aliiev [:rail]]

Comment on attachment 566615 [details] [diff] [review]
update clang

Looks good. I'll take care of packaging and deploying. Thank a lot!

Comment 6

[Rail Aliiev [:rail]]

Hmmm, just failed on linux64:

llvm[4]: Compiling DriverOptions.cpp for Release+Asserts build
llvm[4]: Compiling HostInfo.cpp for Release+Asserts build
/usr/src/redhat/SOURCES/clang/lib/Frontend/InitHeaderSearch.cpp: In member function 'void<unnamed>::InitHeaderSearch::AddMinGWCPlusPlusIn
cludePaths(llvm::StringRef, llvm::StringRef, llvm::StringRef)':
/usr/src/redhat/SOURCES/clang/lib/Frontend/InitHeaderSearch.cpp:195:1: error: unrecognizable insn:
(insn 318 317 46 2 /usr/src/redhat/SOURCES/llvm/include/llvm/ADT/Twine.h:180 (set (reg:DI 23 xmm2)
        (plus:DI (reg:DI 23 xmm2)
            (mem/u/c/i:DI (symbol_ref/u:DI ("*.LC22") [flags 0x2]) [0 S8 A64]))) -1 (expr_list:REG_EQUIV (plus:DI (reg/f:DI 7 sp)
            (mem/u/c/i:DI (symbol_ref/u:DI ("*.LC22") [flags 0x2]) [0 S8 A64]))
        (nil)))
/usr/src/redhat/SOURCES/clang/lib/Frontend/InitHeaderSearch.cpp:195:1: internal compiler error: in extract_insn, at recog.c:2103
Please submit a full bug report, 
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
/bin/rm: cannot remove `/usr/src/redhat/BUILD/clang-3.0-r141703.moz0/tools/clang/lib/Frontend/Release+Asserts/InitHeaderSearch.d.tmp': No
 such file or directory
make[4]: *** [/usr/src/redhat/BUILD/clang-3.0-r141703.moz0/tools/clang/lib/Frontend/Release+Asserts/InitHeaderSearch.o] Error 1

Comment 7

[Rail Aliiev [:rail]]

BTW, linux32 build is OK.

Comment 8

[Rail Aliiev [:rail]]

Build on darwin9 (10.5) failed as well:

llvm[4]: Compiling Diagnostic.cpp for Release+Asserts build
/Users/cltbld/src/clang/clang/lib/Basic/../../include/clang/Basic/Diagnostic.h:590: error: 'clang::DiagnosticsEngine::<anonymous enum> clang::DiagnosticsEngine::MaxArguments' is private
/Users/cltbld/src/clang/clang/lib/Basic/../../include/clang/Basic/PartialDiagnostic.h:36: error: within this context
/Users/cltbld/src/clang/clang/lib/Basic/../../include/clang/Basic/Diagnostic.h:621: error: 'clang::DiagnosticsEngine::<anonymous enum> clang::DiagnosticsEngine::MaxFixItHints' is private
/Users/cltbld/src/clang/clang/lib/Basic/../../include/clang/Basic/PartialDiagnostic.h:68: error: within this context
make[4]: *** [/Users/cltbld/src/clang/build/tools/clang/lib/Basic/Release+Asserts/Diagnostic.o] Error 1
make[3]: *** [Basic/.makeall] Error 2
make[2]: *** [all] Error 1
make[1]: *** [clang/.makeall] Error 2
make: *** [all] Error 1

10.6 is still compiling.

Comment 9

[Rail Aliiev [:rail]]

10.6 build of clang has passed, no problems.

Comment 10

[Rafael Ávila de Espíndola (:espindola) (not reading bugmail)]

(In reply to Rail Aliiev [:rail] from comment #8)
> Build on darwin9 (10.5) failed as well:

What compiler is being used? If gcc 4.0 (the default on 10.5), can you try gcc 4.2? Thanks.

Comment 11

[Rafael Ávila de Espíndola (:espindola) (not reading bugmail)]

(In reply to Rail Aliiev [:rail] from comment #6)
> Hmmm, just failed on linux64:
> 
> llvm[4]: Compiling DriverOptions.cpp for Release+Asserts build
> llvm[4]: Compiling HostInfo.cpp for Release+Asserts build
> /usr/src/redhat/SOURCES/clang/lib/Frontend/InitHeaderSearch.cpp: In member
> function 'void<unnamed>::InitHeaderSearch::AddMinGWCPlusPlusIn
> cludePaths(llvm::StringRef, llvm::StringRef, llvm::StringRef)':
> /usr/src/redhat/SOURCES/clang/lib/Frontend/InitHeaderSearch.cpp:195:1:
> error: unrecognizable insn:
> (insn 318 317 46 2 /usr/src/redhat/SOURCES/llvm/include/llvm/ADT/Twine.h:180
> (set (reg:DI 23 xmm2)
>         (plus:DI (reg:DI 23 xmm2)
>             (mem/u/c/i:DI (symbol_ref/u:DI ("*.LC22") [flags 0x2]) [0 S8
> A64]))) -1 (expr_list:REG_EQUIV (plus:DI (reg/f:DI 7 sp)
>             (mem/u/c/i:DI (symbol_ref/u:DI ("*.LC22") [flags 0x2]) [0 S8
> A64]))
>         (nil)))

Now that is fun! We are using /tools/gcc-4.5/bin/gcc, right? So our gcc fails to build clang :-(

I can try to figure out what the problem in gcc is, but maybe the easiest thing to do is use the old clang binary. I will upload a try patch.

Comment 12

[Rafael Ávila de Espíndola (:espindola) (not reading bugmail)]

Attachment: build clang with clang

Let me know if this fixes the build on linux64.
Thanks.

Comment 13

[Chris AtLee [:catlee]]

hm...this won't be reproducible. we need to be able to build clang with gcc.

Comment 14

[Rafael Ávila de Espíndola (:espindola) (not reading bugmail)]

(In reply to Chris AtLee [:catlee] from comment #13)
> hm...this won't be reproducible. we need to be able to build clang with gcc.

The traditional way of doing this with compilers is to do a bootstrap, i.e., compile clang with the just compiled clang itself. Would that be OK?

Comment 15

[Rafael Ávila de Espíndola (:espindola) (not reading bugmail)]

Attachment: updated spec file for doing a bootstrap

This is what I am testing to try to build clang on the bots. We first build with -O0 to avoid the gcc bug and then use that clang to build the one we install.

Comment 16

[Rail Aliiev [:rail]]

Working on this right now...

Comment 17

[Rafael Ávila de Espíndola (:espindola) (not reading bugmail)]

Attachment: bootstrap clang

This is the patch I am testing on the bots. I had to implement -static-libgcc and -static-libstdc++ in clang and I also fixed a small lib search bug, that is why the revision is newer.

The spec file now builds a first stage with gcc -O0 to avoid a bug and then uses that to build an optimized clang.

Comment 18

[Rail Aliiev [:rail]]

Comment on attachment 567615 [details] [diff] [review]
bootstrap clang

Review of attachment 567615 [details] [diff] [review]:
-----------------------------------------------------------------

Rafael,

Thanks for the patch.

Do you think that for Mac we should use the same approach with stage1/stage2?

::: clang/centos5-i686/clang.spec
@@ +58,4 @@
>  %install
>  install -d -m 755 $RPM_BUILD_ROOT
>  mkdir -p $RPM_BUILD_ROOT/%{install_dir}
>  cd %{toplevel_dir}

Looks like it should be "cd %{toplevel_dir}/stage2". Building with this change. I'll let you know when it compiles or fails.

Comment 19

[Rail Aliiev [:rail]]

Comment on attachment 567615 [details] [diff] [review]
bootstrap clang

r+ if you fix "cd %{toplevel_dir}" to "cd %{toplevel_dir}/stage2". Both linux and linux64 builds have been finished w/o any problems.

Comment 20

[Rail Aliiev [:rail]]

FTR, for macs I used the following script:
======================================
# in ~/src/clang
REPO_REV=142296

svn co -r $REPO_REV http://llvm.org/svn/llvm-project/llvm/trunk llvm
svn co -r $REPO_REV http://llvm.org/svn/llvm-project/cfe/trunk clang

cd llvm/tools/
ln -s ../../clang
cd ../../
mkdir build
cd build
../llvm/configure --enable-optimized --prefix=/tools/clang-3.0 \
 CC=gcc-4.2 CXX=g++-4.2
make -j8

sudo rm -rf /tools/clang-3.0
sudo make install
cd ../../puppet-manifests/ # from http://hg.mozilla.org/build/puppet-manifests
sudo ./create-dmg.sh /tools/clang-3.0 clang-3.0-r$REPO_REV.moz0 clang /tools
===============================================

Comment 21

[Rail Aliiev [:rail]]

Attachment: puppet-manifests

Comment 22

[Chris AtLee [:catlee]]

Comment on attachment 567705 [details] [diff] [review]
puppet-manifests

Review of attachment 567705 [details] [diff] [review]:
-----------------------------------------------------------------

Does this completely overwrite the old version?

Comment 23

[Rafael Ávila de Espíndola (:espindola) (not reading bugmail)]

Comment on attachment 567615 [details] [diff] [review]
bootstrap clang

http://hg.mozilla.org/build/rpm-sources/rev/9cddfe8fe118

Comment 24

[Rail Aliiev [:rail]]

(In reply to Chris AtLee [:catlee] from comment #22)
> Does this completely overwrite the old version?

It shouldn't be an issue for linux, since we use RPM packages. I'll test the DMG packages today.

BTW, I put the binaries here: http://people.mozilla.org/~raliiev/clang/

Comment 25

[Rail Aliiev [:rail]]

(In reply to Rail Aliiev [:rail] from comment #24)
> It shouldn't be an issue for linux, since we use RPM packages. I'll test the
> DMG packages today.

Worked fine. I'm going to deploy the packages in production today as well if there is no objection.

Comment 26

[Rail Aliiev [:rail]]

Comment on attachment 567705 [details] [diff] [review]
puppet-manifests

http://hg.mozilla.org/build/puppet-manifests/rev/846e33a3b221

Comment 27

[Rail Aliiev [:rail]]

All puppet masters have been updated. I'll check the overall status tomorrow.

Comment 28

[Rail Aliiev [:rail]]

Attachment: update revision

rev142537 should be used to avoid ASM parser bug.

Comment 29

[Rail Aliiev [:rail]]

Comment on attachment 568149 [details] [diff] [review]
update revision

http://hg.mozilla.org/build/rpm-sources/rev/4304b0b21065

Comment 30

[Rail Aliiev [:rail]]

Yay! rev 142537 worked fine for linux opt build. I put the binaries here: http://people.mozilla.org/~raliiev/clang/

I have to switch to other tasks, more updates (deployment, new platforms, debug builds, etc) on this one early next week.

Thanks for the help!

Comment 31

[Rail Aliiev [:rail]]

Attachment: puppet-manifests

use r142537

Comment 32

[Rail Aliiev [:rail]]

Comment on attachment 570234 [details] [diff] [review]
puppet-manifests

http://hg.mozilla.org/build/puppet-manifests/rev/1d9532043923

Comment 33

[Rail Aliiev [:rail]]

The updated package should be deployed by now.

Comment 34

[Rafael Ávila de Espíndola (:espindola) (not reading bugmail)]

I did a try run some time ago:
https://tbpl.mozilla.org/?tree=Try&rev=fe13a5c6469f

The main issue I have already debugged is breakpad failing to parse clang's debug output. I am working on it.