Last Comment Bug 693605 - Upgrade clang version on slaves
: Upgrade clang version on slaves
Status: RESOLVED FIXED
[buildslaves][clang]
:
Product: Release Engineering
Classification: Other
Component: Other (show other bugs)
: other
: x86 Linux
: P2 normal (vote)
: ---
Assigned To: Rail Aliiev [:rail]
:
Mentors:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-10-11 08:01 PDT by Rail Aliiev [:rail]
Modified: 2013-08-12 21:54 PDT (History)
2 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
update clang (2.34 KB, patch)
2011-10-12 13:18 PDT, Rafael Ávila de Espíndola (:espindola) (not reading bugmail)
rail: review+
respindola: checked‑in+
Details | Diff | Review
build clang with clang (869 bytes, text/plain)
2011-10-14 09:27 PDT, Rafael Ávila de Espíndola (:espindola) (not reading bugmail)
no flags Details
updated spec file for doing a bootstrap (1.90 KB, text/plain)
2011-10-14 20:17 PDT, Rafael Ávila de Espíndola (:espindola) (not reading bugmail)
no flags Details
bootstrap clang (1.79 KB, patch)
2011-10-17 16:29 PDT, Rafael Ávila de Espíndola (:espindola) (not reading bugmail)
rail: review+
respindola: checked‑in+
Details | Diff | Review
puppet-manifests (2.16 KB, patch)
2011-10-18 04:03 PDT, Rail Aliiev [:rail]
catlee: review+
rail: checked‑in+
Details | Diff | Review
update revision (378 bytes, patch)
2011-10-19 12:21 PDT, Rail Aliiev [:rail]
respindola: review+
rail: checked‑in+
Details | Diff | Review
puppet-manifests (2.16 KB, patch)
2011-10-28 06:11 PDT, Rail Aliiev [:rail]
catlee: review+
rail: checked‑in+
Details | Diff | Review

Description Rail Aliiev [:rail] 2011-10-11 08:01:10 PDT
Rafael said me that we need to upgrade clang version on slaves.

Rafael, could you provide version/revision to be used?
Comment 1 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2011-10-11 08:33:22 PDT
Yes, I just started looking at 693323 and will give you a revision number once I have a patch for that one.
Comment 2 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2011-10-11 14:28:29 PDT
I just finished both a OS X and a linux build with clang revision 141703.
Comment 3 Rail Aliiev [:rail] 2011-10-11 16:05:14 PDT
(In reply to Rafael Ávila de Espíndola (:espindola) from comment #2)
> I just finished both a OS X and a linux build with clang revision 141703.

Just to be clear. Should I upgrade clang to rev 141703?
Comment 4 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2011-10-12 13:18:35 PDT
Created attachment 566615 [details] [diff] [review]
update clang

Yes, that should be a good revision. We can also use the opportunity to drop the hack we had for bug 671711.
Comment 5 Rail Aliiev [:rail] 2011-10-12 13:24:10 PDT
Comment on attachment 566615 [details] [diff] [review]
update clang

Looks good. I'll take care of packaging and deploying. Thank a lot!
Comment 6 Rail Aliiev [:rail] 2011-10-13 07:55:02 PDT
Hmmm, just failed on linux64:

llvm[4]: Compiling DriverOptions.cpp for Release+Asserts build
llvm[4]: Compiling HostInfo.cpp for Release+Asserts build
/usr/src/redhat/SOURCES/clang/lib/Frontend/InitHeaderSearch.cpp: In member function 'void<unnamed>::InitHeaderSearch::AddMinGWCPlusPlusIn
cludePaths(llvm::StringRef, llvm::StringRef, llvm::StringRef)':
/usr/src/redhat/SOURCES/clang/lib/Frontend/InitHeaderSearch.cpp:195:1: error: unrecognizable insn:
(insn 318 317 46 2 /usr/src/redhat/SOURCES/llvm/include/llvm/ADT/Twine.h:180 (set (reg:DI 23 xmm2)
        (plus:DI (reg:DI 23 xmm2)
            (mem/u/c/i:DI (symbol_ref/u:DI ("*.LC22") [flags 0x2]) [0 S8 A64]))) -1 (expr_list:REG_EQUIV (plus:DI (reg/f:DI 7 sp)
            (mem/u/c/i:DI (symbol_ref/u:DI ("*.LC22") [flags 0x2]) [0 S8 A64]))
        (nil)))
/usr/src/redhat/SOURCES/clang/lib/Frontend/InitHeaderSearch.cpp:195:1: internal compiler error: in extract_insn, at recog.c:2103
Please submit a full bug report, 
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
/bin/rm: cannot remove `/usr/src/redhat/BUILD/clang-3.0-r141703.moz0/tools/clang/lib/Frontend/Release+Asserts/InitHeaderSearch.d.tmp': No
 such file or directory
make[4]: *** [/usr/src/redhat/BUILD/clang-3.0-r141703.moz0/tools/clang/lib/Frontend/Release+Asserts/InitHeaderSearch.o] Error 1
Comment 7 Rail Aliiev [:rail] 2011-10-13 07:56:56 PDT
BTW, linux32 build is OK.
Comment 8 Rail Aliiev [:rail] 2011-10-13 08:50:47 PDT
Build on darwin9 (10.5) failed as well:

llvm[4]: Compiling Diagnostic.cpp for Release+Asserts build
/Users/cltbld/src/clang/clang/lib/Basic/../../include/clang/Basic/Diagnostic.h:590: error: 'clang::DiagnosticsEngine::<anonymous enum> clang::DiagnosticsEngine::MaxArguments' is private
/Users/cltbld/src/clang/clang/lib/Basic/../../include/clang/Basic/PartialDiagnostic.h:36: error: within this context
/Users/cltbld/src/clang/clang/lib/Basic/../../include/clang/Basic/Diagnostic.h:621: error: 'clang::DiagnosticsEngine::<anonymous enum> clang::DiagnosticsEngine::MaxFixItHints' is private
/Users/cltbld/src/clang/clang/lib/Basic/../../include/clang/Basic/PartialDiagnostic.h:68: error: within this context
make[4]: *** [/Users/cltbld/src/clang/build/tools/clang/lib/Basic/Release+Asserts/Diagnostic.o] Error 1
make[3]: *** [Basic/.makeall] Error 2
make[2]: *** [all] Error 1
make[1]: *** [clang/.makeall] Error 2
make: *** [all] Error 1

10.6 is still compiling.
Comment 9 Rail Aliiev [:rail] 2011-10-13 09:21:55 PDT
10.6 build of clang has passed, no problems.
Comment 10 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2011-10-14 09:16:23 PDT
(In reply to Rail Aliiev [:rail] from comment #8)
> Build on darwin9 (10.5) failed as well:

What compiler is being used? If gcc 4.0 (the default on 10.5), can you try gcc 4.2? Thanks.
Comment 11 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2011-10-14 09:25:58 PDT
(In reply to Rail Aliiev [:rail] from comment #6)
> Hmmm, just failed on linux64:
> 
> llvm[4]: Compiling DriverOptions.cpp for Release+Asserts build
> llvm[4]: Compiling HostInfo.cpp for Release+Asserts build
> /usr/src/redhat/SOURCES/clang/lib/Frontend/InitHeaderSearch.cpp: In member
> function 'void<unnamed>::InitHeaderSearch::AddMinGWCPlusPlusIn
> cludePaths(llvm::StringRef, llvm::StringRef, llvm::StringRef)':
> /usr/src/redhat/SOURCES/clang/lib/Frontend/InitHeaderSearch.cpp:195:1:
> error: unrecognizable insn:
> (insn 318 317 46 2 /usr/src/redhat/SOURCES/llvm/include/llvm/ADT/Twine.h:180
> (set (reg:DI 23 xmm2)
>         (plus:DI (reg:DI 23 xmm2)
>             (mem/u/c/i:DI (symbol_ref/u:DI ("*.LC22") [flags 0x2]) [0 S8
> A64]))) -1 (expr_list:REG_EQUIV (plus:DI (reg/f:DI 7 sp)
>             (mem/u/c/i:DI (symbol_ref/u:DI ("*.LC22") [flags 0x2]) [0 S8
> A64]))
>         (nil)))

Now that is fun! We are using /tools/gcc-4.5/bin/gcc, right? So our gcc fails to build clang :-(

I can try to figure out what the problem in gcc is, but maybe the easiest thing to do is use the old clang binary. I will upload a try patch.
Comment 12 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2011-10-14 09:27:03 PDT
Created attachment 567113 [details]
build clang with clang

Let me know if this fixes the build on linux64.
Thanks.
Comment 13 Chris AtLee [:catlee] 2011-10-14 09:28:48 PDT
hm...this won't be reproducible. we need to be able to build clang with gcc.
Comment 14 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2011-10-14 09:43:35 PDT
(In reply to Chris AtLee [:catlee] from comment #13)
> hm...this won't be reproducible. we need to be able to build clang with gcc.

The traditional way of doing this with compilers is to do a bootstrap, i.e., compile clang with the just compiled clang itself. Would that be OK?
Comment 15 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2011-10-14 20:17:25 PDT
Created attachment 567248 [details]
updated spec file for doing a bootstrap

This is what I am testing to try to build clang on the bots. We first build with -O0 to avoid the gcc bug and then use that clang to build the one we install.
Comment 16 Rail Aliiev [:rail] 2011-10-17 08:14:09 PDT
Working on this right now...
Comment 17 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2011-10-17 16:29:48 PDT
Created attachment 567615 [details] [diff] [review]
bootstrap clang

This is the patch I am testing on the bots. I had to implement -static-libgcc and -static-libstdc++ in clang and I also fixed a small lib search bug, that is why the revision is newer.

The spec file now builds a first stage with gcc -O0 to avoid a bug and then uses that to build an optimized clang.
Comment 18 Rail Aliiev [:rail] 2011-10-17 19:07:54 PDT
Comment on attachment 567615 [details] [diff] [review]
bootstrap clang

Review of attachment 567615 [details] [diff] [review]:
-----------------------------------------------------------------

Rafael,

Thanks for the patch.

Do you think that for Mac we should use the same approach with stage1/stage2?

::: clang/centos5-i686/clang.spec
@@ +58,4 @@
>  %install
>  install -d -m 755 $RPM_BUILD_ROOT
>  mkdir -p $RPM_BUILD_ROOT/%{install_dir}
>  cd %{toplevel_dir}

Looks like it should be "cd %{toplevel_dir}/stage2". Building with this change. I'll let you know when it compiles or fails.
Comment 19 Rail Aliiev [:rail] 2011-10-18 03:59:33 PDT
Comment on attachment 567615 [details] [diff] [review]
bootstrap clang

r+ if you fix "cd %{toplevel_dir}" to "cd %{toplevel_dir}/stage2". Both linux and linux64 builds have been finished w/o any problems.
Comment 20 Rail Aliiev [:rail] 2011-10-18 04:02:25 PDT
FTR, for macs I used the following script:
======================================
# in ~/src/clang
REPO_REV=142296

svn co -r $REPO_REV http://llvm.org/svn/llvm-project/llvm/trunk llvm
svn co -r $REPO_REV http://llvm.org/svn/llvm-project/cfe/trunk clang

cd llvm/tools/
ln -s ../../clang
cd ../../
mkdir build
cd build
../llvm/configure --enable-optimized --prefix=/tools/clang-3.0 \
 CC=gcc-4.2 CXX=g++-4.2
make -j8

sudo rm -rf /tools/clang-3.0
sudo make install
cd ../../puppet-manifests/ # from http://hg.mozilla.org/build/puppet-manifests
sudo ./create-dmg.sh /tools/clang-3.0 clang-3.0-r$REPO_REV.moz0 clang /tools
===============================================
Comment 21 Rail Aliiev [:rail] 2011-10-18 04:03:34 PDT
Created attachment 567705 [details] [diff] [review]
puppet-manifests
Comment 22 Chris AtLee [:catlee] 2011-10-18 05:33:25 PDT
Comment on attachment 567705 [details] [diff] [review]
puppet-manifests

Review of attachment 567705 [details] [diff] [review]:
-----------------------------------------------------------------

Does this completely overwrite the old version?
Comment 23 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2011-10-18 05:57:00 PDT
Comment on attachment 567615 [details] [diff] [review]
bootstrap clang

http://hg.mozilla.org/build/rpm-sources/rev/9cddfe8fe118
Comment 24 Rail Aliiev [:rail] 2011-10-18 05:59:51 PDT
(In reply to Chris AtLee [:catlee] from comment #22)
> Does this completely overwrite the old version?

It shouldn't be an issue for linux, since we use RPM packages. I'll test the DMG packages today.

BTW, I put the binaries here: http://people.mozilla.org/~raliiev/clang/
Comment 25 Rail Aliiev [:rail] 2011-10-18 06:33:44 PDT
(In reply to Rail Aliiev [:rail] from comment #24)
> It shouldn't be an issue for linux, since we use RPM packages. I'll test the
> DMG packages today.

Worked fine. I'm going to deploy the packages in production today as well if there is no objection.
Comment 26 Rail Aliiev [:rail] 2011-10-18 11:25:06 PDT
Comment on attachment 567705 [details] [diff] [review]
puppet-manifests

http://hg.mozilla.org/build/puppet-manifests/rev/846e33a3b221
Comment 27 Rail Aliiev [:rail] 2011-10-18 11:48:20 PDT
All puppet masters have been updated. I'll check the overall status tomorrow.
Comment 28 Rail Aliiev [:rail] 2011-10-19 12:21:11 PDT
Created attachment 568149 [details] [diff] [review]
update revision

rev142537 should be used to avoid ASM parser bug.
Comment 29 Rail Aliiev [:rail] 2011-10-19 12:32:20 PDT
Comment on attachment 568149 [details] [diff] [review]
update revision

http://hg.mozilla.org/build/rpm-sources/rev/4304b0b21065
Comment 30 Rail Aliiev [:rail] 2011-10-20 08:59:03 PDT
Yay! rev 142537 worked fine for linux opt build. I put the binaries here: http://people.mozilla.org/~raliiev/clang/

I have to switch to other tasks, more updates (deployment, new platforms, debug builds, etc) on this one early next week.

Thanks for the help!
Comment 31 Rail Aliiev [:rail] 2011-10-28 06:11:42 PDT
Created attachment 570234 [details] [diff] [review]
puppet-manifests

use r142537
Comment 32 Rail Aliiev [:rail] 2011-10-31 07:16:42 PDT
Comment on attachment 570234 [details] [diff] [review]
puppet-manifests

http://hg.mozilla.org/build/puppet-manifests/rev/1d9532043923
Comment 33 Rail Aliiev [:rail] 2011-11-02 12:29:44 PDT
The updated package should be deployed by now.
Comment 34 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2011-11-10 13:05:10 PST
I did a try run some time ago:
https://tbpl.mozilla.org/?tree=Try&rev=fe13a5c6469f

The main issue I have already debugged is breakpad failing to parse clang's debug output. I am working on it.

Note You need to log in before you can comment on or make changes to this bug.