Closed Bug 957865 Opened 7 years ago Closed 6 years ago

Upgrade ASan Clang in Q1


(Release Engineering :: General, defect)

Not set


(Not tracked)



(Reporter: decoder, Assigned: decoder)




(2 files, 4 obsolete files)

We are considering to upgrade Clang for ASan to a more recent version in Q1. One of the main reasons is support for TSan (ThreadSanitizer), but also certain fixes/optimizations made for ASan. We don't have a target version yet, but I will try to make myself familiar first with the necessary build steps so we can provide the binaries to deploy.
See also It describes how to generate the binaries using your own CentOS or using vagrant images.
Blocks: 895845
Per :decoder's request I uploaded the following file to tooltool:

size: 70800172

sha512: b5b2479ccad6687cdf4485e8bf3cdb289916e5555dbc8dbcd95d952e349f6eed45a3f6f3505e40dfc8abc62579ebb50d73dcdeebbd6519c4c3f3074176a4b3ed
This is the manifest to be used with Rail's upload.
We have a new Clang version available now for testing with ASan (see manifest above), but you might as well want to use the opportunity to test this without ASan, in case you want to upgrade Clang in the future.
So far the build does great (no ASan regression and TSan is working as it seems).

However, I'll likely do another update/build to include this fix that landed today:

With TSan, I noticed a few times that Firefox was hanging/had a stale process, and this could be associated.
Attached patch build-clang-trunk.patch (obsolete) — Splinter Review
Build script changes to build Clang Trunk. Changes made:

* Changed build option -j8 to -j4 because the Vagrant box will OOM during the build when using -j8 with the newest Clang. -j4 works cleanly.

* Use the newer moz-gcc/g++ when building Clang, as Clang now requires at least GCC 4.7. This also requires to set the LD_LIBRARY_PATH appropriately when building with that custom gcc.

* The no-sse-on-linux.patch either applies to trunk or 3.3 so I made a copy of it for the trunk version. The second patch applies cleanly to all versions.
Attachment #8365957 - Flags: review?(rail)
Attachment #8362560 - Attachment is obsolete: true
Attachment #8365957 - Flags: review?(rail) → review+
Comment on attachment 8365960 [details] [diff] [review]
ASan Clang manifest for r200213. (WIP, for testing)

Rail, I've made some more tests with r200213 and I'm quite confident that we can stick to this revision for ASan/TSan. We're only doing 64 bit testing anyway, so I guess we don't need the 32 bit asan.manifest. However, Mac OSX might be interesting in the future and I don't know how to create a package for that. Can you maybe create the Mac package so we can land the manifest with changes to linux64+mac asan.manifest?
Attachment #8365960 - Flags: feedback?(rail)
Comment on attachment 8365960 [details] [diff] [review]
ASan Clang manifest for r200213. (WIP, for testing)

I started a mac build, but I had to add --disable-compiler-version-checks to the ./configure call -- looks like clang from Xcode is too old. Not sure if this is a problem because we build in multiple stages...
Attachment #8365960 - Flags: feedback?(rail) → feedback+
Hmmmm, it didn't work and failed

  MKDIR:     /builds/slave/moz-toolchain/build/stage1/build/tools/clang/runtime/compiler-rt/clang_macho_embedded/hard_pic/x86_64
  MKDIR:     /builds/slave/moz-toolchain/build/stage1/build/tools/clang/runtime/compiler-rt/clang_macho_embedded/hard_pic
  FINAL-ARCHIVE: clang_darwin/eprintf: /builds/slave/moz-toolchain/build/stage1/build/tools/clang/runtime/compiler-rt/clang_darwin/eprintf/libcompiler_rt.a
lipo: no input files specified
lipo: Usage: lipo [input_file] ... [-arch <arch_type> input_file] ... [-info] [-detailed_info] [-output output_file] [-create] [-arch_blank <arch_type>] [-thin <arch_type>] [-remove <arch_type>] ... [-extract <arch_type>] ... [-extract_family <arch_type>] ... [-verify_arch <arch_type> ...] [-replace <arch_type> <file_name>] ...
make[5]: *** [/builds/slave/moz-toolchain/build/stage1/build/tools/clang/runtime/compiler-rt/clang_darwin/eprintf/libcompiler_rt.a] Error 1
make[5]: *** Waiting for unfinished jobs....
  FINAL-ARCHIVE: clang_darwin/10.4: /builds/slave/moz-toolchain/build/stage1/build/tools/clang/runtime/compiler-rt/clang_darwin/10.4/libcompiler_rt.a
  FINAL-ARCHIVE: clang_darwin/ios: /builds/slave/moz-toolchain/build/stage1/build/tools/clang/runtime/compiler-rt/clang_darwin/ios/libcompiler_rt.a
lipo: no input files specified
lipo: Usage: lipo [input_file] ... [-arch <arch_type> input_file] ... [-info] [-detailed_info] [-output output_file] [-create] [-arch_blank <arch_type>] [-thin <arch_type>] [-remove <arch_type>] ... [-extract <arch_type>] ... [-extract_family <arch_type>] ... [-verify_arch <arch_type> ...] [-replace <arch_type> <file_name>] ...
make[5]: *** [/builds/slave/moz-toolchain/build/stage1/build/tools/clang/runtime/compiler-rt/clang_darwin/10.4/libcompiler_rt.a] Error 1
lipo: no input files specified
lipo: Usage: lipo [input_file] ... [-arch <arch_type> input_file] ... [-info] [-detailed_info] [-output output_file] [-create] [-arch_blank <arch_type>] [-thin <arch_type>] [-remove <arch_type>] ... [-extract <arch_type>] ... [-extract_family <arch_type>] ... [-verify_arch <arch_type> ...] [-replace <arch_type> <file_name>] ...
make[5]: *** [/builds/slave/moz-toolchain/build/stage1/build/tools/clang/runtime/compiler-rt/clang_darwin/ios/libcompiler_rt.a] Error 1
make[4]: *** [BuildRuntimeLibraries] Error 2
rm /builds/slave/moz-toolchain/build/stage1/build/Release/lib/clang/3.5/lib/macho_embedded/.dir /builds/slave/moz-toolchain/build/stage1/build/Release/lib/clang/3.5/lib/darwin/.dir
make[3]: *** [compiler-rt/.makeall] Error 2
make[2]: *** [all] Error 1
make[1]: *** [clang/.makeall] Error 2
make: *** [all] Error 1
@cdiehl, I might need your help to get a Clang Mac package for deploying here.

@rail, could you quickly explain what steps cdiehl would normally have to take to get the build package that we need for deploying?

Thank you both for helping out here :)

If we can't get this working then I'd suggest we proceed with linux64-only for this quarter.
Flags: needinfo?(rail)
Flags: needinfo?(cdiehl)
Per "MacOS X builders have everything installed to build clang", not really clear. My guess the requirements are 

* recent xcode
* mercurial
* subversion
* patch

The build instruction can be found on the same page under (see the last code block in that section + the following paragraph how to find the deliverables).

Hope this helps.
Flags: needinfo?(rail)
Attached patch build-clang-trunk-osx.patch (obsolete) — Splinter Review
The attached patch might be not needed on your machines.

% cat build/unix/build-clang/clang.manifest
> [
> {
> "clang_version": "r200213"
> },
> {
> "size": 47,
> "digest": "2005a41fe97a5e00997063705f39d42b6a43b1cf7ba306cbc7b1513de34cdcd050fc6326efa2107f19ba0cc67914745dbf13154fa748010a93cf072481ef4aaa",
> "algorithm": "sha512",
> "filename": ""
> },
> {
> "size": 58997296,
> "digest": "9757d142142442c881b8d1eb31c2fe80e1979a858e6133473b5574a5a3b9cdaf9abed32b2e246b715c9f0eb0969103337918215fc491feae196219e8fb03f0b1",
> "algorithm": "sha512",
> "filename": "clang.tar.bz2"
> }
> ]
Flags: needinfo?(cdiehl)
(In reply to Christoph Diehl [:cdiehl] from comment #14)

% /Applications/ -version
Xcode 5.1
Build version 5B130a

% sw_vers
ProductName:	Mac OS X
ProductVersion:	10.9.2
BuildVersion:	13C64
Thanks Christoph, that was fast! :)

Rail, how should we proceed? Can you upload the build in comment 14? I guess the next step would be to land my reviewed patch, cdiehl's patch (since you also made that change) and the updated manifests for linux64 and osx?
Flags: needinfo?(rail)
I uploaded the file. It would be great if you can push the patch to try before we proceed.
Flags: needinfo?(rail)
Part 1: Final version of the manifests (rolled in the OSX manifest change), keeping f+.
Attachment #8365960 - Attachment is obsolete: true
Attachment #8398630 - Flags: feedback+
Part 2: Adjust the build script for Clang trunk on Linux and OSX (merged the two patches). Keeping r+ as the first (Linux) has r+ and the second (OSX) is the same change that rail proposed.
Attachment #8365957 - Attachment is obsolete: true
Attachment #8396916 - Attachment is obsolete: true
Attachment #8398634 - Flags: review+
Per comment 17: Linux64 ASan Green on try:

As discussed on IRC, we don't have a job for OSX yet, so not including that in the try push. Proceeding with landing all patches now.
You need to log in before you can comment on or make changes to this bug.