1492037 - Build clang with LLVM as a shared library

Status: RESOLVED FIXED

(Firefox Build System :: Toolchains, enhancement)

Comment 1

[Mike Hommey [:glandium]]

Attachment: Bug 1492037 - Build clang with LLVM as a shared library

Doing so changes the size of the compressed toolchain archive from ~280M
to ~120M, and the decompressed size from ~1500M to ~675M. This will
reduce the overhead of decompression during builds.

As we ship llvm-symbolizer as part of ASan builds, we do need it to
still statically link against LLVM, which we do with a small patch.

CCTools-port needs to be rebuilt in some cases of clang changes, which
this one of, so touch the script so that it happens.

Comment 2

[Nathan Froyd [:froydnj]]

Comment on attachment 9009838 [details]
Bug 1492037 - Build clang with LLVM as a shared library

Nathan Froyd [:froydnj] has approved the revision.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/b0abf5269c25
Build clang with LLVM as a shared library. r=froydnj

Comment 4

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/ceff48930273
Backout changeset b0abf5269c25 to give time to toolchains to build without blocking other landings.

Comment 5

[Mike Hommey [:glandium]]

So this caused interesting problems:
- clang-tidy jobs filter out some files, and end up removing the LLVM shared library, so clang-tidy ends up broken.
- For some reason, on cross-osx builds, ld doesn't like clang's lto objects (although ld _was_ rebuilt against the new clang).

Comment 6

[Mike Hommey [:glandium]]

Other problems:
- The llvm-symbolizer patch doesn't apply cleanly on clang 3.9. But we don't use that for ASan builds, so I guess we can go without the patch.
- Once ld is fixed wrt the problem mentioned in comment 5, I'm now getting ld crashes with:
*** glibc detected *** /builds/worker/workspace/build/src/cctools/bin/x86_64-apple-darwin11-ld: free(): invalid pointer: 0x00000000007eb880 ***

Comment 7

[Mike Hommey [:glandium]]

huh

==27242== Invalid free() / delete / delete[] / realloc()
==27242==    at 0x4C299AC: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27242==    by 0x685A749: llvm::InlineAsm::ConstraintInfo::Parse(llvm::StringRef, std::vector<llvm::InlineAsm::ConstraintInfo, st
d::allocator<llvm::InlineAsm::ConstraintInfo> >&) (in /builds/worker/workspace/build/src/clang/lib/libLLVM-6.0.so)
==27242==    by 0x685AE39: llvm::InlineAsm::ParseConstraints(llvm::StringRef) (in /builds/worker/workspace/build/src/clang/lib/lib
LLVM-6.0.so)
==27242==    by 0x6D2EC7E: llvm::TargetLowering::ParseConstraints(llvm::DataLayout const&, llvm::TargetRegisterInfo const*, llvm::
ImmutableCallSite) const (in /builds/worker/workspace/build/src/clang/lib/libLLVM-6.0.so)
==27242==    by 0x69161DB: (anonymous namespace)::CodeGenPrepare::optimizeInst(llvm::Instruction*, bool&) (in /builds/worker/works
pace/build/src/clang/lib/libLLVM-6.0.so)
==27242==    by 0x690F0CD: (anonymous namespace)::CodeGenPrepare::runOnFunction(llvm::Function&) (in /builds/worker/workspace/buil
d/src/clang/lib/libLLVM-6.0.so)
==27242==    by 0x687DB7A: llvm::FPPassManager::runOnFunction(llvm::Function&) (in /builds/worker/workspace/build/src/clang/lib/li
bLLVM-6.0.so)
==27242==    by 0x687DDE2: llvm::FPPassManager::runOnModule(llvm::Module&) (in /builds/worker/workspace/build/src/clang/lib/libLLV
M-6.0.so)
==27242==    by 0x687E1E4: llvm::legacy::PassManagerImpl::run(llvm::Module&) (in /builds/worker/workspace/build/src/clang/lib/libL
LVM-6.0.so)
==27242==    by 0x74AC9EA: (anonymous namespace)::codegenModule(llvm::Module&, llvm::TargetMachine&) (in /builds/worker/workspace/
build/src/clang/lib/libLLVM-6.0.so)
==27242==    by 0x74B0E93: std::_Function_handler<void (), std::_Bind<llvm::ThinLTOCodeGenerator::run()::$_5 (int)> >::_M_invoke(s
td::_Any_data const&) (in /builds/worker/workspace/build/src/clang/lib/libLLVM-6.0.so)
==27242==    by 0x6758317: std::_Function_handler<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_ba
se::_Deleter> (), std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result<void>, std::__future_base::_Result_
base::_Deleter>, void> >::_M_invoke(std::_Any_data const&) (in /builds/worker/workspace/build/src/clang/lib/libLLVM-6.0.so)
==27242==  Address 0x7eb880 is 0 bytes inside data symbol "_ZNSs4_Rep20_S_empty_rep_storageE"

This is because libLLVM-6.0.so is statically linking libstdc++, and at the same time, forcing all the symbols it exports to have the LLVM_6.0 version. So it exports all the libstdc++ symbols as weak with version LLVM_6.0. But the ld executable has a weak reference to the same symbols under a GLIBCXX_* version.

So ld.so can't actually use the same symbol for both references, and when some stdc++ code in libLLVM checks whether something points to std::string::_Rep::_S_empty_rep_storage, and the object was created on the ld side, with a pointer to ld's, it can't match.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/dba2140cb366
Build clang with LLVM as a shared library. r=froydnj

Comment 9

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/5d69e8c53527
Backout changeset dba2140cb366 to give time to toolchains to build without blocking other landings.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/7e05c2752c5a
Build clang with LLVM as a shared library. r=froydnj

Comment 11

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/7e05c2752c5a