Closed Bug 1685599 Opened 3 years ago Closed 3 years ago

Improve clang-macosx-cross toolchain build

Categories

(Firefox Build System :: Toolchains, task)

task

Tracking

(firefox86 fixed)

RESOLVED FIXED
86 Branch
Tracking Status
firefox86 --- fixed

People

(Reporter: glandium, Assigned: glandium)

References

Details

Attachments

(7 files)

While working on bug 1685034, I noticed that to build linux64-clang-11-macosx-cross, we use macox64-clang-11, which needs linux64-clang-9-macosx-cross, which needs macosx64-clang-9, which needs linux64-clang-9. (And that's without counting cctools-port). That's a quite long tail. macox64-clang-11 needs linux64-clang-9-macosx-cross because it needs a macosx-cross clang, but using linux64-clang-11-macosx-cross would create a dependency loop.

What we do need macosx64-clang-* for in linux64-clang-*-macosx-cross is:

  • compiler-rt
  • llvm-symbolizer

The former is needed for all builds, the latter is only needed for sanitizer builds.

My proposal is the following:

  • Build llvm-symbolizer separately, so that only sanitizer builds need to depend on that
  • Build compiler-rt separately, like we already do for aarch64 macos.

That would lift the dependency on macosx64-clang-* for the macosx-cross builds.

That would lift the dependency on macosx64-clang-* for the macosx-cross builds.

and allow to build macosx64-clang-* without depending on an older macosx64-clang indirectly.

Only sanitizer builds require a native llvm-symbolizer executable.
Ideally, we'd build llvm-symbolizer from scratch, which would be faster,
but for now, let's go the easy route and just extract it from the
corresponding native clang builds.

We don't actually do anything with the llvm-symbolizer executable on
android builds, so we don't install it in $FINAL_TARGET, avoilding
the dependency on android builds (plus, we actually don't have an
android-native llvm-symbolizer, so even if it were already shipped, it
would be the wrong file).

We actually don't need the macosx-cross toolchain to build compiler-rt,
because building compiler-rt doesn't require compiler-rt (duh).

The llvm build system does support building a universal compiler-rt for
multiple platforms at once, but as far as I know it only supports doing
so with the same SDK, while we want to use separate SDKs for each, so
build the x86_64 compiler-rt separately.

The new toolchain contains both aarch64 and x86_64 compiler-rts. We
could have a separate for each, but compiler-rt is small enough that
entirely separate toolchains is not warranted.

Now that linux64-clang-11-macosx-cross doesn't need a macosx64-clang-*
to build, there is no circular dependency to avoid.

Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/68fa5ecf205c
Build llvm-symbolizer separately. r=firefox-build-system-reviewers,mhentges
https://hg.mozilla.org/integration/autoland/rev/5483ce4f5f4c
Build aarch64-compiler-rt using clang-11 instead of clang-11-macosx-cross. r=firefox-build-system-reviewers,mhentges
https://hg.mozilla.org/integration/autoland/rev/c7d4ecfc689a
Build a compiler-rt for x86_64. r=firefox-build-system-reviewers,mhentges
https://hg.mozilla.org/integration/autoland/rev/3ba2f1c048c3
Build a single macosx-cross toolchain. r=firefox-build-system-reviewers,mhentges
https://hg.mozilla.org/integration/autoland/rev/2839d8d39881
Build macosx64-clang-11 with linux64-clang-11-macosx-cross. r=firefox-build-system-reviewers,mhentges
https://hg.mozilla.org/integration/autoland/rev/29eab2f84c77
Remove now unused macosx64-clang-9 and linux64-clang-9-macosx-cross toolchains. r=firefox-build-system-reviewers,mhentges
Blocks: 1685650

The webrender wrench macos build, which is cross-compiled, needs a
macOS native libLLVM (a .dylib) to link against. The file is currently
part of the macosx-cross toolchain, but that was more incidental than
intentional. As we're going to change the macosx-cross toolchain in a
way that will remove the libLLVM.dylib, pull the file from the macOS
native clang.

Flags: needinfo?(mh+mozilla)
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/00ed2e36099a
Build llvm-symbolizer separately. r=firefox-build-system-reviewers,mhentges
https://hg.mozilla.org/integration/autoland/rev/3dcceef9ec6b
Don't rely on macosx-cross toolchain containing a mac native libLLVM. r=firefox-build-system-reviewers,andi,dmajor
https://hg.mozilla.org/integration/autoland/rev/b0b1df3a8a2d
Build aarch64-compiler-rt using clang-11 instead of clang-11-macosx-cross. r=firefox-build-system-reviewers,mhentges
https://hg.mozilla.org/integration/autoland/rev/ef756f7c3211
Build a compiler-rt for x86_64. r=firefox-build-system-reviewers,mhentges
https://hg.mozilla.org/integration/autoland/rev/2a3ddd89d882
Build a single macosx-cross toolchain. r=firefox-build-system-reviewers,mhentges
https://hg.mozilla.org/integration/autoland/rev/71810213a26c
Build macosx64-clang-11 with linux64-clang-11-macosx-cross. r=firefox-build-system-reviewers,mhentges
https://hg.mozilla.org/integration/autoland/rev/0e47b711f906
Remove now unused macosx64-clang-9 and linux64-clang-9-macosx-cross toolchains. r=firefox-build-system-reviewers,mhentges
Regressions: 1685854
Blocks: 1685859
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: