1758780 - Update builders to rustc 1.60 and clang 14

Status: RESOLVED FIXED

(Firefox Build System :: Toolchains, task)

Description

[Mike Hommey [:glandium]]

Rust 1.60.0 is due to be released on April 7, and clang 14 on March 15 (the latter may slip, though). We can't upgrade one without the other because cross-language PGO doesn't like different versions of LLVM, and Rust 1.60.0 is the first version with LLVM 14.

Comment 1

[Arthur K. (he/him)]

(In reply to Mike Hommey [:glandium] from comment #0)

Rust 1.60.0 is due to be released on April 7, and clang 14 on March 15 (the latter may slip, though). We can't upgrade one without the other because cross-language PGO doesn't like different versions of LLVM, and Rust 1.60.0 is the first version with LLVM 14.

I see that (https://github.com/llvm/llvm-project/milestone/3) shows 7 open issues but the likelihood of LLVM 14 slipping into April is probably high as well considering how 13.0.1 went. Probably targeting 101?

Comment 2

[Arthur K. (he/him)]

Loosk like LLVM 14 went live early this morning.

Comment 3

[Mike Hommey [:glandium]]

Attachment: Bug 1758780 - Don't build tests when building clang.

We never run them, so there's no point building them.

Comment 4

[Mike Hommey [:glandium]]

Attachment: Bug 1758780 - Use LLVM's lib and link tools when building clang with clang.

Comment 5

[Mike Hommey [:glandium]]

Attachment: Bug 1758780 - Take the ms-compat version used to build clang from a LLVM cmake file.

While the llvm/cmake/platforms/WinMsvc.cmake file only supports
cross-compiles, it contains the right -fms-compatibility-version
to use to build clang/llvm (modulo when upstream forgets to update it).

It's better to rely on that, than to keep a version in build-clang.py
that would possibly need to cover multiple versions of clang.

Comment 6

[Mike Hommey [:glandium]]

Attachment: Bug 1758780 - Use the same version of clang as for other builds for sm fuzzing builds.

It used to be this way, until bug 1731582 made it tied to clang-12 to
keep it to that version. This was subsequently upgraded to clang-13 in
bug 1736459, but this should really have been changed back to
unversioned clang.

Comment 7

[Mike Hommey [:glandium]]

Attachment: Bug 1758780 - Avoid building LLVM when building wasi compiler-rt.

Instead, do the same trick as for the wasi sysroot, and reuse clang
itself.

Comment 8

[Mike Hommey [:glandium]]

Attachment: Bug 1758780 - Avoid linking libclang both statically and dynamically.

This applies the same as https://reviews.llvm.org/D81967 to our
CMakeLists.txt. Interestingly, the upstream change landed in LLVM 11,
but not applying the same on our end doesn't cause problems until LLVM
14.

Comment 9

[Mike Hommey [:glandium]]

Attachment: Bug 1758780 - Fallthrough to <atomic> for wasi when using clang >= 14.

Clang 14 added a dummy backoff policy for atomic polling that doesn't
know about threads, which presumably WASI can use.
See https://reviews.llvm.org/D114109.

Comment 10

[Mike Hommey [:glandium]]

Attachment: Bug 1758780 - Add a toolchain for Visual Studio 2019.

We're going to need it to bootstrap LLVM 14.

Comment 11

[Mike Hommey [:glandium]]

Attachment: Bug 1758780 - Update builders to clang 14.

Comment 12

[Mike Hommey [:glandium]]

Attachment: Bug 1758780 - Update builders to rustc 1.60.

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6c5a34659edd
Don't build tests when building clang. r=firefox-build-system-reviewers,mhentges
https://hg.mozilla.org/integration/autoland/rev/05ea08e25976
Use LLVM's lib and link tools when building clang with clang. r=firefox-build-system-reviewers,mhentges
https://hg.mozilla.org/integration/autoland/rev/6debe82932bf
Take the ms-compat version used to build clang from a LLVM cmake file. r=firefox-build-system-reviewers,mhentges
https://hg.mozilla.org/integration/autoland/rev/6d6a694b7091
Use the same version of clang as for other builds for sm fuzzing builds. r=firefox-build-system-reviewers,mhentges
https://hg.mozilla.org/integration/autoland/rev/f1447363a4b9
Avoid building LLVM when building wasi compiler-rt. r=firefox-build-system-reviewers,mhentges

Comment 14

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/6c5a34659edd
https://hg.mozilla.org/mozilla-central/rev/05ea08e25976
https://hg.mozilla.org/mozilla-central/rev/6debe82932bf
https://hg.mozilla.org/mozilla-central/rev/6d6a694b7091
https://hg.mozilla.org/mozilla-central/rev/f1447363a4b9

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fd6ac7ba625f
Avoid linking libclang both statically and dynamically. r=firefox-build-system-reviewers,andi
https://hg.mozilla.org/integration/autoland/rev/d5f9c05036b4
Fallthrough to <atomic> for wasi when using clang >= 14. r=nika
https://hg.mozilla.org/integration/autoland/rev/19e434c7f6c2
Add a toolchain for Visual Studio 2019. r=firefox-build-system-reviewers,mhentges

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e390fac06dae
Update builders to clang 14. r=firefox-build-system-reviewers,mhentges,andi
https://hg.mozilla.org/integration/autoland/rev/0f6aff1f340e
Update builders to rustc 1.60. r=firefox-build-system-reviewers,mhentges

Comment 17

[Atila Butkovits]

Backed out as requested.

Backout link: https://hg.mozilla.org/integration/autoland/rev/a0a3110885524aea8de9e782ab42e3fac6ae359f

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/330065f83d19
Update builders to clang 14. r=firefox-build-system-reviewers,mhentges,andi
https://hg.mozilla.org/integration/autoland/rev/2a3c0204b519
Update builders to rustc 1.60. r=firefox-build-system-reviewers,mhentges

Comment 19

[Cristina Cozmuta (:CrissCozmuta)]

https://hg.mozilla.org/mozilla-central/rev/fd6ac7ba625f
https://hg.mozilla.org/mozilla-central/rev/d5f9c05036b4
https://hg.mozilla.org/mozilla-central/rev/19e434c7f6c2

Comment 20

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/35a6605827b5
Backed out 2 changesets for causing OSX failures

Comment 21

[Noemi Erli[:noemi_erli]]

Backed out 2 changesets (Bug 1758780) for causing OSX failures

Push with failures:
https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&collapsedPushes=842382&selectedTaskRun=f70duRsUQp-K2hYjnzulBg.0&resultStatus=success%2Ctestfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&fromchange=6b2bdfbe9f12236fa6dd5828aee47f7405d6258e&searchStr=linux%2C18.04%2Cx64%2Cwebrender%2Cshippable%2Copt%2Cxpcshell%2Ctests%2Cwithout%2Cfission%2Cenabled%2Ctest-linux1804-64-shippable-qr%2Fopt-xpcshell-nofis-e10s%2Cx2&tochange=fa6795d107b10fdd812be351f426522b28a4f363

https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&collapsedPushes=842382&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&searchStr=os%2Cx%2C11%2Cwebrender%2Cshippable%2Copt%2Cmochitests%2Cwith%2Cwebgl%2Cover%2Cipc%2Cwith%2Cfission%2Cenabled%2Ctest-macosx1100-64-shippable-qr%2Fopt-mochitest-media-gli-fis-e10s%2Cmda1&revision=2a3c0204b5191edee1fbc64c6a8b9cb62e5afe2d

https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&collapsedPushes=842382&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&searchStr=os%2Cx%2C11%2Cwebrender%2Cshippable%2Copt%2Cmochitests%2Cwith%2Cnetworking%2Con%2Csocket%2Cprocess%2Cwith%2Cfission%2Cenabled%2Ctest-macosx1100-64-shippable-qr%2Fopt-mochitest-browser-chrome-spi-nw-fis-e10s%2Cbc2&revision=2a3c0204b5191edee1fbc64c6a8b9cb62e5afe2d

Log: https://treeherder.mozilla.org/logviewer?job_id=374337508&repo=autoland&lineNumber=3435
https://treeherder.mozilla.org/logviewer?job_id=374328185&repo=autoland&lineNumber=4894
https://treeherder.mozilla.org/logviewer?job_id=374328195&repo=autoland&lineNumber=6386

Backout: https://hg.mozilla.org/integration/autoland/rev/35a6605827b50e35e9aeab8f6beb811108122594

Comment 22

[Mike Hommey [:glandium]]

Attachment: Bug 1758780 - Increase test's fuzzTolerance to accomodate clang 14.

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/116072b466c3
Increase test's fuzzTolerance to accomodate clang 14. r=padenot

Comment 24

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/116072b466c3

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/481062597411
Update builders to clang 14. r=firefox-build-system-reviewers,mhentges,andi
https://hg.mozilla.org/integration/autoland/rev/10e2414392ae
Update builders to rustc 1.60. r=firefox-build-system-reviewers,mhentges

Comment 26

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/481062597411
https://hg.mozilla.org/mozilla-central/rev/10e2414392ae

Comment 27

[Narcis Beleuzu [:NarcisB]]

Backed out for jsreftest failure on fp-evaluation-order.js

Backout link: https://hg.mozilla.org/integration/autoland/rev/f78fb89b9c2f6255da18795f55dd420dcb1be6b2
Log link: https://treeherder.mozilla.org/logviewer?job_id=375051016&repo=autoland&lineNumber=51232

Comment 28

[Narcis Beleuzu [:NarcisB]]

Please also check this reftest failure

Comment 29

[Atila Butkovits]

Backed out as requested.
Backout link: https://hg.mozilla.org/integration/autoland/rev/3394502af98e6006fc870cdeae0a4193f8c15c4f

Comment 30

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/8dff65812a56
Backed out changeset 116072b466c3 as requested by the dev.

Comment 31

[Atila Butkovits]

Backout merged to central: https://hg.mozilla.org/mozilla-central/rev/8dff65812a565046db0c6201b5ee0364368f34d5

Comment 32

[Cristian Tuns]

Backout merged to central: https://hg.mozilla.org/mozilla-central/rev/3394502af98e

Comment 33

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6db7394933d9
Update builders to clang 14. r=firefox-build-system-reviewers,mhentges,andi
https://hg.mozilla.org/integration/autoland/rev/5bebcdfc5dd8
Update builders to rustc 1.60. r=firefox-build-system-reviewers,mhentges

Comment 34

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/6db7394933d9
https://hg.mozilla.org/mozilla-central/rev/5bebcdfc5dd8

Comment 35

[Alexandru Ionescu (needinfo me) [:alexandrui]]

(In reply to Pulsebot from comment #33)

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6db7394933d9
Update builders to clang 14. r=firefox-build-system-reviewers,mhentges,andi
https://hg.mozilla.org/integration/autoland/rev/5bebcdfc5dd8
Update builders to rustc 1.60. r=firefox-build-system-reviewers,mhentges

== Change summary for alert #33909 (as of Fri, 22 Apr 2022 14:27:38 GMT) ==

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
15%	allrecipes ContentfulSpeedIndex	android-hw-g5-7-0-arm7-shippable-qr	warm webrender	2,522.92 -> 2,898.25
9%	allrecipes PerceptualSpeedIndex	android-hw-g5-7-0-arm7-shippable-qr	warm webrender	2,876.67 -> 3,129.92
4%	espn PerceptualSpeedIndex	android-hw-g5-7-0-arm7-shippable-qr	warm webrender	4,481.67 -> 4,658.08
4%	youtube loadtime	android-hw-g5-7-0-arm7-shippable-qr	warm webrender	1,400.46 -> 1,453.88
4%	youtube SpeedIndex	android-hw-g5-7-0-arm7-shippable-qr	warm webrender	1,436.46 -> 1,490.75
4%	youtube fcp	android-hw-g5-7-0-arm7-shippable-qr	warm webrender	1,104.25 -> 1,142.96
3%	youtube ContentfulSpeedIndex	android-hw-g5-7-0-arm7-shippable-qr	warm webrender	986.17 -> 1,020.33
3%	espn ContentfulSpeedIndex	android-hw-g5-7-0-arm7-shippable-qr	warm webrender	3,093.83 -> 3,199.17
3%	youtube PerceptualSpeedIndex	android-hw-g5-7-0-arm7-shippable-qr	warm webrender	969.21 -> 1,001.67
3%	youtube dcf	android-hw-g5-7-0-arm7-shippable-qr	warm webrender	922.27 -> 953.00
...	...	...	...	...
2%	espn FirstVisualChange	android-hw-g5-7-0-arm7-shippable-qr	warm webrender	1,194.46 -> 1,223.25
2%	youtube LastVisualChange	android-hw-g5-7-0-arm7-shippable-qr	cold webrender	2,281.38 -> 2,335.50
2%	youtube LastVisualChange	android-hw-g5-7-0-arm7-shippable-qr	warm webrender	1,657.92 -> 1,697.08
2%	booking loadtime	android-hw-g5-7-0-arm7-shippable-qr	warm webrender	1,193.68 -> 1,221.75
2%	ebay-kleinanzeigen LastVisualChange	android-hw-g5-7-0-arm7-shippable-qr	warm webrender	3,989.54 -> 4,071.75

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=33909

Comment 36

[Andra Esanu (needinfo me)]

(In reply to Natalia Csoregi [:nataliaCs] from comment #34)

https://hg.mozilla.org/mozilla-central/rev/6db7394933d9
https://hg.mozilla.org/mozilla-central/rev/5bebcdfc5dd8

== Change summary for alert #33990 (as of Mon, 02 May 2022 13:27:06 GMT) ==

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
5%	build times	macosx64	buildbot-unknown rusttests	619.62 -> 587.04

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=33990

Comment 37

[Arthur K. (he/him)]

Too late for an update to 14.0.4?