Add clang-based base toolchain jobs

RESOLVED FIXED in Firefox 67

Status

task
RESOLVED FIXED
3 months ago
2 months ago

People

(Reporter: glandium, Assigned: glandium)

Tracking

unspecified
mozilla67

Firefox Tracking Flags

(firefox67 fixed)

Details

Attachments

(1 attachment)

(Assignee)

Description

3 months ago

We now default to building with clang, and we have a declared minimum supported version, but we don't actually check that version works. We should.

(Assignee)

Comment 1

3 months ago

These are copies of the corresponding gcc-based base toolchain tasks,
with FORCE_GCC and the gcc dependency removed.

We also tweak things a little for those builds to actually end up green.

Comment 2

3 months ago
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/439db957d3e2
Add a clang-based base toolchains task. r=froydnj

Comment 3

3 months ago
Backout by shindli@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6d6ab46f6428
Backed out changeset 439db957d3e2 for linux64 toolchains bustages in toolkit/library/gtest/target CLOSED TREE

Backed out changeset 439db957d3e2 (bug 1525760) for linux64 toolchains bustages in toolkit/library/gtest/target CLOSED TREE

Backout link: https://hg.mozilla.org/integration/autoland/rev/6d6ab46f64283ed228ebe32cd5b67f614fb26cca

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&revision=439db957d3e21f3fcaa2cea6b08dd7eae93c0eaf&selectedJob=227340126

Log link: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=227340126&repo=autoland&lineNumber=36736

Log snippet:

[task 2019-02-09T03:58:43.893Z] 03:58:43 INFO - /builds/worker/workspace/build/src/sccache2/sccache /builds/worker/workspace/build/src/clang/bin/clang++ -std=gnu++14 -Qunused-arguments -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -Qunused-arguments -Wall -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtype-limits -Wunreachable-code -Wunreachable-code-return -Wwrite-strings -Wno-invalid-offsetof -Wclass-varargs -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Wc++1z-compat -Wcomma -Wimplicit-fallthrough -Werror=non-literal-null-conversion -Wstring-conversion -Wtautological-overlap-compare -Wno-inline-new-delete -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wformat -Wformat-security -Wno-gnu-zero-variadic-macro-arguments -Wno-unknown-warning-option -Wno-return-type-c-linkage -D_GLIBCXX_USE_CXX11_ABI=0 -fno-sized-deallocation -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -g -Xclang -load -Xclang /builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -Os -fno-omit-frame-pointer -funwind-tables -Werror -fPIC -shared -Wl,-z,defs -Wl,--gc-sections -Wl,-h,libxul.so -o libxul.so /builds/worker/workspace/build/src/obj-firefox/toolkit/library/libxul_so.list -lpthread -Wl,-z,noexecstack -Wl,-z,text -Wl,-z,relro -Wl,-z,nocopyreloc -Wl,-Bsymbolic-functions -Wl,--build-id=sha1 /builds/worker/workspace/build/src/toolkit/library/StaticXULComponents.ld -Wl,-rpath-link,/builds/worker/workspace/build/src/obj-firefox/dist/bin -Wl,-rpath-link,/usr/local/lib ../../security/nss/lib/crmf/crmf_crmf/libcrmf.a ../../js/src/build/libjs_static.a /builds/worker/workspace/build/src/obj-firefox/x86_64-unknown-linux-gnu/debug/libgkrust.a ../../security/sandbox/linux/libmozsandbox.so ../../config/external/nspr/pr/libnspr4.so ../../config/external/nspr/libc/libplc4.so ../../config/external/nspr/ds/libplds4.so ../../config/external/lgpllibs/liblgpllibs.so ../../security/nss/lib/nss/nss_nss3/libnss3.so ../../security/nss/lib/util/util_nssutil3/libnssutil3.so ../../security/nss/lib/smime/smime_smime3/libsmime3.so ../../config/external/sqlite/libmozsqlite3.so ../../security/nss/lib/ssl/ssl_ssl3/libssl3.so ../../widget/gtk/mozgtk/stub/libmozgtk_stub.so -Wl,--version-script,symverscript -ldl -lrt -lm -lX11 -lX11-xcb -lxcb -lXcomposite -lXcursor -lXdamage -lXext -lXfixes -lXi -lXrender -lpthread -ldl -lc -lfreetype -lfontconfig -Wl,--version-script,/builds/worker/workspace/build/src/build/unix/stdc++compat/hide_std.ld -ldbus-glib-1 -ldbus-1 -lgobject-2.0 -lglib-2.0 -latk-1.0 -lpangocairo-1.0 -lgdk_pixbuf-2.0 -lcairo-gobject -lpango-1.0 -lcairo -lgio-2.0 -lxcb-shm -lpangoft2-1.0 -lXt -lgthread-2.0
[task 2019-02-09T03:58:43.894Z] 03:58:43 INFO - /usr/bin/ld: /builds/worker/workspace/build/src/obj-firefox/x86_64-unknown-linux-gnu/debug/libgkrust.a(mdb.o)(text_env+0x96d): unresolvable AWAVAUATUSH��h�*H�T$(���� relocation against symbol `stderr@@GLIBC_2.2.5'
[task 2019-02-09T03:58:43.894Z] 03:58:43 INFO - /usr/bin/ld: BFD (GNU Binutils for Debian) 2.22 internal error, aborting at ../../bfd/reloc.c line 443 in bfd_get_reloc_size
[task 2019-02-09T03:58:43.894Z] 03:58:43 INFO - /usr/bin/ld: Please report this bug.
[task 2019-02-09T03:58:43.894Z] 03:58:43 INFO - clang-3.9: error: linker command failed with exit code 1 (use -v to see invocation)
[task 2019-02-09T03:58:43.894Z] 03:58:43 INFO - /builds/worker/workspace/build/src/config/rules.mk:674: recipe for target 'libxul.so' failed
[task 2019-02-09T03:58:43.894Z] 03:58:43 ERROR - make[4]: *** [libxul.so] Error 1
[task 2019-02-09T03:58:43.895Z] 03:58:43 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/toolkit/library'
[task 2019-02-09T03:58:43.895Z] 03:58:43 INFO - /builds/worker/workspace/build/src/config/recurse.mk:74: recipe for target 'toolkit/library/target' failed
[task 2019-02-09T03:58:43.895Z] 03:58:43 ERROR - make[3]: *** [toolkit/library/target] Error 2
[task 2019-02-09T03:58:43.895Z] 03:58:43 INFO - /builds/worker/workspace/build/src/config/recurse.mk:32: recipe for target 'compile' failed
[task 2019-02-09T03:58:43.895Z] 03:58:43 ERROR - make[2]: *** [compile] Error 2
[task 2019-02-09T03:58:43.896Z] 03:58:43 INFO - /builds/worker/workspace/build/src/config/rules.mk:400: recipe for target 'default' failed
[task 2019-02-09T03:58:43.896Z] 03:58:43 ERROR - make[1]: *** [default] Error 2
[task 2019-02-09T03:58:43.896Z] 03:58:43 INFO - client.mk:125: recipe for target 'build' failed
[task 2019-02-09T03:58:43.896Z] 03:58:43 INFO - make: *** [build] Error 2
[task 2019-02-09T03:58:43.924Z] 03:58:43 INFO - 290 compiler warnings present.
[task 2019-02-09T03:58:44.000Z] 03:58:44 INFO - Notification center failed: Install notify-send (usually part of the libnotify package) to get a notification when the build finishes.
[task 2019-02-09T03:58:44.061Z] 03:58:44 ERROR - Return code: 2
[task 2019-02-09T03:58:44.061Z] 03:58:44 WARNING - setting return code to 2
[task 2019-02-09T03:58:44.061Z] 03:58:44 FATAL - 'mach build -v' did not run successfully. Please check log for errors.
[task 2019-02-09T03:58:44.061Z] 03:58:44 FATAL - Running post_fatal callback...
[task 2019-02-09T03:58:44.061Z] 03:58:44 FATAL - Exiting -1
[task 2019-02-09T03:58:44.062Z] 03:58:44 INFO - [mozharness: 2019-02-09 03:58:44.062019Z] Finished build step (failed)

Flags: needinfo?(mh+mozilla)
(Assignee)

Comment 5

3 months ago

Funny coincidences.

This job was building fine before the recent rkv changes landed (either bug 1500259 or bug 1490496), but is now busted, but bug 1394825 fixes it.

Flags: needinfo?(mh+mozilla)

Comment 6

3 months ago
Pushed by ealvarez@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/cdcbd5b13210
Add a clang-based base toolchains task. r=froydnj

Comment 7

2 months ago
bugherder
Status: NEW → RESOLVED
Last Resolved: 2 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla67

Comment 8

2 months ago
Pushed by emilio@crisal.io:
https://hg.mozilla.org/mozilla-central/rev/1b147d9934f1
Disable base-toolchains-clang jobs for now. a=ccoroiu r=bustage

I had to disable the jobs. Copying the reason from the commit message from https://hg.mozilla.org/mozilla-central/rev/1b147d9934f1, it's failing in a way that doesn't make any sense to me.

It's hitting the same error as comment 4 in some revisions, but not others. More annoyingly, it hits the error in some revisions while being green in a different revision without any source-code change between them.

This is green:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=952b928f1605ad5676ae4ccfd41612b34523bae5

And this is red:

https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=952b928f1605ad5676ae4ccfd41612b34523bae5

And the diff between them is nothing. Which is quite baffling.

I disabled the new jobs on CI for now in order to avoid backing out the stack of dependent patches (bug 1394825, bug 1523071 and bug 1523140) that landed afterwards.

Mike, do you have any idea of how can this happen? Are those builds incremental and are we hitting a strange linker / compiler bug? I wasn't sure if touching CLOBBER would help, but disabling the jobs seemed a less risky approach for now.

Status: RESOLVED → REOPENED
Flags: needinfo?(mh+mozilla)
Resolution: FIXED → ---

Comment 10

2 months ago
Pushed by emilio@crisal.io:
https://hg.mozilla.org/mozilla-central/rev/50309e384ceb
followup: Make linters happy about a comment. a=ccoroiu r=bustage-fix-bustage
(Assignee)

Comment 11

2 months ago

https://github.com/rust-lang/rust/issues/58393 is what happened. I'll reenable these jobs with sccache disabled.

Flags: needinfo?(mh+mozilla)

Comment 12

2 months ago
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/mozilla-inbound/rev/8b2e8f177263
Followup: re-enable the tasks without sccache. r=me

Comment 13

2 months ago
bugherder
Status: REOPENED → RESOLVED
Last Resolved: 2 months ago2 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.