1477306 - point -fcrash-diagnostics-dir at MOZ_UPLOAD_DIR when compiling with clang in automation

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Description

[(not currently active) Ted Mielczarek]

When clang crashes it produces a nice set of diagnostic files that it writes to /tmp (this example is from bug 1477216):
[task 2018-07-19T13:02:58.196Z] 13:02:58     INFO -  clang-6.0: note: diagnostic msg: PLEASE submit a bug report to http://llvm.org/bugs/ and include the crash backtrace, preprocessed source, and associated run script.
[task 2018-07-19T13:02:58.196Z] 13:02:58     INFO -  clang-6.0: note: diagnostic msg:
[task 2018-07-19T13:02:58.196Z] 13:02:58     INFO -  ********************
[task 2018-07-19T13:02:58.197Z] 13:02:58     INFO -  PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
[task 2018-07-19T13:02:58.197Z] 13:02:58     INFO -  Preprocessed source(s) and associated run script(s) are located at:
[task 2018-07-19T13:02:58.197Z] 13:02:58     INFO -  clang-6.0: note: diagnostic msg: /tmp/UnifiedBindings22-210e16.cpp
[task 2018-07-19T13:02:58.197Z] 13:02:58     INFO -  clang-6.0: note: diagnostic msg: /tmp/UnifiedBindings22-210e16.sh
[task 2018-07-19T13:02:58.197Z] 13:02:58     INFO -  clang-6.0: note: diagnostic msg:
[task 2018-07-19T13:02:58.197Z] 13:02:58     INFO -  ********************

Someone asked recently if we could scrape clang's output to notice when this happens and save the diagnostic files. I dug into this a little just now and found that someone added a `-fcrash-diagnostics-dir` option just last week to control where these diagnostic files get written: https://reviews.llvm.org/D48601

We should either update clang or cherry-pick that patch and then use `-fcrash-diagnostics-dir=$UPLOAD_DIR` in CI so that when clang crashes it writes the diagnostics directly to a place where they'll get uploaded as artifacts.

Comment 1

[(Away)]

This would solve bug 1420336 without writing a wrapper; I like it.

Comment 2

[(not currently active) Ted Mielczarek]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=910287c32ac03792533631de810dc201280e1845

Comment 3

[(not currently active) Ted Mielczarek]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=678bda01db393712a17d178ebeb946b29c39fa55

Comment 4

[(Away)]

We'll need to pick up 339424 to make this work on clang-cl: https://github.com/llvm-mirror/clang/commit/8c7ddf2e8931f8aeebd6cb80701bd58cdbc31c7a

Comment 5

[(not currently active) Ted Mielczarek]

Feel free to try again with this patch once that happens. Also we could use this for other platforms if we update clang for them. The patch here has commented-out bits for supporting builds on other platforms.

Comment 6

[(not currently active) Ted Mielczarek]

With bug 1492663 landed we could try this again.

Comment 7

[(Away)]

The patch to make this work in the clang-cl frontend didn't make it to 7.0.0, so we'd still need a patch or a move to trunk.

Comment 8

[(not currently active) Ted Mielczarek]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=479e41967e277737c1db4e33aa3dd4c873dd648f

Comment 9

[(not currently active) Ted Mielczarek]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9d7949cf0383d9e94e826dbbe523185a04a8a23e

Comment 10

[(not currently active) Ted Mielczarek]

That didn't *quite* work for Windows builds:
12:04:44     INFO -    CFLAGS= -fcrash-diagnostics-dir=z:\build\public\build
12:04:54     INFO -  checking whether the C compiler (z:/build/build/src/sccache2/sccache.exe z:/build/build/src/clang/bin/clang-cl.exe -Xclang -std=gnu99 -fms-compatibility-version=19.15.26726  -fcrash-diagnostics-dir=z:\build\public\build  ) works... yes

12:09:34     INFO -  z:/build/build/src/sccache2/sccache.exe z:/build/build/src/clang/bin/clang.exe --driver-mode=cl -Xclang -std=gnu99 -fms-compatibility-version=19.15.26726 -FoTestNANTestingExprC.obj -c  -DNDEBUG=1 -DTRIMMED=1 -Iz:/build/build/src/build/clang-plugin/tests -Iz:/build/build/src/obj-firefox/build/clang-plugin/tests -Iz:/build/build/src/obj-firefox/dist/include -Iz:/build/build/src/obj-firefox/dist/include/nspr -Iz:/build/build/src/obj-firefox/dist/include/nss -MD -FI z:/build/build/src/obj-firefox/mozilla-config.h -DMOZILLA_CLIENT -Qunused-arguments -fcrash-diagnostics-dir=z:buildpublicbuild -nologo -wd4091 -D_HAS_EXCEPTIONS=0 -Gy -Zc:inline -Gw -wd4244 -wd4267 -we4553 -fsyntax-only -Xclang -verify -ferror-limit=0 -std=c11 -Wno-invalid-noreturn -Z7 -Xclang -load -Xclang z:/build/build/src/obj-firefox/build/clang-plugin/clang-plugin.dll -Xclang -add-plugin -Xclang moz-check -O2 -Oy-  -Xclang -MP -Xclang -dependency-file -Xclang .deps/TestNANTestingExprC.obj.pp -Xclang -MT -Xclang TestNANTestingExprC.obj    z:/build/build/src/build/clang-plugin/tests/TestNANTestingExprC.c

We lost some backslashes in the actual command execution:
-fcrash-diagnostics-dir=z:buildpublicbuild

Comment 11

[(not currently active) Ted Mielczarek]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=2f66ff05a37edafbf817df5e16e8276870f87171

Comment 12

[(Away)]

Stealing this - I managed to get around the try failure on Windows.

Comment 13

[(Away)]

Attachment: Point -fcrash-diagnostics-dir at UPLOAD_DIR when compiling with clang in automation

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/31a754e0302d
Point -fcrash-diagnostics-dir at UPLOAD_DIR when compiling with clang in automation r=firefox-build-system-reviewers,chmanchester

Comment 15

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/31a754e0302d