1361256 - Build win64-asan on TaskCluster failed

Status: RESOLVED FIXED

(Core :: General, defect)

Description

[Ting-Yu Chou [:ting] (away)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=f4f816a6220a142648d0b0e0fb8460cb18aeec90&filter-tier=3&selectedJob=95801498

03:46:56     INFO -  checking what kind of list files are supported by the linker... list
03:46:56     INFO -  configure: error: Unable to parse cl -showIncludes prefix. This compiler's locale has an unsupported formatting.
03:46:56     INFO -  DEBUG: <truncated - see config.log for full output>
03:46:56     INFO -  DEBUG: configure:15679: checking for valid C compiler optimization flags
03:46:56     INFO -  DEBUG: configure:15690: z:/build/build/src/sccache2/sccache.exe z:/build/build/src/clang/bin/clang-cl.exe -Xclang -std=gnu99 -c -fsanitize=address -fsanitize-blacklist=z:/build/build/src/build/sanitizers/asan_blacklist_win.txt  -TC -nologo -wd4091 -D_HAS_EXCEPTIONS=0  -W3 -Gy -Zc:inline -wd4244 -wd4267 -Wno-unknown-pragmas -Wno-ignored-pragmas -Wno-deprecated-declarations -Wno-invalid-noreturn -we4553 -O2 -gline-tables-only -Qunused-arguments   conftest.c 1>&5
03:46:56     INFO -  DEBUG: configure:16914: checking what kind of list files are supported by the linker
03:46:56     INFO -  DEBUG: configure:16919: z:/build/build/src/sccache2/sccache.exe z:/build/build/src/clang/bin/clang-cl.exe -Xclang -std=gnu99 -o conftest.obj -c -fsanitize=address -fsanitize-blacklist=z:/build/build/src/build/sanitizers/asan_blacklist_win.txt  -TC -nologo -wd4091 -D_HAS_EXCEPTIONS=0  -W3 -Gy -Zc:inline -wd4244 -wd4267 -Wno-unknown-pragmas -Wno-ignored-pragmas -Wno-deprecated-declarations -Wno-invalid-noreturn -we4553 -Qunused-arguments   conftest.c 1>&5
03:46:56     INFO -  DEBUG: configure:16921: z:/build/build/src/sccache2/sccache.exe z:/build/build/src/clang/bin/clang-cl.exe -Xclang -std=gnu99 -o conftest clang_rt.asan_dynamic-x86_64.lib clang_rt.asan_dynamic_runtime_thunk-x86_64.lib -LARGEADDRESSAWARE -NXCOMPAT -RELEASE -DYNAMICBASE conftest.list  kernel32.lib user32.lib gdi32.lib winmm.lib wsock32.lib advapi32.lib secur32.lib 1>&5
03:46:56     INFO -  DEBUG: conftest.list : fatal error LNK1107: invalid or corrupt file: cannot read at 0x14
03:46:56     INFO -  DEBUG: clang-cl.exe: warning: unknown argument ignored in clang-cl: '-LARGEADDRESSAWARE' [-Wunknown-argument]
03:46:56     INFO -  DEBUG: clang-cl.exe: warning: unknown argument ignored in clang-cl: '-NXCOMPAT' [-Wunknown-argument]
03:46:56     INFO -  DEBUG: clang-cl.exe: error: linker command failed with exit code 1107 (use -v to see invocation)
03:46:56     INFO -  DEBUG: configure:16925: z:/build/build/src/sccache2/sccache.exe z:/build/build/src/clang/bin/clang-cl.exe -Xclang -std=gnu99 -o conftest clang_rt.asan_dynamic-x86_64.lib clang_rt.asan_dynamic_runtime_thunk-x86_64.lib -LARGEADDRESSAWARE -NXCOMPAT -RELEASE -DYNAMICBASE -Wl,-filelist,conftest.list  kernel32.lib user32.lib gdi32.lib winmm.lib wsock32.lib advapi32.lib secur32.lib 1>&5
03:46:56     INFO -  DEBUG: LINK : warning LNK4001: no object files specified; libraries used
03:46:56     INFO -  DEBUG: LINK : warning LNK4068: /MACHINE not specified; defaulting to X64
03:46:56     INFO -  DEBUG: LINK : fatal error LNK1561: entry point must be defined
03:46:56     INFO -  DEBUG: clang-cl.exe: warning: unknown argument ignored in clang-cl: '-LARGEADDRESSAWARE' [-Wunknown-argument]
03:46:56     INFO -  DEBUG: clang-cl.exe: warning: unknown argument ignored in clang-cl: '-NXCOMPAT' [-Wunknown-argument]
03:46:56     INFO -  DEBUG: clang-cl.exe: error: linker command failed with exit code 1561 (use -v to see invocation)
03:46:56     INFO -  DEBUG: configure:16927: z:/build/build/src/sccache2/sccache.exe z:/build/build/src/clang/bin/clang-cl.exe -Xclang -std=gnu99 -o conftest clang_rt.asan_dynamic-x86_64.lib clang_rt.asan_dynamic_runtime_thunk-x86_64.lib -LARGEADDRESSAWARE -NXCOMPAT -RELEASE -DYNAMICBASE @conftest.list  kernel32.lib user32.lib gdi32.lib winmm.lib wsock32.lib advapi32.lib secur32.lib 1>&5
03:46:56     INFO -  DEBUG: clang-cl.exe: warning: unknown argument ignored in clang-cl: '-LARGEADDRESSAWARE' [-Wunknown-argument]
03:46:56     INFO -  DEBUG: clang-cl.exe: warning: unknown argument ignored in clang-cl: '-NXCOMPAT' [-Wunknown-argument]
03:46:56     INFO -  DEBUG: configure: error: Unable to parse cl -showIncludes prefix. This compiler's locale has an unsupported formatting.
03:46:56     INFO -  ERROR: old-configure failed
03:46:56     INFO -  *** Fix above errors and then restart with\
03:46:56     INFO -                 "z:/build/build/src/mozmake.EXE -f client.mk build"
03:46:56     INFO -  client.mk:379: recipe for target 'configure' failed
03:46:56     INFO -  mozmake.EXE: *** [configure] Error 1

Comment 1

[Ting-Yu Chou [:ting] (away)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=79e7f54c318ea518c353b26f59874eecf583e042&filter-tier=1&filter-tier=2&filter-tier=3&selectedJob=96820775

07:06:19     INFO -  # z:/build/build/src/sccache2/sccache.exe z:/build/build/src/clang/bin/clang-cl.exe -Xclang -std=gnu99 -showIncludes -c -Fonul dummy-hello.c 2>&1
07:06:19     INFO -  ## clang-cl.exe: warning: argument unused during compilation: '--show-includes' [-Wunused-command-line-argument]
07:06:19     INFO -  ###

Somehow sccache changes "-showIncludes" to "--show-includes', any ideas?

Comment 2

[(not currently active) Ted Mielczarek]

I disabled sccache for clang-cl builds in bug 1355099. sccache doesn't actually know how to handle clang-cl properly yet. I think if you update to pick up that changeset you should avoid this issue.

Comment 3

[Ting-Yu Chou [:ting] (away)]

Attachment: Bug 1361256 part 1 - Disable sccache for Win64 ASan because it doesn't actually support clang-cl.

sccache changes "--showIncludes" to "--show-includes" which clang-cl
doesn't understand. According to the owner's comment, this is because
it doesn't actually support clang-cl, disable it for now.

Review commit: https://reviewboard.mozilla.org/r/139142/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/139142/

Comment 4

[Ting-Yu Chou [:ting] (away)]

The build is still failed after disabling sccahe:

02:54:15     INFO -  z:/build/build/src/build/win64/mozconfig.asan: line 4: cd: z:/build/build/src/clang/lib/clang/*: No such file or directory

This time seems about tooltoool...

Comment 5

[Ting-Yu Chou [:ting] (away)]

Attachment: Bug 1361256 part 2 - Avoid mozconfig failures when clang-cl has't been downloaded yet.

The `cd $PATH && pwd` pattern no longer works because tooltool
packages could haven't been downloaded yet, check the directory
existence beforehand.

Review commit: https://reviewboard.mozilla.org/r/139162/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/139162/

Comment 6

[Ting-Yu Chou [:ting] (away)]

Comment on attachment 8867589 [details]
Bug 1361256 part 1 - Disable sccache for Win64 ASan because it doesn't actually support clang-cl.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/139142/diff/1-2/

Comment 7

[Ting-Yu Chou [:ting] (away)]

Comment on attachment 8867603 [details]
Bug 1361256 part 2 - Avoid mozconfig failures when clang-cl has't been downloaded yet.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/139162/diff/1-2/

Comment 8

[Michael Shal [:mshal]]

Comment on attachment 8867589 [details]
Bug 1361256 part 1 - Disable sccache for Win64 ASan because it doesn't actually support clang-cl.

https://reviewboard.mozilla.org/r/139142/#review142558

Comment 9

[(not currently active) Ted Mielczarek]

Comment on attachment 8867603 [details]
Bug 1361256 part 2 - Avoid mozconfig failures when clang-cl has't been downloaded yet.

I don't think I quite understand this. I'm going to ask glandium to take a look at this instead.

Comment 10

[Mike Hommey [:glandium]]

Comment on attachment 8867603 [details]
Bug 1361256 part 2 - Avoid mozconfig failures when clang-cl has't been downloaded yet.

https://reviewboard.mozilla.org/r/139162/#review152810

::: commit-message-21f91:3
(Diff revision 2)
> +The `cd $PATH && pwd` pattern no longer works because tooltool
> +packages could haven't been downloaded yet, check the directory
> +existence beforehand.

Please make the commit message clearer about what this works around.

I'm going to guess what you mean is that mozconfig is evaluated once before tooltool packages are downloaded, and that fails (I guess that comes from the mach artifact toolchain command itself, which does the downloading).

Comment 11

[Ting-Yu Chou [:ting] (away)]

Comment on attachment 8867589 [details]
Bug 1361256 part 1 - Disable sccache for Win64 ASan because it doesn't actually support clang-cl.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/139142/diff/2-3/

Comment 12

[Ting-Yu Chou [:ting] (away)]

Comment on attachment 8867603 [details]
Bug 1361256 part 2 - Avoid mozconfig failures when clang-cl has't been downloaded yet.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/139162/diff/2-3/

Comment 13

[Ting-Yu Chou [:ting] (away)]

Comment on attachment 8867603 [details]
Bug 1361256 part 2 - Avoid mozconfig failures when clang-cl has't been downloaded yet.

https://reviewboard.mozilla.org/r/139162/#review152810

> Please make the commit message clearer about what this works around.
> 
> I'm going to guess what you mean is that mozconfig is evaluated once before tooltool packages are downloaded, and that fails (I guess that comes from the mach artifact toolchain command itself, which does the downloading).

Changed it a bit, let me know if it's still unclear.

Comment 14

[Mike Hommey [:glandium]]

Comment on attachment 8867603 [details]
Bug 1361256 part 2 - Avoid mozconfig failures when clang-cl has't been downloaded yet.

https://reviewboard.mozilla.org/r/139162/#review152830

::: commit-message-21f91:3
(Diff revision 3)
> +Bug 1361256 part 2 - Avoid mozconfig failures when clang-cl has't been downloaded yet. r?glandium
> +
> +The `cd $PATH && pwd` pattern no longer works because tooltool packages could

"could haven't been"? I guess you mean "haven't been".

::: commit-message-21f91:4
(Diff revision 3)
> +Bug 1361256 part 2 - Avoid mozconfig failures when clang-cl has't been downloaded yet. r?glandium
> +
> +The `cd $PATH && pwd` pattern no longer works because tooltool packages could
> +haven't been downloaded yet when mach evaluates mozconigs, check the directory

typo: mozconigs

Comment 15

[Ting-Yu Chou [:ting] (away)]

Comment on attachment 8867603 [details]
Bug 1361256 part 2 - Avoid mozconfig failures when clang-cl has't been downloaded yet.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/139162/diff/3-4/

Comment 16

[Mike Hommey [:glandium]]

Comment on attachment 8867603 [details]
Bug 1361256 part 2 - Avoid mozconfig failures when clang-cl has't been downloaded yet.

https://reviewboard.mozilla.org/r/139162/#review152834

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/58bced492339
part 1 - Disable sccache for Win64 ASan because it doesn't actually support clang-cl. r=mshal
https://hg.mozilla.org/integration/autoland/rev/bf2b3066a087
part 2 - Avoid mozconfig failures when clang-cl has't been downloaded yet. r=glandium

Comment 18

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/58bced492339
https://hg.mozilla.org/mozilla-central/rev/bf2b3066a087