Disable Rust unit tests on Windows due to bug 1420472

RESOLVED FIXED in Firefox 64

Status

defect
P1
normal
RESOLVED FIXED
2 years ago
8 months ago

People

(Reporter: ato, Assigned: ato)

Tracking

(Blocks 1 bug)

Version 3
mozilla64
x86_64
Windows
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox58 unaffected, firefox59 disabled, firefox64 fixed)

Details

Attachments

(1 attachment)

Adding the geckodriver Rust unit tests to the try build in
https://bugzil.la/1395933 caused a spike in the intermittency rate
for https://bugzil.la/1420472.  Until this has been investigated I
will disable the Rust tests on Windows.

This will not fix the intermittent but will reduce it.  The
intermittent will again become a problem, I suspect, when someone
adds new Rust unit tests.
Assignee: nobody → ato
Status: NEW → ASSIGNED
Blocks: 1420472
Depends on: 1395933
Pushed by atolfsen@mozilla.com:
https://hg.mozilla.org/mozilla-central/rev/2abe464e8452
Disable geckodriver Rust tests on Windows. r=me a=Aryx
https://hg.mozilla.org/mozilla-central/rev/2abe464e8452
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla59
Status: RESOLVED → REOPENED
Keywords: test-disabled
Resolution: FIXED → ---
I don’t understand why this bug was re-opened.  Is there any further
work I need to do here?
We temporarily disabled the geckodriver unit tests on Windows due to the issue on bug 1420472. As such we still have to track the state, and re-enable the tests whenever this dependency gets fixed. This is how we handle such situations usually,
No longer blocks: 1420472
Depends on: 1420472
OK let me unassign myself so it doesn’t turn up in my work list then.
Assignee: ato → nobody
Priority: -- → P2
Priority: P2 → P3
Apparently Rust tests are in a job called BR now, which is tier-2
on Linux.  I can’t find a BR job at all for Windows anymore.
Do we not want to run Rust tests on Windows?
Flags: needinfo?(ted)
Aryx messaged my directly and said it runs under Windows _2012_
tier-2, not Windows 10 where I was looking.  I guess it makes sense
that they run on the builders.
Flags: needinfo?(ted)
I will do another try run.
Flags: needinfo?(ato)
Summary: Disable geckodriver tests on Windows due to bug 1420472 → Disable Rust unit tests on Windows due to bug 1420472
Enabling geckodriver et al. tests on Windows makes the BR job fail,
but in a different way than before:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=dcbfa82f80b6f4412afc8f429571cb972687042b

This seems to be an sccache timeout, but I’m not sure for what
reason.
Flags: needinfo?(ato)
It looks like the webdriver and geckodriver unit tests are passing
(174 passed), but that sccache times out trying to compile serde:

> 8:07:20     INFO - test result: ok. 174 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
> 18:07:20     INFO - error: failed to execute compile
> 18:07:20     INFO - caused by: error reading compile response from server
> 18:07:20     INFO - caused by: Failed to read response header
> 18:07:20     INFO - caused by: An existing connection was forcibly closed by the remote host. (os error 10054)
> 18:07:20     INFO - mozmake.EXE[2]: Leaving directory 'z:/build/build/src/obj-firefox/testing/geckodriver'
> 18:07:20     INFO - error: Could not compile `serde`.
> 18:07:20     INFO - Caused by:
> 18:07:20     INFO -   process didn't exit successfully: `z:/build/build/src/sccache2/sccache.exe 'z:/build/build/src/rustc/bin/rustc.exe' --crate-name serde 'third_party\rust\serde\src\lib.rs' --crate-type lib --emit=dep-info,link -C opt-level=2 -C codegen-units=1 --cfg 'feature="default"' --cfg 'feature="rc"' --cfg 'feature="serde_derive"' --cfg 'feature="std"' -C metadata=3b988fac4a9d3e60 -C extra-filename=-3b988fac4a9d3e60 --out-dir 'z:/build/build/src/obj-firefox/toolkit/library\i686-pc-windows-msvc\release\deps' --target i686-pc-windows-msvc -L 'dependency=z:/build/build/src/obj-firefox/toolkit/library\i686-pc-windows-msvc\release\deps' -L 'dependency=z:/build/build/src/obj-firefox/toolkit/library\release\deps' --extern 'serde_derive=z:/build/build/src/obj-firefox/toolkit/library\release\deps\serde_derive-2271e9788f4dcf51.dll' --cap-lints warn -C opt-level=2 -C debuginfo=2 --cfg de_boxed_c_str --cfg de_rc_dst --cfg core_duration --cfg integer128 --cfg num_nonzero` (exit code: 2)

It’s not clear to me why serde is not already built by this point.
I would’ve thought serde is being _used_ by tests such as
response::tests::test_json_window_rect_response?
Flags: needinfo?(ted)
Flags: needinfo?(hskupin)
Here some output from a job:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=dcbfa82f80b6f4412afc8f429571cb972687042b&selectedJob=201764740

Serde gets compiled for toolkit first:

> 20:19:18     INFO -     Compiling serde v1.0.66
> 20:19:18     INFO -       Running `z:/build/build/src/sccache2/sccache.exe 'z:/build/build/src/rustc/bin/rustc.exe' --crate-name build_script_build 'third_party\rust\serde\build.rs' --crate-type bin --emit=dep-info,link -C opt-level=2 -C codegen-units=1 --cfg 'feature="default"' --cfg 'feature="rc"' --cfg 'feature="serde_derive"' --cfg 'feature="std"' -C metadata=a833dfff5d9941a9 -C extra-filename=-a833dfff5d9941a9 --out-dir 'z:/build/build/src/obj-firefox/toolkit/library\release\build\serde-a833dfff5d9941a9' -C 'linker=z:/build/build/src/obj-firefox/build/win64/cargo-linker.bat' -L 'dependency=z:/build/build/src/obj-firefox/toolkit/library\release\deps' --cap-lints warn`

Then we compile it again for geckodriver:

> 20:20:14     INFO -     Compiling serde v1.0.66
> 20:20:14     INFO -       Running `z:/build/build/src/sccache2/sccache.exe 'z:/build/build/src/rustc/bin/rustc.exe' --crate-name build_script_build 'third_party\rust\serde\build.rs' --crate-type bin --emit=dep-info,link -C opt-level=2 -C codegen-units=1 --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=1c947763419dfe71 -C extra-filename=-1c947763419dfe71 --out-dir 'z:\build\build\src\obj-firefox\testing\geckodriver\.\release\build\serde-1c947763419dfe71' -C 'linker=z:/build/build/src/obj-firefox

And once more but with msvc for geckodriver:

> 20:31:57     INFO -    Compiling serde v1.0.66
> 20:31:57     INFO -      Running `z:/build/build/src/sccache2/sccache.exe 'z:/build/build/src/rustc/bin/rustc.exe' --crate-name serde 'third_party\rust\serde\src\lib.rs' --crate-type lib --emit=dep-info,link -C opt-level=2 -C codegen-units=1 --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=7a39c7d00625e267 -C extra-filename=-7a39c7d00625e267 --out-dir 'z:\build\build\src\obj-firefox\testing\geckodriver\.\i686-pc-windows-msvc\release\deps' --target i686-pc-windows-msvc -L 'dependency=z:\build\build\src\obj-firefox\testing\geckodriver\.\i686-pc-windows-msvc\release\deps' -L 'dependency=z:\build\build\src\obj-firefox\testing\geckodriver\.\release\deps' --cap-

... and for toolkit:

> 20:32:34     INFO -    Compiling serde v1.0.66
> 20:32:34     INFO -      Running `z:/build/build/src/sccache2/sccache.exe 'z:/build/build/src/rustc/bin/rustc.exe' --crate-name serde 'third_party\rust\serde\src\lib.rs' --crate-type lib --emit=dep-info,link -C opt-level=2 -C codegen-units=1 --cfg 'feature="default"' --cfg 'feature="rc"' --cfg 'feature="serde_derive"' --cfg 'feature="std"' -C metadata=3b988fac4a9d3e60 -C extra-filename=-3b988fac4a9d3e60 --out-dir 'z:/build/build/src/obj-firefox/toolkit/library\i686-pc-windows-msvc\release\deps' --target i686-pc-windows-msvc -L 'dependency=z:/build/build/src/obj-firefox/toolkit/library\i686-pc-windows-msvc\release\deps' -L 'dependency=z:/build/build/src/obj-firefox/toolkit/library\release\deps' --extern 'serde_derive=z:/build/build/src/obj-firefox/toolkit/library\release\deps\serde_derive-2271e9788f4dcf51.dll' --cap-lints warn -C opt-level=2 -C debuginfo=2 --cfg de_boxed_c_str --cfg de_rc_dst --cfg core_duration --cfg integer128 --cfg num_nonzero`

I assume multiple processes are used for this job and as such we have an intersection. While serde is correctly compiled for geckodriver and our tests run successfully, it fails for the toolkit component. Not that the order we compile is different. But I doubt that this has any impact here.
Flags: needinfo?(hskupin)
Blocks: 1372587
I have a patch in bug 1495798 that I'm testing on try:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c664672e729a7d9b7ea7c67491e7fd74eb372cb3

https://hg.mozilla.org/try/rev/a2c8beece69a5c508325c4cdd63df98b2595b0b2

Could you push your patch to try atop my patch and see what happens? If it doesn't fix the issue then it might at least give us better diagnostics. Also it sounds like bug 1420472 is no longer involved?
Flags: needinfo?(ted)
Andreas, the patch from Ted landed. Can you please try again to enable the geckodriver rust tests on Windows?
Flags: needinfo?(ato)
I can't guarantee that my patches will have fixed your issue, but hopefully if it still fails we can get a clearer picture of what's wrong.
Also: if this is fixed it's very likely that bug 1396845 was in fact what fixed it.
Depends on: 1396845
Thanks ted!  Let’s get these tests running.
Assignee: nobody → ato
Status: REOPENED → ASSIGNED
Priority: P3 → P1
Pushed by atolfsen@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/04459ddd517a
geckodriver: re-enable rust tests on windows; r=whimboo
https://hg.mozilla.org/mozilla-central/rev/04459ddd517a
Status: ASSIGNED → RESOLVED
Closed: 2 years ago10 months ago
Resolution: --- → FIXED

The tests were enabled when the patch landed for Firefox 64.

Keywords: test-disabled
Target Milestone: mozilla59 → mozilla64
You need to log in before you can comment on or make changes to this bug.