1359760 - enable green tc win 10 tests

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: General, task)

Description

[Rob Thijssen [:grenade (EET/UTC+0300)]]

this bug will just track what has been validated green and then enabled by default for Taskcluster Windows 10 test suites.

Comment 1

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Attachment: Bug 1359760 - enable: a11y, awsy, cpp, j, jit, jp, mn, ss suites on tc win10;

Review commit: https://reviewboard.mozilla.org/r/133876/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/133876/

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8861852 [details]
Bug 1359760 - enable: a11y, awsy, cpp, j, jit, jp, mn, ss suites on tc win10;

https://reviewboard.mozilla.org/r/133876/#review136796

with these changes, I am concerned that turning on tests for windows 10 will require us to make exclusions for windows10-asan (which are being removed here).  Can we not just adjust the test-sets for now?

::: taskcluster/ci/test/tests.yml:1375
(Diff revision 1)
>      description: "Are we slim yet"
>      suite: awsy
>      treeherder-symbol: tc-SY(sy)
> -    run-on-projects: ['mozilla-beta', 'mozilla-aurora', 'mozilla-central', 'mozilla-inbound', 'autoland', 'try']
> +    run-on-projects:
> +        by-test-platform:
> +            windows10.*: []

awsy runs on windows 10 currently, we should keep that running there

Comment 3

[Rob Thijssen [:grenade (EET/UTC+0300)]]

awsy failed on win 10 in my pre-push tests (but succeeded on win 7), this is why it was disabled: https://treeherder.mozilla.org/#/jobs?repo=try&revision=fbc489dc0333cd18c0f90f47c29fe0c33e0f4a39&filter-searchStr=sy

i believe the asan exclusions were introduced because most tests were failing see bug 1333003, comment 132 onward. however the new exclusions just disable all non-green tests for all win 10. the tests which are green now, should also be green for asan builds. tests that are not green on win 10 are not likely to be green in asan.

there's no way to adjust just the test sets (except creating separate sets for win 7 and win 10, but you said previously you did not want that approach).

this patch was the only way i could come up with to enable all green windows tests on both win 7 and 10 (where different tests pass/fail) without forking test sets for 7 & 10. happy to hear alternate ideas though.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

so awsy started failing yesterday- it should be fixed and continue to run (I filed bug 1359805 to track this).  Historically tests are not green on linux64 asan vs opt/debug, so I don't want to make assumptions that they will be green or we will want to run tests on that configuration.

I think keeping awsy going and I will be convinced of a r+.  There are no other great ideas I have for this.

Comment 5

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Comment on attachment 8861852 [details]
Bug 1359760 - enable: a11y, awsy, cpp, j, jit, jp, mn, ss suites on tc win10;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/133876/diff/1-2/

Comment 6

[Rob Thijssen [:grenade (EET/UTC+0300)]]

ok, new review request pushed which adds:

- a11y
- awsy
- marionette

to (what was included in earlier push):

- cppunit
- jetpack
- jit
- jsreftest

the three additional test suites go green because the worker types for win 10 are switched to runAsCurrentUser (matching win 7 configuration).

Comment 7

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Comment on attachment 8861852 [details]
Bug 1359760 - enable: a11y, awsy, cpp, j, jit, jp, mn, ss suites on tc win10;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/133876/diff/2-3/

Comment 8

[Rob Thijssen [:grenade (EET/UTC+0300)]]

i forgot to mention mochitest-browser-screenshots, which is also enabled (and greened due to runAsCurrentUser) in this patch.

this means bug 1332945 can be resolved with this patch too.

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8861852 [details]
Bug 1359760 - enable: a11y, awsy, cpp, j, jit, jp, mn, ss suites on tc win10;

https://reviewboard.mozilla.org/r/133876/#review136896

thanks for the work on this

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/856951fadbb6
enable: a11y, awsy, cpp, j, jit, jp, mn, ss suites on tc win10; r=jmaher

Comment 11

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/856951fadbb6

Comment 12

[Rob Thijssen [:grenade (EET/UTC+0300)]]

gtest also now greened and enabled (see bug 1352137, comment 7)

Comment 13

[Joel Maher ( :jmaher ) (UTC -8)]

can we resolve this bug?

Comment 14

[Rob Thijssen [:grenade (EET/UTC+0300)]]

this is just a tracker bug. when the mochitest-media and reftest bugs are resolved, this can be closed.

Comment 15

[Rob Thijssen [:grenade (EET/UTC+0300)]]

closing as no longer useful

Comment 16

[Firefox Bug Husbandry Bot]

Removing leave-open keyword from resolved bugs, per :sylvestre.