1694835 - Create Fission mochitest-plain + xorigin test tasks for more platforms

Status: RESOLVED FIXED

(Testing :: Mochitest, task, P2)

Description

[Chris Peterson [:cpeterson]]

Our testing strategy and list of tests we'd like to enable on which platforms:

https://docs.google.com/document/d/19MergQfMob2mdb_s7hlz5Ilj7Bn-Bbi324r6d2bZfxw/edit

We want to run these tests on:

• debug and opt builds
  • Where “shippable” opt builds available, we prefer to test shippable opt builds on central and plain opt builds on autoland.
• autoland and central
• We prefer to run tests with Fission + WebRender (“qr”) where available.

We should probably enable the Fission mochitest-plain tests in bug 1694833 before enabling these mochitest-plain + xorigin tests.

Add platforms:

• linux1804-64-qr/debug (We're already running fission+xorigin tests on linux1804-64/debug without WebRender, but we really should test with WebRender instead.)
• linux1804-64-qr/opt on autoland, shippable-qr/opt on central
• linux1804-64-asan/opt
• linux1804-64-tsan/opt
• linux1804-64-tsan/opt
• macosx1014-64-qr/debug
• macosx1014-64-qr/opt on autoland, shippable-qr/opt on central
• windows10-64-qr/debug
• windows10-64-qr/opt on autoland, shippable-qr/opt on central

Remove platforms:

• linux1804-64/debug without WebRender (Once we're running fission+xorigin tests on linux1804-64-qr/debug with WebRender, but we won't need to test without WebRender.)

Comment 1

[Neha Kochar [:neha]]

Andrew, can you please enable the additional platforms for these tests?

Comment 2

[Andrew Halberstadt [:ahal]]

This has been added to our current sprint.

Comment 3

[Chris Peterson [:cpeterson]]

Andrew, also: the mochitest-plain xorigin tests are current tier 2, but we would like them boosted to tier 1.

Comment 4

[Andrew Halberstadt [:ahal]]

Sounds good, I'll be starting this bug today or tomorrow.

Comment 5

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1694835 - [ci] Create mochitest-plain fission-xorigin tasks on all platforms, r?jmaher

While this will add many new tasks to the taskgraph, it makes no changes to
what actually runs in CI (via the target task graph).

Comment 6

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1694835 - [ci] Run mochitest-plain fission xorigin tasks on more platforms

Depends on D108783

Comment 7

[Andrew Halberstadt [:ahal]]

Hey Chris, looks like the xorigin tests need some work:
https://treeherder.mozilla.org/jobs?repo=try&revision=a4813c46eff128482994870bf8a06677022cee24

Not surprising since without D108783 it was not even possible to run those tasks on try. Once this revision lands, developers will be able to schedule those failing tasks on try via ./mach try {fuzzy|chooser} --full. Just warn them that this command will include platforms we are not trying to stand up here.

Comment 8

[Chris Peterson [:cpeterson]]

(In reply to Andrew Halberstadt [:ahal] from comment #7)

Hey Chris, looks like the xorigin tests need some work:
https://treeherder.mozilla.org/jobs?repo=try&revision=a4813c46eff128482994870bf8a06677022cee24

Not surprising since without D108783 it was not even possible to run those tasks on try. Once this revision lands, developers will be able to schedule those failing tasks on try via ./mach try {fuzzy|chooser} --full. Just warn them that this command will include platforms we are not trying to stand up here.

Thanks for the heads up. I'll file new bugs for these xorigin test failures.

btw, in that Try push, I see M-fis-xorig is tier 1 on "Linux 18.04 x86 WebRender debug", but tier 2 on other all platforms. Can we make M-fis-xorig tier 1 on all platforms (once the tests are fixed or skipped)?

Comment 9

[Andrew Halberstadt [:ahal]]

Yeah, will do once the tests are ready. I just didn't bother fiddling with the tiers yet.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ed6f7487c36e
[ci] Create mochitest-plain fission-xorigin tasks on all platforms, r=jmaher

Comment 11

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/ed6f7487c36e

Comment 12

[Chris Peterson [:cpeterson]]

Renaming this bug because it created the new test tasks, but didn't schedule the tests to run yet (because we need to fix the new test failures first). Then we can enable the tests in bug 1700781.

Comment 13

[Phabricator Automation]

Comment on attachment 9209735 [details]
Bug 1694835 - [ci] Run mochitest-plain fission xorigin tasks on more platforms

Revision D108784 was moved to bug 1700781. Setting attachment 9209735 [details] to obsolete.