Closed Bug 1644235 Opened 8 months ago Closed 5 months ago

Run Marionette unit tests with Fission enabled in CI

Categories

(Testing :: Marionette, task, P1)

task

Tracking

(Fission Milestone:M6c, firefox81 fixed)

RESOLVED FIXED
81 Branch
Fission Milestone M6c
Tracking Status
firefox81 --- fixed

People

(Reporter: whimboo, Assigned: whimboo)

References

Details

(Whiteboard: [marionette-fission-mvp])

Attachments

(1 file)

As noticed on bug 1640141 we do not run the Marionette tests with Fission enabled in CI. To make Marionette Fission compatible it would help a lot to get these scheduled regularly on autoland and mozilla-central.

Note that Marionette is also in use for web-platform-tests (which run with Fission enabled) and partly for setting up the Mochitests. Both test suites look fine so far. But to make sure that the Mn test suite is not completely broken here a try build as triggered with fuzzy --full:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=75f6efd00ec0f3a0b5cf403d0327bce1777606f0

Those tests are all green. Joel, would it be ok to get them running regularly?

Flags: needinfo?(jmaher)

thanks for working on this and getting it green. I did some retriggers (5x) to see how stable this is.

I think the only one unstable is osx.

for the rest I am happy to see them get added to the autoland queue and worked into our scheduling algorithm.

Flags: needinfo?(jmaher)

Looks like bug 1625410 is the one blocker here. Maja, maybe you wanna have a look at this bug?

Depends on: 1625410

Tracking for Fission Nightly M6c (work expected to land in Q3). The blocker bug 1625410 is being tracked for Fission Nightly M6a (work expected to land in June)

Fission Milestone: --- → M6c

Henrik, if bug 1625410 is the only intermittent failure left, can we skip this test for now and start running these tests?

Flags: needinfo?(hskupin)

Note that right now we do not run WebRender jobs for Marionette. As such I wonder if we should add those, and where we would have to run fission jobs. Chris, do you have feedback for us? Maybe we should do it like mochitests do?

https://searchfox.org/mozilla-central/source/taskcluster/ci/test/mochitest.yml#19-25

Flags: needinfo?(cpeterson)

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+2] from comment #7)

Note that right now we do not run WebRender jobs for Marionette. As such I wonder if we should add those, and where we would have to run fission jobs. Chris, do you have feedback for us? Maybe we should do it like mochitests do?

WebRender is not a hard requirement for Fission. So as long as we can run the Mn tests with Fission somewhere (with or without WebRender), then the Fission team is happy.

We can run Mn on the non-QR platforms. We want to test both Linux64 and Windows 10 and both opt and debug. (Some of our other Fission tests are opt-only, but really should have included debug. I don't want to overlook that again here.) We're not running Fission mochitests on macOS yet, so we shouldn't try running Fission Mn on macOS yet.

Given the platforms that currently run Fission mochitests:

https://searchfox.org/mozilla-central/rev/3434a9df60373a997263107e6f124fb164ddebf2/taskcluster/ci/test/mochitest.yml#19-25

I suggest we run Fission Mn on:

            linux.*64/.*: ['autoland']
            linux.*64-shippable/.*: ['mozilla-central']
            windows10-64/.*: ['autoland']
            windows10-64-shippable/.*: ['mozilla-central']

These Mn tests can all be tier 2.

Flags: needinfo?(cpeterson) → needinfo?(hskupin)

Sounds good. I will get that added.

Assignee: nobody → hskupin
Status: NEW → ASSIGNED
Flags: needinfo?(hskupin)
Priority: P3 → P1
Whiteboard: [marionette-fission-mvp]
Pushed by hskupin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/9bf594aab9d0
[marionette] Enable Marionette unit tests with Fission in CI. r=cpeterson,marionette-reviewers,maja_zf
Status: ASSIGNED → RESOLVED
Closed: 5 months ago
Resolution: --- → FIXED
Target Milestone: --- → 81 Branch
You need to log in before you can comment on or make changes to this bug.