1743714 - Make wptrunner default to --enable-fission when run locally (and add --disable-fission flag)

Status: RESOLVED FIXED

(Testing :: web-platform-tests, task, P2)

Description

[Chris Peterson [:cpeterson]]

(This is my summary of instructions from James Graham.)

1. Before fission is enabled by default, we should change the logic at [1] to reverse the Fission default. We will still be running some non-fission configurations (e10s on debug Linux) and they will need an explicit pref/command line flag to set {fission: False}.

2. Once we've done that, the existing metadata (wpt test expectations for Fission) should be correct

3. For the actual wpt sync but, the only change required is to ensure we're still selecting all the relevant builds to run on try. Currently the query strings we pass to mach try fuzzy are [2], so if we're going to start doing non-fission builds on central that won't match that query, we'll need to update.

4. In a follow-up bug after Fission is the default on both desktop and Android:

• The logic at [1] can always set {fission: True} in the run info.
• We can remove the fission variable from the set that's used by the wpt sync.
• We can remove any if fission conditions from the wpt test expectations metadata and finally remove the fission flag from the run_info data.

[1] https://searchfox.org/mozilla-central/rev/ead7da2d9c5400bc7034ff3f06a030531bd7e5b9/testing/web-platform/tests/tools/wptrunner/wptrunner/browsers/firefox.py#240

[2] TBD

Comment 1

[Chris Peterson [:cpeterson]]

Hsin-Yi says this task is specifically about making wptrunner default to --enable-fission when run locally, not in CI. We'll want a --disable-fission flag so engineers can run tests locally without Fission.

Comment 2

[James Graham [:jgraham]]

I talked to ahal about this a bit and I'm planning to take the following approach:

• Remove the fission pref from the base profile as set in Bug 1736859, and put it in individual harness profiles. This should make it easier to migrate harnesses one at a time.
• For wpt always set a fission pref in the mozharness wrapper. This ensures that Gecko CI jobs continue to have the same behaviour as today.
• In wptrunner, if we don't pass either --[no-]enable-fission or --setpref=fission.autostart=[true|false] default fission to true, but always set the pref to one thing or another so we don't need to land this together with the change in defaults.

First try push at https://treeherder.mozilla.org/#/jobs?repo=try&revision=01291994532a8a6fb597b42ba52e514cc389934a just for the wpt part, but it could well be broken at this stage :)

Comment 3

[James Graham [:jgraham]]

Attachment: Bug 1743714 - Set fission defaults in each test's profile settings,

This excludes web-platform-tests which will be handled later in the commit series.
Not using the base profile for everything allows us to migrate the suite-specific
fission handling one suite at a time.

Comment 4

[James Graham [:jgraham]]

Attachment: Bug 1743714 - Always pass a fission pref to wpt via mozharness,

This ensures that fission is always explictly enabled or disabled
when running in gecko CI.

Comment 5

[James Graham [:jgraham]]

Attachment: Bug 1743714 - Default to fission enabled in wpt,

With this change, fission will be enabled by default, unless
--enable-fission or --setpref=fission.autostart=false is passed
on startup. We always set a fission setting, irrespective of
the browser defaults.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/34cfa1f16dbe
Set fission defaults in each test's profile settings, r=ahal,perftest-reviewers,sparky
https://hg.mozilla.org/integration/autoland/rev/bf73c68058b9
Always pass a fission pref to wpt via mozharness, r=ahal
https://hg.mozilla.org/integration/autoland/rev/773776ba6f7c
Default to fission enabled in wpt, r=ahal

Comment 7

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/31880 for changes under testing/web-platform/tests

Comment 8

[Alexandru Michis [:malexandru]]

https://hg.mozilla.org/mozilla-central/rev/34cfa1f16dbe
https://hg.mozilla.org/mozilla-central/rev/bf73c68058b9
https://hg.mozilla.org/mozilla-central/rev/773776ba6f7c

Comment 9

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR was closed without merging

Comment 10

[Chris Peterson [:cpeterson]]

(In reply to Web Platform Test Sync Bot (Matrix: #interop:mozilla.org) from comment #9)

Upstream PR was closed without merging

James, I see you just touched this bug's WPT PR: https://github.com/web-platform-tests/wpt/pull/31880. Is that WPT PR still needed now?

Fission is enabled by default in Firefox and no one outside Mozilla should need to run wptrunner with --disable-fission or --setpref=fission.autostart=false.

Comment 11

[James Graham [:jgraham]]

I was trying to work out why it's failing; unmerged upstream PRs lead to our copy of wpt getting out of sync with the upstream copy.