1724062 - Test fission performance on autoland

Status: RESOLVED FIXED

(Testing :: Performance, task, P2)

Description

[Randell Jesup [:jesup] (needinfo me)]

Currently we only run fission tests on m-c; we should run them on autoland in order to get better alerts for fission-related regressions

Comment 1

[Chris Peterson [:cpeterson]]

(In reply to Randell Jesup [:jesup] (needinfo me) from comment #0)

Currently we only run fission tests on m-c; we should run them on autoland in order to get better alerts for fission-related regressions

Which Fission tests are we running on m-c but not autoland? Some of the performance tests?

We are running many other tests (mochitests, WPT, crashtests, reftests) on both m-c and autoland.

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

We're not currently running any Fission enabled performance tests (Talos, Raptor, AWSY) on autoland. We recently reduced tests running without WebRender, so I imagine we have some capacity to increase Fission testing.

:jmaher can you confirm if we have the necessary capacity? I imagine we would want to run AWSY, Talos, and high-value Raptor tests on autoland with Fission enabled.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

we have capacity for fission perftests awsy, talos, high-value raptor on linux, win10, and osx10.15.

Looking forward to these getting enabled.

Comment 4

[Chris Peterson [:cpeterson]]

Note that we hope to enable Fission by default (for all users and tests) in 2021 Q4. Then we'll be able to shut off all these extra "fis" tests on mozilla-central and autoland.

Comment 5

[Chris Peterson [:cpeterson]]

Tentatively tracking this bug as soft blocker for Fission MVP. Running these perf tests on autoland doesn't block shipping Fission, but having them running soon would be nice. After we enable Fission by default in Q4, this bug will be obsolete because all tests will be Fission tests.

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Chris Peterson [:cpeterson] from comment #4)

Note that we hope to enable Fission by default (for all users and tests) in 2021 Q4. Then we'll be able to shut off all these extra "fis" tests on mozilla-central and autoland.

Will there be a way to disable Fission? Should we anticipate continuing to test with Fission disabled until we enable it for the majority of our release population?

Comment 7

[Chris Peterson [:cpeterson]]

(In reply to Dave Hunt [:davehunt] [he/him] ⌚GMT from comment #6)

(In reply to Chris Peterson [:cpeterson] from comment #4)

Note that we hope to enable Fission by default (for all users and tests) in 2021 Q4. Then we'll be able to shut off all these extra "fis" tests on mozilla-central and autoland.

Will there be a way to disable Fission? Should we anticipate continuing to test with Fission disabled until we enable it for the majority of our release population?

We don't intend to support users with Fission disabled. I know some "1proc" tests currently disable e10s. We'll still need to be able to disable both e10s and Fission for those "1proc" tests. But once Fission is the default for users, we won't need to run tests with e10s. Every test will hopefully be Fission or "1proc".

Note that Fission will ship on desktop in 2021 Q4 but won't ship on Android until 2022 H1, which may complicate the test transition. However, we might be able to run Android in a partial Fission mode to simplify the transition.

This will hopefully be an opportunity to remove -e10s and -fis from many test names. :)

Comment 8

[Dave Hunt [:davehunt] [he/him] ⌚BST]

We can target variants based on platform so don't imagine the desktop/android differences will be much of a problem. Looking forward to the eventual simplification of our task names!

Comment 9

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Attachment: Bug 1724062 Test fission performance on autoland r=#perftest

Comment 10

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Attachment: Bug 1724062 - Fix talos perfdocs typo r=#perftest

Depends on D122956

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/66a4fa89f6cd
Test fission performance on autoland r=jmaher

Comment 12

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/66a4fa89f6cd

Comment 13

[Chris Peterson [:cpeterson]]

Setting status-firefox92=wontfix because we don't need to uplift this test change to Beta.

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/721748f6d460
Fix talos perfdocs typo r=perftest-reviewers,alexandru.irimovici

Comment 15

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/721748f6d460