1617684 - (WPT-Fis-Tier-1) Enable wpt tests on Fission as Tier 1

Status: RESOLVED FIXED

(Testing :: web-platform-tests, task, P3)

Comment 1

[Chris Peterson [:cpeterson]]

The DOM Fission team is relying on feature teams to debug and fix Fission failures in their tests. If the failure looks like a bug in Fission's DOM or IPC changes, you can send the bug back to me.

We're hoping to enable Fission for a subset of Nightly users in early Q3, so we would like WPT tests to be green for Fission by end of Q2. Whether a particular test failure actually blocks shipping Fission is up to the discretion of the feature team. You all would know better than the DOM Fission which test failures are most important.

You can enable Fission when running WPT tests locally using mach wpt --enable-fission.

Comment 2

[Chris Peterson [:cpeterson]]

Tracking WPT Fission bugs for Fission M6b (Q2)

Comment 3

[Neha Kochar [:neha]]

Geoff, what's blocking this?

Comment 4

[Geoff Brown [:gbrown]]

There were some recent frequent test failures (like bug 1655619) but those appear to be resolved: the last few runs on m-c look good!

I'll have a look...

Comment 5

[Geoff Brown [:gbrown]]

I see fission web-platform tests only run on a few platforms:

            linux.*64-qr/debug
            linux.*64-shippable-qr/opt: mozilla-central only
            windows10-64-shippable-qr/opt: mozilla-central only

The mozilla-central only platforms would need to be changed so they run on autoland as well, then all 3 platforms could run as tier 1. Other platforms (osx, android, windows 7, windows 10 debug, etc) would remain unchanged: not running, except on try where they would remain tier 2.

Does that sound right?

Once that's done, should this bug remain open for follow-up on other platforms?

Comment 6

[Geoff Brown [:gbrown]]

Attachment: Bug 1617684 - Run existing fission web-platform tests as tier 1; r=

For all fission web-platform tests currently running on mozilla-central as tier 2,
promote to tier 1 and run on all trunk trees.

Comment 7

[Neha Kochar [:neha]]

(In reply to Geoff Brown [:gbrown] from comment #5)

The mozilla-central only platforms would need to be changed so they run on autoland as well, then all 3 platforms could run as tier 1. Other platforms (osx, android, windows 7, windows 10 debug, etc) would remain unchanged: not running, except on try where they would remain tier 2.

Does that sound right?

Once that's done, should this bug remain open for follow-up on other platforms?

That sounds correct, Geoff. Thank you!
I think once we enable on tier1, we can close this, and the follow-up work can be tracked under bug 1586790.

Comment 8

[Chris Peterson [:cpeterson]]

(In reply to Geoff Brown [:gbrown] from comment #5)

I see fission web-platform tests only run on a few platforms:

            linux.*64-qr/debug
            linux.*64-shippable-qr/opt: mozilla-central only
            windows10-64-shippable-qr/opt: mozilla-central only

@ Geoff, what is the difference between "linux.64-qr" and "linux.*64-shippable-qr"? If we are going to test these platforms on both mozilla-central and autoland, do we test both the shippable and non-shippable variants?

If we are only testing one Windows platform, I wonder why we chose Windows 10 opt instead of Windows 10 debug. Does making a platform tier 1 require testing both opt and debug?

The mozilla-central only platforms would need to be changed so they run on autoland as well, then all 3 platforms could run as tier 1. Other platforms (osx, android, windows 7, windows 10 debug, etc) would remain unchanged: not running, except on try where they would remain tier 2.

That sounds OK for now.

Comment 9

[Geoff Brown [:gbrown]]

(In reply to Chris Peterson [:cpeterson] from comment #8)

(In reply to Geoff Brown [:gbrown] from comment #5)
@ Geoff, what is the difference between "linux.64-qr" and "linux.*64-shippable-qr"? If we are going to test these platforms on both mozilla-central and autoland, do we test both the shippable and non-shippable variants?

I think the main difference is that shippable is pgo. There has been discussion on phab about this issue as well and resolution is that we will start running on non-shippable as well.

If we are only testing one Windows platform, I wonder why we chose Windows 10 opt instead of Windows 10 debug. Does making a platform tier 1 require testing both opt and debug?

I can't find reasoning for Windows 10 opt vs debug in hg or bugzilla, but I'm likely missing something - it's a complicated history! It looks like that's the way it has been since last year.

Tier 1 does not require both opt and debug - no problem there.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0cf5110b4388
Run existing fission web-platform tests as tier 1; r=jmaher

Comment 11

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/0cf5110b4388