Turn off web-platform-tests on Fennec
Categories
(Firefox for Android Graveyard :: Testing, task, P3)
Tracking
(firefox69 fixed)
Tracking | Status | |
---|---|---|
firefox69 | --- | fixed |
People
(Reporter: impossibus, Assigned: impossibus)
References
Details
Attachments
(1 file)
The current status of web-platform-tests on Android is:
- web-platform-tests (including reftests) run on Fennec on mozilla-central and try. (Tier-2)
- web-platform-tests (Tier-2) and web-platform-tests-reftests (Tier-3) run on GeckoView on mozilla-central and try. These are being greened up and will soon be Tier-2.
Now that we're only interested in Fennec on ESR68, we can turn the corresponding wpt tests off, at least on mozilla-central.
To be clear, there are no web-platform-tests running on Android on mozilla-esr68.
We originally stood up wpt on Fennec last year to validate that wpt can run in an Android environment in CI. This was a bit before GeckoView/TestRunnerActivity had all the prerequisites ready for wpt.
Assignee | ||
Comment 1•5 years ago
|
||
Assignee | ||
Comment 2•5 years ago
|
||
James, wpt-sync is the main consumer of wpt results for Fennec on try. Given that Fennec is part of ESR68, do you still want to include Fennec results in the wpt-sync metadata update?
Comment 3•5 years ago
•
|
||
We wanted to wait for Firefox for Android to ship the first ESR of no return which is ESR 71 before disabling Fennec tests. See bug 1548659 comment 0.
Assignee | ||
Comment 4•5 years ago
|
||
(In reply to Kevin Brosnan [:kbrosnan] from comment #3)
We wanted to wait for Firefox for Android to ship the first ESR of no return which is ESR 71 before disabling Fennec tests. See bug 1548659 comment 0.
No rush, happy to hold off.
Comment 5•5 years ago
|
||
Fromt he wpt-sync point of view the requirements are purely functional; we need to have results on try for anything that's running on integration branches or central. I don't have any other specific opinions on the value of fennec tests so I defer to the relevant engineering teams.
Updated•5 years ago
|
Comment 6•5 years ago
|
||
(In reply to Kevin Brosnan [:kbrosnan] from comment #3)
We wanted to wait for Firefox for Android to ship the first ESR of no return which is ESR 71 before disabling Fennec tests. See bug 1548659 comment 0.
Can we reconsider? These run on packet.net, which has limited capacity. The Fennec wpt tests on Android 7.0 are not part of the "traditional" Android 4.3 range of test suites, run as tier 2 and have several perma-fails -- I'm not sure they really provide much coverage in this state.
Comment 7•5 years ago
|
||
If there are well thought out reasons for disabling then I am happy with that decision. I just wanted to make sure that we thought about the reasons before disabling tests wholesale before 71.
Dropping a ni to Ryan for his info. Feel free to clear without comment if there is nothing to add.
Comment 8•5 years ago
|
||
If it's not a test suite we care about for Fennec (and it sounds like that's the case), I don't see the harm in turning them off. Mainly we just want to avoid breaking any Fennec test suites we do care about until we're past the point of no return for Fennec shipping from anywhere other than ESR68.
Looking at it from another perspective, it sounds like this is something we would have wanted to do even if Fennec were still riding the trains as usual, further making me think we're fine to proceed here.
Thanks for checking, though!
Assignee | ||
Comment 9•5 years ago
|
||
(In reply to Ryan VanderMeulen [:RyanVM] from comment #8)
Looking at it from another perspective, it sounds like this is something we would have wanted to do even if Fennec were still riding the trains as usual, further making me think we're fine to proceed here.
Well, if Fennec were riding the trains we'd make an effort to green these up properly. It makes more sense to focus on wpt on GeckoView now.
Comment 10•5 years ago
|
||
Comment 11•5 years ago
|
||
bugherder |
Updated•4 years ago
|
Description
•