Closed Bug 1550895 Opened 1 year ago Closed 1 year ago

Green up geckoview wpt so they can be upgraded to tier-2 visibility.

Categories

(Firefox for Android :: Testing, task, P3)

task

Tracking

()

RESOLVED FIXED

People

(Reporter: KWierso, Assigned: KWierso)

References

Details

Attachments

(10 files)

47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review

Now that the wpt suite is running against geckoview and expectation metadata has been updated to account for that platform's current state, there are still several tests/dirs that are frequently intermittently failing in unexpected ways. Frequent enough that it prevents me from moving these tests from tier-3 to tier-2 (where they get shown by default in treeherder and sheriffs actively file bugs about intermittent failures).

I'm going to disable my way to victory on these tests so I can bump them to tier-2 in the next week or so. (Can't do tier-1, we don't have the capacity at the moment to run these per-push on integration branches, so they only run on mozilla-central and try at the moment. Treeherder's Add New Jobs (Search) feature CAN run these jobs on inbound and autoland if you filter for android x86_64 web-platform.)

Setting the bug as leave-open for now because I expect I'll have to play whackamole on the tests over a few rounds of disablings to get things truly green. I'll resolve the bug when I bump the jobs to tier-2.

This disables a few additional tests and updates the annotations for other disabled tests to point to this bug instead of the general geckoview-wpt bug.

Assignee: nobody → wkocher
Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/cd771a0b9c07
Disable some additional frequently failing geckoview wpt tests
Priority: -- → P3
Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/c23eb772cf2d
Disable some additional frequently failing geckoview wpt tests
Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/64bfa1117b35
disable some additional frequently failing geckoview wpt tests
Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e1080b62268c
Disable some more frequently failing wpt tests on geckoview

This patch runs wpt against opt and debug builds of geckoview as tier-2 jobs on mozilla-central and try.

This takes the android wpt jobs that have previously been constrained to running against mozilla-central and try, and will begin also running them on integration branches.

Depends on D31581

Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a4c80ac02e50
Run geckoview wpt as tier-2 jobs r=jgraham

They're tier-2 now, so I think we can resolve this bug. I might still use this as a meta bug for tracking all of the gv wpt failures that sheriffs file, at least for a while until they get greened up more.

Keywords: leave-open

Bug 1553584 was filed and shows a race condition in the test harness that could make fixing bugs difficult. Bumping these back to tier-3 until that can be fixed.

We need to rehide these until bug 1553584 can be fixed.

Keywords: leave-open
Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/217038bea1c7
Bump android gv wpt back up to tier-3

I think we should have put the reftests down to Tier 3 and left the other tests as Tier 2.

Taskcluster didn't correctly apply the change back to tier-3 in the previous patch because the wildcard setting the tier to 2 by default beat out the specific line setting tier-3 for the geckoview tests. This patch specifically changes the tier for non-geckoview reftests (fennec runs on x86 emulators) to tier-2, while geckoview reftests (TRA runs on x86_64 emulators) to tier-3.

This change also explicitly sets the non-reftest tests to tier-2 for all android platforms, because they didn't have the permafails that the reftests seem to have.

Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/61e78fd6c946
Clear up ambiguities in android wpt tier definitions so they apply correctly.
Depends on: 1557375
Alias: wpt-geckoview
See Also: → 1544621
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/autoland/rev/59ed9ee0bd98
Bump web-platform-tests-reftests back to Tier-2 on Android; r=jgraham
Pushed by mjzffr@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/253fa3bd2d74
Run the android web-platform tests on all built-projects r=gbrown,coop
Regressions: 1560268

I'm going to update expectation metadata for a few more tests, then resolve this bug.

Attachment #9073100 - Attachment description: Bug 1550895 - Mark more tests as expected FAIL to green up wpt on GeckoView; r?jgraham → Bug 1550895 - Disable a few more tests to green up wpt on GeckoView; r=jgraham
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/autoland/rev/7b99cdff2976
Disable a few more tests to green up wpt on GeckoView; r=jgraham

These have been Tier-2 for a while so let's close this bug.

A number of tests have been disabled in the past months: I'll organize the work to be done under Bug 1563334.

Status: NEW → RESOLVED
Closed: 1 year ago
Keywords: leave-open
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.