Bug 1550895 (wpt-geckoview)

Green up geckoview wpt so they can be upgraded to tier-2 visibility.

NEW
Assigned to

Status

()

task
P3
normal
Last month
5 days ago

People

(Reporter: KWierso, Assigned: KWierso)

Tracking

(Depends on 6 bugs, Blocks 4 bugs, {leave-open})

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(8 attachments)

Assignee

Description

Last month

Now that the wpt suite is running against geckoview and expectation metadata has been updated to account for that platform's current state, there are still several tests/dirs that are frequently intermittently failing in unexpected ways. Frequent enough that it prevents me from moving these tests from tier-3 to tier-2 (where they get shown by default in treeherder and sheriffs actively file bugs about intermittent failures).

I'm going to disable my way to victory on these tests so I can bump them to tier-2 in the next week or so. (Can't do tier-1, we don't have the capacity at the moment to run these per-push on integration branches, so they only run on mozilla-central and try at the moment. Treeherder's Add New Jobs (Search) feature CAN run these jobs on inbound and autoland if you filter for android x86_64 web-platform.)

Setting the bug as leave-open for now because I expect I'll have to play whackamole on the tests over a few rounds of disablings to get things truly green. I'll resolve the bug when I bump the jobs to tier-2.

Assignee

Comment 1

Last month

This disables a few additional tests and updates the annotations for other disabled tests to point to this bug instead of the general geckoview-wpt bug.

Assignee

Updated

Last month
Assignee: nobody → wkocher

Comment 2

Last month
Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/cd771a0b9c07
Disable some additional frequently failing geckoview wpt tests
Assignee

Updated

Last month
Type: defect → task
Priority: -- → P3

Comment 5

Last month
Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/c23eb772cf2d
Disable some additional frequently failing geckoview wpt tests

Comment 8

Last month
Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/64bfa1117b35
disable some additional frequently failing geckoview wpt tests

Comment 11

Last month
Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e1080b62268c
Disable some more frequently failing wpt tests on geckoview
Assignee

Comment 13

Last month

This patch runs wpt against opt and debug builds of geckoview as tier-2 jobs on mozilla-central and try.

Assignee

Comment 14

Last month

This takes the android wpt jobs that have previously been constrained to running against mozilla-central and try, and will begin also running them on integration branches.

Depends on D31581

Comment 15

29 days ago
Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a4c80ac02e50
Run geckoview wpt as tier-2 jobs r=jgraham
Assignee

Comment 17

27 days ago

They're tier-2 now, so I think we can resolve this bug. I might still use this as a meta bug for tracking all of the gv wpt failures that sheriffs file, at least for a while until they get greened up more.

Keywords: leave-open
Assignee

Updated

27 days ago
Blocks: 1553408
Assignee

Updated

27 days ago
Blocks: 1553407
Assignee

Updated

27 days ago
Blocks: 1553440
Assignee

Updated

27 days ago
Blocks: 1553046
Assignee

Updated

27 days ago
Blocks: 1553045
Assignee

Updated

27 days ago
Blocks: 1464643
Assignee

Updated

27 days ago
Blocks: 1553652
Assignee

Comment 18

27 days ago

Bug 1553584 was filed and shows a race condition in the test harness that could make fixing bugs difficult. Bumping these back to tier-3 until that can be fixed.

Assignee

Updated

27 days ago
Depends on: 1553584
Assignee

Comment 19

27 days ago

We need to rehide these until bug 1553584 can be fixed.

Keywords: leave-open

Comment 20

27 days ago
Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/217038bea1c7
Bump android gv wpt back up to tier-3

I think we should have put the reftests down to Tier 3 and left the other tests as Tier 2.

Assignee

Comment 23

21 days ago

Taskcluster didn't correctly apply the change back to tier-3 in the previous patch because the wildcard setting the tier to 2 by default beat out the specific line setting tier-3 for the geckoview tests. This patch specifically changes the tier for non-geckoview reftests (fennec runs on x86 emulators) to tier-2, while geckoview reftests (TRA runs on x86_64 emulators) to tier-3.

This change also explicitly sets the non-reftest tests to tier-2 for all android platforms, because they didn't have the permafails that the reftests seem to have.

Comment 24

21 days ago
Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/61e78fd6c946
Clear up ambiguities in android wpt tier definitions so they apply correctly.
Depends on: 1557375
You need to log in before you can comment on or make changes to this bug.