Closed Bug 1567138 Opened 6 months ago Closed 4 months ago

Promote AWSY tp6 tests to tier 1

Categories

(Testing :: AWSY, task, P1)

Version 3
task

Tracking

(firefox71 fixed)

RESOLVED FIXED
mozilla71
Tracking Status
firefox71 --- fixed

People

(Reporter: davehunt, Assigned: stephend)

References

Details

Attachments

(1 file)

The AWSY tests are currently running as tier 2 due to frequent network issues, which may be related to the bug 1517323, which has been fixed for Raptor. Once bug 1567137 is resolved, let's review the failure rate and see if we can promote these tests to tier 1.

Blocks: 1567122

See bug 1540886 for more background on the failures we were seeing.

Depends on: 1540886

It looks like we haven't seen any intermittents like bug 1540886 or bug 1541183 in around 4 months. I think we can promote this test to tier 1.

Assignee: nobody → stephen.donner
Status: NEW → ASSIGNED
Priority: -- → P1

Roping in Nathan, David, and Gabriele - apologies for casting the net wide, and without much on-IRC heads-up (and thanks for your help on bug 1549272, Gabriele, David!)

The few failures [0][1] (earlier retriggers have passed, since, and there are new ones pending) seem mostly in variants of bug 1543552 and bug 1377988, and also seem limited (in my few runs) to Linux x64 CCov debug, which is also currently tier 2.

Thoughts/comments, both on general approach with awsy tp6 (before, and later, with Fission), and specific to the patch and intended coverage, here?

The need-info for you, Nathan, is to help load-balance me bugging Gabriele and David elsewhere :-)

Cheers!

[0] https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&author=sdonner%40mozilla.com&selectedJob=267490692
[1] https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&author=sdonner%40mozilla.com&selectedJob=267517907

Flags: needinfo?(nfroyd)

(In reply to Stephen Donner [:stephend] from comment #5)

Roping in Nathan, David, and Gabriele - apologies for casting the net wide, and without much on-IRC heads-up (and thanks for your help on bug 1549272, Gabriele, David!)

The few failures [0][1] (earlier retriggers have passed, since, and there are new ones pending) seem mostly in variants of bug 1543552 and bug 1377988, and also seem limited (in my few runs) to Linux x64 CCov debug, which is also currently tier 2.

It's not clear to me that we should even be running talos tests against coverage builds in the first place, so I wouldn't worry too much about that...but the awsy-tp6 tests are not supposed to be running against coverage builds:

https://hg.mozilla.org/try/rev/4074a1a78d95bdc4f3f9eaca8396952b05f28c3a#l1.5

so why are they running against coverage builds? Is this a casing issue (CCov vs. ccov) in the job name?

Flags: needinfo?(nfroyd)

(In reply to Nathan Froyd [:froydnj] from comment #6)

It's not clear to me that we should even be running talos tests against coverage builds in the first place, so I wouldn't worry too much about that...but the awsy-tp6 tests are not supposed to be running against coverage builds:

https://hg.mozilla.org/try/rev/4074a1a78d95bdc4f3f9eaca8396952b05f28c3a#l1.5

so why are they running against coverage builds? Is this a casing issue (CCov vs. ccov) in the job name?

From what I /think/ I've discovered so far, the job-defaults here[2] for awsy (which I think awsy-tp6 inherits, and only overrides when needed) point to awsy_script.py. That (appears to) unilaterally import and set up CodeCoverageMixin[3], but that's just a slightly-better-than-wild guess.

[2] https://searchfox.org/mozilla-central/rev/e18057a9613ffda06dfd3640209ca234ed7dc37d/taskcluster/ci/test/awsy.yml#6
[3] https://searchfox.org/mozilla-central/rev/e18057a9613ffda06dfd3640209ca234ed7dc37d/testing/mozharness/scripts/awsy_script.py#30-32

I'll land the tier-1 promotion bit, and file the ccov issue separately, and cross-reference here.

Pushed by sdonner@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/96844e92a0f6
Promote AWSY tp6 tests to tier 1. r=froydnj,gsvelto
Status: ASSIGNED → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla71
You need to log in before you can comment on or make changes to this bug.