Closed Bug 1134349 Opened 9 years ago Closed 8 years ago

Reconsider Android x86 "sets"

Categories

(Testing :: General, defect)

Product:
Testing
Component:
General
Platform:
x86_64
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: gbrown, Assigned: gbrown)

References

Details

Android x86 tests run in "sets" of up to 4 test suites per set. This allows us to run up to 4 jobs concurrently on the same ix test slave, without significantly increasing the wall time required for each job -- it's very efficient.

But there are disadvantages to sets:
 - Failure reporting in treeherder is non-standard and a little confusing. If one job fails, the set is turned orange; if 2 or more jobs fail, the set is turned orange just the same. 
 - If a suite has a significant on-going problem, sheriffs can only hide the entire set, impacting suites that may be running fine.
 - trychooser syntax is confusing and problematic (bug 964589).

:ryanvm does not like sets.

We currently only run xpcshell in set S4, and S4 is the only set run on all trees (S1/2/3 are on cedar), so running one suite at a time would not be a significant change...but we are considering adding crashtests to S4 in bug 1132210...and who knows what the future of x86 testing holds?!
See Also: → 1164866
Assignee: gbrown → nobody
On taskcluster, Android x86 tests run on aws, one test per instance. The "set" concept will retire with buildbot Android x86 tests.
Assignee: nobody → gbrown
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.