1219437 - Implement two-tier support in SETA, to allow some tests to be coalesced more aggressively than others

Status: RESOLVED INVALID

(Testing :: General, defect)

Description

[Jonathan Griffin (:jgriffin)]

We'd like to use this ability to coalesce Windows XP tests twice as aggressively as other SETA jobs; this should help free up some capacity so we can move additional XP test machines to the Windows 7 pool, thereby allowing us to enable more e10s tests there.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

Kim added bug 1175568 as a dependent bug.  That will support per platform.  If we want a multi tiered approach to SETA, we would need to have SETA output the skip counts as part of the results from the API.  Then we could add weights to it.

That would require more SETA changes and then changes to the buildbot ingestion code as well.  It is probably twice the volume of work as bug 1175568, which isn't too much, but another day of hacking.  We would need to determine where the right place is to put this information.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

we have support for per platform skipconfig settings in buildbot now.  This is really cool.  Do we have a need for a more comprehensive tiered system?  If we were to build that, then it would most likely need to be held inside the SETA API.  A few benefits to this:
* we could determine from mozci/pulse-actions what the skipcount is for a specific job and backfill the appropriate limit
* we could defined as many tiers as we want with one set of changes

The problem is that this would require a reasonable amount of work in SETA as well as buildbot.  If other tools would take advantage of this, they would need work as well.  I wanted to poll the key players here to determine any interest.  if there is none, we can close this.  If there is a little we can leave this around, if there is a lot I can make the SETA piece happen in November.

Comment 3

[Armen [:armenzg]]

I believe I can grab the data I need from allthething.json by parsing the name of the schedulers, however, this might be useful for when we move SETA to TC.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

I am not sure if this bug applies in the land of taskcluster fully.  As this is 13 months old, I would like to see us close this and open any new bugs that are more specific to the current landscape and treeherder/taskcluster.