1387640 - Use Tiers instead of exclusion profiles

Status: RESOLVED FIXED

(Tree Management :: Treeherder, enhancement, P1)

Description

[Cameron Dawson [:camd]]

Some sub-bugs that should come from this bug are:

1. Stop applying a Tier to TaskCluster jobs based on exclusion profiles.  Just use the value that is specified in the task definition.  Tier-3 jobs are hidden.

2. Create a white or black list in the Treeherder codebase ETL layer for jobs from BuildBot that will be hidden.  These jobs will be set to Tier-3

3. Create a similar list for Buildbot Tier-2 jobs.  The only Buildbot Tier-2 jobs I can find on m-i, m-c and autoland are Autophone and Talos.  So this should be a pretty short list.

The proposal for this is a work-in-progress since it can have wide-ranging effects:
https://docs.google.com/document/d/1-BzCN97aCZKovWeh1zU6ASyp_FVtg0mbITN7C_iWXzg/edit#heading=h.dhpfyfxjvgg9

Comment 1

[Cameron Dawson [:camd]]

Hidden Taskcluster Jobs:

Looks like funsize and valgrind jobs need to be set to Tier-3

mozilla-central: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&exclusion_profile=default&visibility=excluded&filter-tier=1&filter-tier=2&filter-tier=3&filter-build_system_type=taskcluster

None of the rest of these have any hidden TC jobs when I checked just now:

mozilla-inbound:  https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&exclusion_profile=default&visibility=excluded&filter-tier=1&filter-tier=2&filter-tier=3&filter-build_system_type=taskcluster

try: https://treeherder.mozilla.org/#/jobs?repo=try&exclusion_profile=default&visibility=excluded&filter-tier=1&filter-tier=2&filter-tier=3&filter-build_system_type=taskcluster

autoland: https://treeherder.mozilla.org/#/jobs?repo=try&exclusion_profile=default&visibility=excluded&filter-tier=1&filter-tier=2&filter-tier=3&filter-build_system_type=taskcluster

Comment 2

[Ed Morley [:emorley]]

Bug 1390192 is another example of how the new way will mean less confusion all around.

CCing Andrew, since it sounds like we all want the same thing:
https://groups.google.com/forum/#!topic/mozilla.tools.treeherder/ygqPmBob3Z0

Comment 3

[Cameron Dawson [:camd]]

I'm going to start working on this bug this month, so bumping to P1.

Comment 4

[Cameron Dawson [:camd]]

The first PR for bug 1398386 will switch so the ingestion sets tiers for jobs, including Tier-3 for buildbot jobs that should be hidden.

The second PR will make the /jobs endpoint stop checking the exclusion profiles and remove the models for excluding.  It will also remove using the exclusion param from the UI.

Comment 5

[GitHub Autolander Bot]

Attachment: [treeherder] mozilla:remove-exclusion-profiles-2 > mozilla:master

Comment 6

[Cameron Dawson [:camd]]

Comment on attachment 8907260 [details] [review]
[treeherder] mozilla:remove-exclusion-profiles-2 > mozilla:master

This will be followed up by a PR that removes all the models and tables for exclusion profiles.  I just wanted to leave them intact juuuuust in case there was a problem.

Comment 7

[Ed Morley [:emorley]]

Comment on attachment 8907260 [details] [review]
[treeherder] mozilla:remove-exclusion-profiles-2 > mozilla:master

(In reply to Cameron Dawson [:camd] from comment #6)
> I just wanted to leave them intact juuuuust in case there was a problem.

Good idea :-)

Comment 8

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/bde8e20f1f950c8edb953b1278b33575cc45da72
Bug 1387640 - Stop using exclusion profiles

Note: This leaves the models and tables intact so that we can
revert without data loss in case we discover an issue.  A follow-up
commit will remove those tables and models.

Comment 9

[GitHub Autolander Bot]

Attachment: [treeherder] mozilla:remove-exclusion-profiles-3 > mozilla:master

Comment 10

[Cameron Dawson [:camd]]

Comment on attachment 8908265 [details] [review]
[treeherder] mozilla:remove-exclusion-profiles-3 > mozilla:master

Thanks for the previous pass.  I believe I addressed your concerns.

Comment 11

[Ed Morley [:emorley]]

Comment on attachment 8908265 [details] [review]
[treeherder] mozilla:remove-exclusion-profiles-3 > mozilla:master

Looks great!

How long should we wait before merging this to stage and then prod?

Comment 12

[Cameron Dawson [:camd]]

I was thinking soaking PR2 for maybe 1 week.  I'm fairly certain we can address any non-hiddenness quickly enough that we won't need to re-enable exclusion profiles, but there's little hurry to remove the tables once they're not in use.  After 1 week and nobody is asking for my head, then I'll merge this latest PR.  :)

Comment 13

[Ed Morley [:emorley]]

Attachment: Exclusion profile export

I've exported the current exclusion profiles in case it's useful to refer back to them in the future (with a bit of editing to remove the chronic thousands of entries for the ~"all" case of the particular platform/...).

Now we have this export (which will help with figuring out why things were hidden/bug numbers etc), perhaps we can land the last PR here sooner than a few week away?

Comment 14

[Cameron Dawson [:camd]]

Sure, sounds good to me!  Thanks for creating that export.  I'll go ahead and merge now.  :)

Comment 15

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/863f772a8b13bfb2039574937c0480b5b00c52d5
Bug 1387640 - Remove Exclusion Profiles models/tables

Removes all the models and tables for the ExclusionProfile and
JobExclusion models and related.