[raptor] Hide raptor tasks that post to TH production

RESOLVED FIXED

Status

Tree Management
Visibility Requests
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: rwood, Assigned: RyanVM)

Tracking

Details

Attachments

(1 attachment)

(Reporter)

Description

3 years ago
I am ready to turn on some raptor performance tests via taskcluster gaia, that will report to treeherder gaia production; however I would like to have the tasks hidden on TH please until we iron out any issues. Thanks!

There are three raptor tasks, each reporting to the same TH group, here's the info:

"treeherder": {
  "groupSymbol": "Rpt",
  "groupName": "Raptor Performance Tests",
  "symbol": "se1"
},

"treeherder": {
  "groupSymbol": "Rpt",
  "groupName": "Raptor Performance Tests",
  "symbol": "se2"
},

"treeherder": {
  "groupSymbol": "Rpt",
  "groupName": "Raptor Performance Tests",
  "symbol": "se"
},
(Reporter)

Updated

3 years ago
Blocks: 1149266
(Assignee)

Comment 1

3 years ago
Jobs can't be hidden with Treeherder until they've run on a given tree.
(Reporter)

Comment 2

3 years ago
Ah, thanks Ryan. The Raptor tasks are now turned on and reporting to gaia TH production. If they could be hidden now please that'd be awesome, thanks!

https://treeherder.mozilla.org/#/jobs?repo=gaia&revision=8ce7900c55bbc0dba4c2f687ebc7c15e4838c954
Flags: needinfo?(ryanvm)
(Reporter)

Comment 3

3 years ago
Note: They're currently on TH gaia under 'Rpt' emulator-ics and 'Rpt' gaia; I will be moving them all under the same section eventually (Bug 1159345) but if you could please hide them all that'd be great.
(Reporter)

Comment 4

3 years ago
Update - all the raptor tasks are now reporting in the 'Rpt' treeherder group under 'gaia opt'
(Assignee)

Comment 5

3 years ago
Hidden.
Assignee: nobody → ryanvm
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Flags: needinfo?(ryanvm)
Resolution: --- → FIXED
(Reporter)

Comment 6

3 years ago
We changed the worker type for the raptor tests, but now they are not hidden anymore on treeherder :(

https://treeherder.mozilla.org/#/jobs?repo=gaia

The Rpt group is appearing at the top (in the raptor opt section).

Ryan (or another sheriff) could you please update so the raptor tasks are hidden on treeherder again please? Thanks very much!
Status: RESOLVED → REOPENED
Flags: needinfo?(ryanvm)
Resolution: FIXED → ---
(Assignee)

Comment 7

3 years ago
I'm trying but it's not taking for some reason. Any idea what's going on, Cam?
Flags: needinfo?(ryanvm) → needinfo?(cdawson)
Investigating now.  I checked the settings and they all LOOK right.  But I can see that the exclusion profile does NOT contain the signatures it should.  

Ahh, so there are more than 1 job types using the same symbol "se2".  And we're getting confused there and using the wrong job signature for it.  So we're "hiding" the wrong thing.  

I think we should be able to handle this case.  But it's on data ingestion as well as the exclusion profile generation.  Saving the job is getting the wrong signature.  And Adding the signature to the exclusion list is ALSO getting the wrong signature.

Fwiw: these are the three job types that have the same symbol:

Raptor Launch Test
Settings Launch Test (on base + pr)
Settings Launch Test (on base -1 commit)
Flags: needinfo?(cdawson)

Updated

3 years ago
Depends on: 1166118
Ryan - Would you consider this a candidate for your Top-5 list?
Flags: needinfo?(ryanvm)
(Assignee)

Comment 10

3 years ago
Gaia's not a tree we monitor, so no. Can't speak for those who do look at it, though.
Flags: needinfo?(ryanvm)
Ok.  Well, I'll keep it on my radar, because this could potentially show in other branches, too.  I'm not certain about that, but possibly.
(Reporter)

Comment 12

3 years ago
There is no need to hide the Raptor tasks anymore, as we are very close to having raptor ready to be 'live' anyway.
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
Resolution: --- → WONTFIX
(Reporter)

Comment 13

3 years ago
Reopened, as we wish to hide the raptor tasks again on treeherder (too much results variance causing false alarms).
Status: RESOLVED → REOPENED
Resolution: WONTFIX → ---
(Reporter)

Comment 14

3 years ago
Created attachment 8623807 [details] [review]
https://github.com/mozilla-b2g/gaia/pull/30632

Temporary work-around (as hiding is blocked at the moment); just don't post the raptor-tester and raptor-results tasks on treeherder. Tests will still run, and can be looked at via the raptor decision task which will still be visible.
Attachment #8623807 - Flags: review?(garndt)

Updated

3 years ago
Attachment #8623807 - Flags: review?(garndt) → review+
(Reporter)

Comment 16

3 years ago
The Raptor task names are now changed and reporting to treeherder production (gaia and gaia-master) as unique names only:

Raptor(cal clk con dlr eml fm gal mus set sms tst vid)

i.e. https://treeherder.mozilla.org/#/jobs?repo=gaia&revision=0f13023ab56c3276a5542121c259684204041a39

Can all of the Raptor tasks please be hidden on treeherder ASAP please? Thanks :camd, appreciated!
Flags: needinfo?(cdawson)
OK, the tests mentioned in comment 16 are now hidden.  Please let me know if you need anything else hidden that I missed.
Flags: needinfo?(cdawson)
(Reporter)

Comment 18

3 years ago
Looks awesome, thanks Cameron!
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.