Update in-tree task definitions to use the right Tier

RESOLVED FIXED

Status

Taskcluster
Task Configuration
RESOLVED FIXED
5 months ago
5 months ago

People

(Reporter: camd, Assigned: dustin)

Tracking

Details

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(5 attachments)

(Reporter)

Updated

5 months ago
Depends on: 1387640
I can help with the tier changes, but I'd like a little help figuring out which jobs.  Is there a way to get a list?  Even just a list of <groupSymbol>(<symbol>) would be good.
(Reporter)

Comment 2

5 months ago
(In reply to Dustin J. Mitchell [:dustin] from comment #1)
> I can help with the tier changes, but I'd like a little help figuring out
> which jobs.  Is there a way to get a list?  Even just a list of
> <groupSymbol>(<symbol>) would be good.

You can view the list in Treeherder with the links above, if that helps.  Or I can create a text or json output if you like.  I'll attach a couple json blobs for you.  Unfortunately the funsize jobs are a huge list...
(Reporter)

Comment 3

5 months ago
Created attachment 8905713 [details]
mozilla-central-1
(Reporter)

Comment 4

5 months ago
Created attachment 8905714 [details]
mozilla-central-2
(Reporter)

Comment 5

5 months ago
Created attachment 8905716 [details]
try.json
Oh, that's awesome, thanks!
Assignee: nobody → dustin
(Reporter)

Comment 7

5 months ago
I may have one or two more of those attachments to add today.  But good to know that's a good format for you.  :)
That is probably enough for now -- I expect there will be overlap with the other quries.  So I'll get this much fixed up and landed, and then we can revisit.  One thing that might be easy though: can you dump out the existing exclusion rules and I'll seek out the task definitions to which they apply?
(Reporter)

Comment 9

5 months ago
Created attachment 8906050 [details]
exclusion_profile_values.json

Not sure if this format will help you or not.  It was the easiest way to extract the values that make up the exclusion profiles from the DB.  If you need something different, please let me know.

Though, like you say, after you update some, we can revisit and the list will get shorter and shorter.  

In reality, the worst thing that will happen here is that a few tests that should be hidden will start popping up, and we can then hide them.  Probably not catastrophic.  Just annoying...  :)
(Reporter)

Updated

5 months ago
Blocks: 1387640
No longer depends on: 1387640
So, the bulk of those are funzise, and that has only just landed in bug 1342392 and are already at tier 3.  You may have seen them in try in an earlier revision of this patch.

That just leaves

linux64/opt tc-M-V(<chunk>)
linux64-nightly/opt tc-M-V(<chunk>)
windows10-64/opt tc-M-e10s(<chunk>)

tc-M-V was fixed in bug 1391708

As for tc-M-e10s, the exclusion profiles suggest that should be disabled for both e10s and non-e10s.  That's a quick fix.

I really don't understand the rest of it (the exclusion profiles specifically) well enough.  Just too much impedence mismatch between treeherder and the task graph generation.

It's much better to show something that should be hidden, I think, than the reverse.  And we shouldn't be hiding anything except when there's a project underway to green it up -- anything else is a (potentially huge) waste of resources.  So I think we should ship this, disable the exclusions, and then fix up anything that appears unexpectedly.
Comment hidden (mozreview-request)
(Assignee)

Updated

5 months ago
Attachment #8906780 - Flags: review?(janus926)
(guessing at reviewer based on bug 1384433)

Updated

5 months ago
Depends on: 1391708
Attachment #8906780 - Flags: review?(janus926) → review?(jmaher)

Comment 13

5 months ago
mozreview-review
Comment on attachment 8906780 [details]
Bug 1397877: put windows10-64/opt at tier 3 (previously a treeherder exclusion)

https://reviewboard.mozilla.org/r/178504/#review183602

window10 opt is green:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-searchStr=windows%20x64%20mochitest-e10s%20tc&group_state=expanded

if there are intermittents we should push to fix or disable the specific tests- in fact almost all tests are running on windows 10 now and green <15 know failures to resolve.
Attachment #8906780 - Flags: review?(jmaher) → review-
Awesome.  Then as far as I can see, tiers are correct, and there may be some bugs in the exclusion profiles.  Those bugs will go away when they're turned off :)

Cam / Ed: maybe the right next steps are to start disabling exclusion profiles one by one, and if someone screams put them back and address the tier issue that has been uncovered?  I suspect we'll find a few more cases like this where we have been unintentially excluding green jobs.  If we find the reverse, it's easy to fix in-tree.
(Reporter)

Comment 15

5 months ago
Fantastic!  Thanks Dustin!  Yeah, I agree with your position in comment 10.  And not a bad idea to disable the exclusions one by one and measure the screaming...  :D  Thanks for all your help on this!!
(Reporter)

Comment 16

5 months ago
BTW - I have to push my first PR in the project out to prod before we start disabling exclusions.  But shortly after that, I'll start that process.
I'm happy to be cc'd on any screaming and do the in-tree tier changes.
I think the remainder of this is on the parent bug (and the to-be-filed "screaming" bugs :)
Status: NEW → RESOLVED
Last Resolved: 5 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.