Closed Bug 1397877 Opened 7 years ago Closed 7 years ago

Update in-tree task definitions to use the right Tier

Categories

(Firefox Build System :: Task Configuration, task)

task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: camd, Assigned: dustin)

References

Details

Attachments

(5 files)

Depends on: 1387640
I can help with the tier changes, but I'd like a little help figuring out which jobs.  Is there a way to get a list?  Even just a list of <groupSymbol>(<symbol>) would be good.
(In reply to Dustin J. Mitchell [:dustin] from comment #1)
> I can help with the tier changes, but I'd like a little help figuring out
> which jobs.  Is there a way to get a list?  Even just a list of
> <groupSymbol>(<symbol>) would be good.

You can view the list in Treeherder with the links above, if that helps.  Or I can create a text or json output if you like.  I'll attach a couple json blobs for you.  Unfortunately the funsize jobs are a huge list...
Attached file mozilla-central-1
Attached file mozilla-central-2
Attached file try.json
Oh, that's awesome, thanks!
Assignee: nobody → dustin
I may have one or two more of those attachments to add today.  But good to know that's a good format for you.  :)
That is probably enough for now -- I expect there will be overlap with the other quries.  So I'll get this much fixed up and landed, and then we can revisit.  One thing that might be easy though: can you dump out the existing exclusion rules and I'll seek out the task definitions to which they apply?
Not sure if this format will help you or not.  It was the easiest way to extract the values that make up the exclusion profiles from the DB.  If you need something different, please let me know.

Though, like you say, after you update some, we can revisit and the list will get shorter and shorter.  

In reality, the worst thing that will happen here is that a few tests that should be hidden will start popping up, and we can then hide them.  Probably not catastrophic.  Just annoying...  :)
Blocks: 1387640
No longer depends on: 1387640
So, the bulk of those are funzise, and that has only just landed in bug 1342392 and are already at tier 3.  You may have seen them in try in an earlier revision of this patch.

That just leaves

linux64/opt tc-M-V(<chunk>)
linux64-nightly/opt tc-M-V(<chunk>)
windows10-64/opt tc-M-e10s(<chunk>)

tc-M-V was fixed in bug 1391708

As for tc-M-e10s, the exclusion profiles suggest that should be disabled for both e10s and non-e10s.  That's a quick fix.

I really don't understand the rest of it (the exclusion profiles specifically) well enough.  Just too much impedence mismatch between treeherder and the task graph generation.

It's much better to show something that should be hidden, I think, than the reverse.  And we shouldn't be hiding anything except when there's a project underway to green it up -- anything else is a (potentially huge) waste of resources.  So I think we should ship this, disable the exclusions, and then fix up anything that appears unexpectedly.
Attachment #8906780 - Flags: review?(janus926)
(guessing at reviewer based on bug 1384433)
Depends on: 1391708
Attachment #8906780 - Flags: review?(janus926) → review?(jmaher)
Comment on attachment 8906780 [details]
Bug 1397877: put windows10-64/opt at tier 3 (previously a treeherder exclusion)

https://reviewboard.mozilla.org/r/178504/#review183602

window10 opt is green:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-searchStr=windows%20x64%20mochitest-e10s%20tc&group_state=expanded

if there are intermittents we should push to fix or disable the specific tests- in fact almost all tests are running on windows 10 now and green <15 know failures to resolve.
Attachment #8906780 - Flags: review?(jmaher) → review-
Awesome.  Then as far as I can see, tiers are correct, and there may be some bugs in the exclusion profiles.  Those bugs will go away when they're turned off :)

Cam / Ed: maybe the right next steps are to start disabling exclusion profiles one by one, and if someone screams put them back and address the tier issue that has been uncovered?  I suspect we'll find a few more cases like this where we have been unintentially excluding green jobs.  If we find the reverse, it's easy to fix in-tree.
Fantastic!  Thanks Dustin!  Yeah, I agree with your position in comment 10.  And not a bad idea to disable the exclusions one by one and measure the screaming...  :D  Thanks for all your help on this!!
BTW - I have to push my first PR in the project out to prod before we start disabling exclusions.  But shortly after that, I'll start that process.
I'm happy to be cc'd on any screaming and do the in-tree tier changes.
I think the remainder of this is on the parent bug (and the to-be-filed "screaming" bugs :)
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Product: TaskCluster → Firefox Build System
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: