|Submitter||Diff||Changes||Open Issues||Last Updated|
|Error loading review requests:|
39.46 KB, application/json
1.31 KB, application/json
157 bytes, application/json
2.03 MB, application/json
59 bytes, text/x-review-board-request
|Details | Review|
Some jobs are hidden in Treeherder, but do not have the correct Tier of 3. In Bug 1387640 we are moving to using Tiers only, instead of the exclusion profiles. So we'd like to get these all set properly so they are hidden. Some of the jobs that are hidden, but not yet tier-3 can be found in these queries: Try: https://treeherder.mozilla.org/#/jobs?repo=try&exclusion_profile=default&visibility=excluded&revision=d957263190348fc3c1ee70aacf8f161b791d03e7&group_state=expanded Mozilla-Central: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&exclusion_profile=default&visibility=excluded&revision=f64e2b4dcf5eec0b4ad456c149680a67b7c26dc4 https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&exclusion_profile=default&visibility=excluded&revision=e8400551c2e39f24c75a009ebed496c7acd7bf47 Mozilla-Beta: https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&exclusion_profile=default&visibility=excluded&revision=bfb0af0451ec5acf746f103e039c644f0efab641 Mozilla-ESR52: https://treeherder.mozilla.org/#/jobs?repo=mozilla-esr52&exclusion_profile=default&visibility=excluded&revision=f01155fe4d540cc4b6eac2bc43a996e5a7d48262 Mozilla-Release: https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&exclusion_profile=default&visibility=excluded Can someone update these tests? Or help direct me to whom I should file a bug against to get them fixed?
I can help with the tier changes, but I'd like a little help figuring out which jobs. Is there a way to get a list? Even just a list of <groupSymbol>(<symbol>) would be good.
(In reply to Dustin J. Mitchell [:dustin] from comment #1) > I can help with the tier changes, but I'd like a little help figuring out > which jobs. Is there a way to get a list? Even just a list of > <groupSymbol>(<symbol>) would be good. You can view the list in Treeherder with the links above, if that helps. Or I can create a text or json output if you like. I'll attach a couple json blobs for you. Unfortunately the funsize jobs are a huge list...
Oh, that's awesome, thanks!
Assignee: nobody → dustin
I may have one or two more of those attachments to add today. But good to know that's a good format for you. :)
That is probably enough for now -- I expect there will be overlap with the other quries. So I'll get this much fixed up and landed, and then we can revisit. One thing that might be easy though: can you dump out the existing exclusion rules and I'll seek out the task definitions to which they apply?
Created attachment 8906050 [details] exclusion_profile_values.json Not sure if this format will help you or not. It was the easiest way to extract the values that make up the exclusion profiles from the DB. If you need something different, please let me know. Though, like you say, after you update some, we can revisit and the list will get shorter and shorter. In reality, the worst thing that will happen here is that a few tests that should be hidden will start popping up, and we can then hide them. Probably not catastrophic. Just annoying... :)
So, the bulk of those are funzise, and that has only just landed in bug 1342392 and are already at tier 3. You may have seen them in try in an earlier revision of this patch. That just leaves linux64/opt tc-M-V(<chunk>) linux64-nightly/opt tc-M-V(<chunk>) windows10-64/opt tc-M-e10s(<chunk>) tc-M-V was fixed in bug 1391708 As for tc-M-e10s, the exclusion profiles suggest that should be disabled for both e10s and non-e10s. That's a quick fix. I really don't understand the rest of it (the exclusion profiles specifically) well enough. Just too much impedence mismatch between treeherder and the task graph generation. It's much better to show something that should be hidden, I think, than the reverse. And we shouldn't be hiding anything except when there's a project underway to green it up -- anything else is a (potentially huge) waste of resources. So I think we should ship this, disable the exclusions, and then fix up anything that appears unexpectedly.
Attachment #8906780 - Flags: review?(janus926)
(guessing at reviewer based on bug 1384433)
Attachment #8906780 - Flags: review?(janus926) → review?(jmaher)
Comment on attachment 8906780 [details] Bug 1397877: put windows10-64/opt at tier 3 (previously a treeherder exclusion) https://reviewboard.mozilla.org/r/178504/#review183602 window10 opt is green: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-searchStr=windows%20x64%20mochitest-e10s%20tc&group_state=expanded if there are intermittents we should push to fix or disable the specific tests- in fact almost all tests are running on windows 10 now and green <15 know failures to resolve.
Attachment #8906780 - Flags: review?(jmaher) → review-
Awesome. Then as far as I can see, tiers are correct, and there may be some bugs in the exclusion profiles. Those bugs will go away when they're turned off :) Cam / Ed: maybe the right next steps are to start disabling exclusion profiles one by one, and if someone screams put them back and address the tier issue that has been uncovered? I suspect we'll find a few more cases like this where we have been unintentially excluding green jobs. If we find the reverse, it's easy to fix in-tree.
Fantastic! Thanks Dustin! Yeah, I agree with your position in comment 10. And not a bad idea to disable the exclusions one by one and measure the screaming... :D Thanks for all your help on this!!
BTW - I have to push my first PR in the project out to prod before we start disabling exclusions. But shortly after that, I'll start that process.
I'm happy to be cc'd on any screaming and do the in-tree tier changes.
I think the remainder of this is on the parent bug (and the to-be-filed "screaming" bugs :)
Status: NEW → RESOLVED
Last Resolved: 5 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.