Closed
Bug 1433075
Opened 7 years ago
Closed 7 years ago
Surprisingly, h1 and h2 started to run on Windows
Categories
(Testing :: Talos, enhancement)
Testing
Talos
Tracking
(firefox59 fixed, firefox60 fixed)
RESOLVED
FIXED
mozilla60
People
(Reporter: igoldan, Assigned: jmaher)
References
Details
Attachments
(1 file, 2 obsolete files)
2.87 KB,
patch
|
rwood
:
review+
|
Details | Diff | Splinter Review |
Due to frequent timeout errors, we turned off h1 and h2 Talos tests from autoland and inbound. All went fine until backout from [1] caused h1 to run on Windows again.
Investigate how the backout re-enabled them, then shut them down again.
[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1431533#c47
Reporter | ||
Comment 1•7 years ago
|
||
Assignee | ||
Comment 2•7 years ago
|
||
odd, the definition to limit on windows and linux to try only is here:
https://searchfox.org/mozilla-central/source/taskcluster/ci/test/talos.yml#296
and the backout doesn't have anything related to taskcluster.
I see some action tasks (after the decision task ran we scheduled other tasks):
https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=cc87ad81ff86109c7ea0187424fa9a8ed3b4af6c&filter-searchStr=decision
this is:
* run all talos tests
* run all missing tests
so I suspect that we need to fix the action tasks to respect what branches to run on.
Comment 3•7 years ago
|
||
It's a matter of definition, really -- an action can run a test if it was in the full task graph generated by the decision task. If that task is omitted due to target task selection (which is where run-on-branches is seen) or optimization, that doesn't affect the full-task-graph that the actions are based on. The whole idea of these actions is to override task-graph selection and optimization.
So, if we have tasks that should *never* run on windows, they shouldn't be in the full-task-graph at all, which means a change to the test-sets, rather than run-on-branches.
Assignee | ||
Comment 4•7 years ago
|
||
thanks Dustin- that helps clarify things. We often have tests we want to run on try only until they are stable- is there a better way to support this? I would like to explore other options before removing these tests from a test-set.yml definition, although in a patch to test it we could always add it back.
Comment 5•7 years ago
|
||
You could tag the affected tasks with some kind of not_part_of_all_talos_tests=True and not_part_of_missing_tests=True or something like that, and then modify the action implementation to skip tasks with that tag. It does seem like a bit of a hack, though. I assume these are tier-2 or tier-3? If so, won't they just not appear on treeherder anyway?
Assignee | ||
Comment 6•7 years ago
|
||
these are tier-1 and they consume too many cpu hours on our physical workers- so they cannot run by default. I think we should just remove them from the test_sets.yml, that will be easier to 'debug'.
Assignee | ||
Comment 7•7 years ago
|
||
:rwood, let me know if this looks like a reasonable solution.
Comment 8•7 years ago
|
||
Sounds good. I think the tools available here are tiers, run-on-projects, and test-sets, and it's simplest to use the tool that fits the needs most closely, rather than creating a new tool.
Assignee | ||
Comment 9•7 years ago
|
||
Attachment #8945496 -
Attachment is obsolete: true
Attachment #8945496 -
Flags: review?(rwood)
Attachment #8945523 -
Flags: review?(rwood)
Comment 10•7 years ago
|
||
Comment on attachment 8945523 [details] [diff] [review]
remove h1/h2 from test_sets.yml
Review of attachment 8945523 [details] [diff] [review]:
-----------------------------------------------------------------
::: taskcluster/ci/test/test-sets.yml
@@ -270,5 @@
> - talos-tp6
> - talos-tp6-stylo-threads
> - talos-speedometer
> - - talos-h1
> - - talos-h2
This removes it from mac, we want to leave it on mac (correct?) and remove it from windows and linux
Attachment #8945523 -
Flags: review?(rwood) → review-
Assignee | ||
Comment 11•7 years ago
|
||
I thought this left the mac ones alone, apologies if I overlooked that
Assignee | ||
Comment 12•7 years ago
|
||
this is for linux (including -qr) and windows- we only run on macosx now.
Attachment #8945523 -
Attachment is obsolete: true
Attachment #8945807 -
Flags: review?(rwood)
Comment 13•7 years ago
|
||
Comment on attachment 8945807 [details] [diff] [review]
remove h1/h2 from test_sets.yml
Review of attachment 8945807 [details] [diff] [review]:
-----------------------------------------------------------------
Thank you sir!
Attachment #8945807 -
Flags: review?(rwood) → review+
Comment 14•7 years ago
|
||
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/7ff05f538453
removed h1/h2 from test-sets so they are not accidentally run. r=rwood
Comment 15•7 years ago
|
||
bugherder |
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
status-firefox60:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla60
Comment hidden (Intermittent Failures Robot) |
Comment 17•7 years ago
|
||
bugherder uplift |
status-firefox59:
--- → fixed
You need to log in
before you can comment on or make changes to this bug.
Description
•