Closed Bug 1433075 Opened 7 years ago Closed 7 years ago

Surprisingly, h1 and h2 started to run on Windows

Categories

(Testing :: Talos, enhancement)

enhancement
Not set
normal

Tracking

(firefox59 fixed, firefox60 fixed)

RESOLVED FIXED
mozilla60
Tracking Status
firefox59 --- fixed
firefox60 --- fixed

People

(Reporter: igoldan, Assigned: jmaher)

References

Details

Attachments

(1 file, 2 obsolete files)

Due to frequent timeout errors, we turned off h1 and h2 Talos tests from autoland and inbound. All went fine until backout from [1] caused h1 to run on Windows again. Investigate how the backout re-enabled them, then shut them down again. [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1431533#c47
odd, the definition to limit on windows and linux to try only is here: https://searchfox.org/mozilla-central/source/taskcluster/ci/test/talos.yml#296 and the backout doesn't have anything related to taskcluster. I see some action tasks (after the decision task ran we scheduled other tasks): https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=cc87ad81ff86109c7ea0187424fa9a8ed3b4af6c&filter-searchStr=decision this is: * run all talos tests * run all missing tests so I suspect that we need to fix the action tasks to respect what branches to run on.
It's a matter of definition, really -- an action can run a test if it was in the full task graph generated by the decision task. If that task is omitted due to target task selection (which is where run-on-branches is seen) or optimization, that doesn't affect the full-task-graph that the actions are based on. The whole idea of these actions is to override task-graph selection and optimization. So, if we have tasks that should *never* run on windows, they shouldn't be in the full-task-graph at all, which means a change to the test-sets, rather than run-on-branches.
thanks Dustin- that helps clarify things. We often have tests we want to run on try only until they are stable- is there a better way to support this? I would like to explore other options before removing these tests from a test-set.yml definition, although in a patch to test it we could always add it back.
You could tag the affected tasks with some kind of not_part_of_all_talos_tests=True and not_part_of_missing_tests=True or something like that, and then modify the action implementation to skip tasks with that tag. It does seem like a bit of a hack, though. I assume these are tier-2 or tier-3? If so, won't they just not appear on treeherder anyway?
these are tier-1 and they consume too many cpu hours on our physical workers- so they cannot run by default. I think we should just remove them from the test_sets.yml, that will be easier to 'debug'.
Attached patch remove h1/h2 from test_sets.yml (obsolete) — Splinter Review
:rwood, let me know if this looks like a reasonable solution.
Assignee: nobody → jmaher
Status: NEW → ASSIGNED
Attachment #8945496 - Flags: review?(rwood)
Sounds good. I think the tools available here are tiers, run-on-projects, and test-sets, and it's simplest to use the tool that fits the needs most closely, rather than creating a new tool.
Attached patch remove h1/h2 from test_sets.yml (obsolete) — Splinter Review
Attachment #8945496 - Attachment is obsolete: true
Attachment #8945496 - Flags: review?(rwood)
Attachment #8945523 - Flags: review?(rwood)
Comment on attachment 8945523 [details] [diff] [review] remove h1/h2 from test_sets.yml Review of attachment 8945523 [details] [diff] [review]: ----------------------------------------------------------------- ::: taskcluster/ci/test/test-sets.yml @@ -270,5 @@ > - talos-tp6 > - talos-tp6-stylo-threads > - talos-speedometer > - - talos-h1 > - - talos-h2 This removes it from mac, we want to leave it on mac (correct?) and remove it from windows and linux
Attachment #8945523 - Flags: review?(rwood) → review-
I thought this left the mac ones alone, apologies if I overlooked that
this is for linux (including -qr) and windows- we only run on macosx now.
Attachment #8945523 - Attachment is obsolete: true
Attachment #8945807 - Flags: review?(rwood)
Comment on attachment 8945807 [details] [diff] [review] remove h1/h2 from test_sets.yml Review of attachment 8945807 [details] [diff] [review]: ----------------------------------------------------------------- Thank you sir!
Attachment #8945807 - Flags: review?(rwood) → review+
Pushed by jmaher@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/7ff05f538453 removed h1/h2 from test-sets so they are not accidentally run. r=rwood
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla60
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: