Closed
Bug 1475196
Opened 6 years ago
Closed 6 years ago
we are running talos on win7/10 and win7/10-msvc builds
Categories
(Testing :: Talos, enhancement)
Testing
Talos
Tracking
(firefox63 fixed)
RESOLVED
FIXED
mozilla63
Tracking | Status | |
---|---|---|
firefox63 | --- | fixed |
People
(Reporter: jmaher, Assigned: ahal)
Details
Attachments
(3 files, 1 obsolete file)
we recently turned on msvc builds for windows, and it appears we are duplicating our perf tests on all the integration branches on windows- this is making our windows infrastructure often backlogged.
Reporter | ||
Comment 1•6 years ago
|
||
:dmajor- would you know more about what tests are expected to run and on which branches for the windows msvc? if we are doing this for a short time, it is ok, but longer than a month I would like to restrict what we run.
Flags: needinfo?(dmajor)
(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #0) > we recently turned on msvc builds for windows, and it appears we are > duplicating our perf tests on all the integration branches on windows- this > is making our windows infrastructure often backlogged. Am I understanding correctly, that the msvc builds run on m-c only, but somehow there is duplication of perf tests across _all_ integration branches? Can you point me to an example of such a job on, say, m-i?
Flags: needinfo?(dmajor) → needinfo?(jmaher)
Reporter | ||
Comment 3•6 years ago
|
||
inbound: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-searchStr=msvc%20talos autoland: https://treeherder.mozilla.org/#/jobs?repo=autoland&filter-searchStr=msvc%20talos mozilla-central: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-searchStr=msvc%20talos I thought we were going to run this and all tests on mozilla-central as this is tier-2; if you remove my treeherder filter word of talos, you can see all the unittests running as well.
Flags: needinfo?(jmaher)
The intent was certainly just to run on m-c. I removed the filter for "talos" leaving just "msvc" and I don't see the unittests on m-i though. Is there something special about talos that doesn't honor my run-on-projects attribute?
Reporter | ||
Comment 5•6 years ago
|
||
sorry, I had seen some test-verify jobs on mozilla-inbound and as I was switching branches I saw the full set on mozilla-central. I admit I jumped the gun with connecting the dots incorrectly. I think it is ok to have test-verify run on there, but it isn't necessary.
This is probably coming from lines such as this one https://searchfox.org/mozilla-central/source/taskcluster/ci/test/talos.yml#87 The direct fix would be to add a by-test-platform for msvc that restricts them to m-c. But perhaps it would be better to have these jobs use `built-projects`? (I don't feel super comfortable making the latter change myself though)
Reporter | ||
Comment 7•6 years ago
|
||
:ahal, could you help pick this up? I will be on pto and back on Tuesday.
Flags: needinfo?(ahal)
(In reply to Joel Maher ( :jmaher ) (UTC+2) (PTO: Back July 17) from comment #5) > I think it is ok to have > test-verify run on there, but it isn't necessary. We should probably disable test-verify too, because (I suspect) its presence is also triggering build tasks on every m-i/autoland push, which wasn't intended.
Assignee | ||
Comment 9•6 years ago
|
||
Just clarifying.. The goal here is to disable talos on Windows msvc platforms on autoland and inbound (but leave them on central and try)?
Assignee: nobody → ahal
Status: NEW → ASSIGNED
Flags: needinfo?(ahal)
Comment 10•6 years ago
|
||
(In reply to Andrew Halberstadt [:ahal] from comment #9) > Just clarifying.. > > The goal here is to disable talos on Windows msvc platforms on autoland and > inbound (but leave them on central and try)? Correct. And the same for test-verify jobs.
Assignee | ||
Comment 11•6 years ago
|
||
This refactoring will make it easier to set 'run-on-projects' keys across all talos tasks in one go (rather than needing to modify each task individually). The job-defaults are recursively (and naively) merged into the task definitions, so each task that changes 'run-on-projects' also needs to use the 'by-test-platform' key.
Assignee | ||
Comment 12•6 years ago
|
||
The windows7-32.* line is modified because it also matches the msvc tasks, and taskgraph errors out if there is any ambiguity over which value to use. Depends on D2136.
Assignee | ||
Comment 13•6 years ago
|
||
Depends on D2137.
Assignee | ||
Comment 14•6 years ago
|
||
This should work, but I want to diff the taskgraphs before/after first to make sure there are no unintended side effects. I'll likely flag them for review on Monday.
Comment 15•6 years ago
|
||
Great, thanks. I'd feel better if someone better versed in this stuff would be the reviewer, Dustin maybe?
Assignee | ||
Comment 16•6 years ago
|
||
Jmaher is probably the best reviewer. Since he'll be back tomorrow (and I'm working through some unintended side effects in my patch anyway), I'll just flag him.
Updated•6 years ago
|
Attachment #8992041 -
Attachment is obsolete: true
Assignee | ||
Comment 17•6 years ago
|
||
This is a diff of the full task graph (before and after). You'll notice that a lot of the "test-verify" and "code-coverage" tasks have been changed from 1 chunk to 0 chunks. Joel, this is happening from that "perfile_number_of_chunks" function you added awhile back: Is this chunk changing normal/expected? Should I be worried about these differences? It's a bit of a mystery to me.
Flags: needinfo?(jmaher)
Assignee | ||
Comment 18•6 years ago
|
||
Forgot to paste in the link: https://searchfox.org/mozilla-central/source/taskcluster/taskgraph/transforms/tests.py#823
Assignee | ||
Updated•6 years ago
|
Attachment #8992362 -
Attachment is patch: false
Comment 19•6 years ago
|
||
Comment on attachment 8992040 [details] Bug 1475196 - [ci] Move 'run-on-projects' to the job-defaults section in taskcluster/ci/talos.yml Joel Maher ( :jmaher ) (UTC+2) has approved the revision. https://phabricator.services.mozilla.com/D2136
Attachment #8992040 -
Flags: review+
Comment 20•6 years ago
|
||
Comment on attachment 8992042 [details] Bug 1475196 - [ci] Stop scheduling test-verify tasks on Windows msvc builds on inbound/autoland Joel Maher ( :jmaher ) (UTC+2) has approved the revision. https://phabricator.services.mozilla.com/D2138
Attachment #8992042 -
Flags: review+
Reporter | ||
Comment 21•6 years ago
|
||
I am not sure why you are seeing different chunks for verify and coverage jobs. Could it be that you are comparing apples to apples with the same base revision on mozilla-central? One thing I found while working on test-verify is that while ./mach locally, it would use the latest taskgraph/data from the tip of mozilla-central, it would work as expected one minute and running it a minute later when the tip changes on m-c I would get different results.
Flags: needinfo?(jmaher)
Assignee | ||
Comment 22•6 years ago
|
||
(In reply to Joel Maher ( :jmaher ) (UTC+2) from comment #21) > I am not sure why you are seeing different chunks for verify and coverage > jobs. Could it be that you are comparing apples to apples with the same > base revision on mozilla-central? One thing I found while working on > test-verify is that while ./mach locally, it would use the latest > taskgraph/data from the tip of mozilla-central, it would work as expected > one minute and running it a minute later when the tip changes on m-c I would > get different results. I'm comparing locally only (without my patch vs with my patch). But looking again, it looks like that 'perfile_number_of_chunks' function you wrote is using the changed files of the commit as an input. So I guess having the chunks change there isn't an error, I just happen to not be changing anything that would trigger those tasks. In other words, I think that change is expected and safe to ignore.
Comment 23•6 years ago
|
||
Pushed by ahalberstadt@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/e6291f882109 [ci] Move 'run-on-projects' to the job-defaults section in taskcluster/ci/talos.yml r=jmaher
Comment 24•6 years ago
|
||
Pushed by ahalberstadt@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/1d510f299cf3 [ci] Stop scheduling test-verify tasks on Windows msvc builds on inbound/autoland r=jmaher
Assignee | ||
Comment 25•6 years ago
|
||
Hm, running `arc diff` didn't update the commit message.. that's not good.
Comment 26•6 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/e6291f882109 https://hg.mozilla.org/mozilla-central/rev/1d510f299cf3
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
status-firefox63:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla63
Comment 27•6 years ago
|
||
Just to make sure: is it expected we no longer schedule MSVC builds on inbound/autoland? I was just wondering why they disappeared and this bug seems to suggest we only wanted to change things for tests/Talos.
Comment 28•6 years ago
|
||
Yes, the intent was to run builds and tests on m-c only. The builds weren't explicitly set to run on inbound, but the tests triggered the builds as a dependency, so removing the tests got us back to the intended situation.
You need to log in
before you can comment on or make changes to this bug.
Description
•