Closed
Bug 1375039
Opened 7 years ago
Closed 7 years ago
determine if AWSY on mozilla-beta is running properly
Categories
(Testing :: Talos, defect)
Testing
Talos
Tracking
(firefox56 verified, firefox57 verified)
VERIFIED
FIXED
mozilla57
People
(Reporter: jmaher, Assigned: jmaher)
References
Details
(Whiteboard: [PI:August])
Attachments
(2 files)
2.01 KB,
patch
|
bc
:
review+
|
Details | Diff | Splinter Review |
19.03 KB,
text/plain
|
Details |
I noticed on a AWSY regression on mozilla beta that we are: * running on a taskcluster build (we don't ship this) * running on opt (typically perf is done on pgo) * not run on pgo are these valid things to test with?
Assignee | ||
Updated•7 years ago
|
Whiteboard: [PI:June]
Comment 1•7 years ago
|
||
That's what we do with Nightly as well, right? The original work there was either directly uplifted to Aurora/Beta or is just riding the trains. What is the risk of a difference between the taskcluster build and a buildbot build showing significant differences that would justify the work of dealing with buildbot? I'll leave the pgo decision to you. Talos does pgo builds only? I would think both would be a requirement as pgo might hide stuff?
Assignee | ||
Comment 2•7 years ago
|
||
I think using the TC builds is probably safe- it has given us reliable results on trunk. ss for PGO, we have historically only run talos on pgo for mozilla-beta- I believe we do PGO for every commit in beta. There is probably no danger is running on opt and pgo, but that is duplicative IMO. :erahm, do you have thoughts on how we should be running AWSY on mozilla-beta?
Flags: needinfo?(erahm)
Comment 3•7 years ago
|
||
On beta running on PGO seems reasonable, that's effectively what we ship right? I don't think we need to do both as long as every beta commit has a PGO build. On the other hand if it's an undue burden I'm fine with just sticking with plain old opt builds.
Flags: needinfo?(erahm)
Assignee | ||
Comment 4•7 years ago
|
||
ok, we have PGO for every commit- lets start with that. I have this on my list for next week!
Assignee | ||
Updated•7 years ago
|
Whiteboard: [PI:June] → [PI:July]
Assignee | ||
Comment 5•7 years ago
|
||
ok, back into looking at this- we were running win7/10 "opt", and now we are running osx opt, win7 opt/pgo/devedition/nightly, win10 opt/pgo/nightly. Note, there is no linux AWSY running. I would propose only running on one configuration per platform- so we should see one run for linux, osx, win7, win10. When bug 1386776 is resolved, I will circle back here and try to get this cleaned up.
Depends on: 1386776
Whiteboard: [PI:July] → [PI:August]
Assignee | ||
Comment 6•7 years ago
|
||
osx/linux there is no awsy on mozilla-beta, but win7/10 for nightly and for devedition- we should fix this problem this week.
Assignee | ||
Comment 7•7 years ago
|
||
when bug 1387377 is resolved and uplifted to beta, lets check on what is running. in trying to figure out why we are not running on osx and linux64, I am at a loss. I looked in the config and there is nothing that restricts us from running on beta: http://searchfox.org/mozilla-central/source/taskcluster/ci/test/tests.yml#8 and why on mozilla-beta are we running win7/win10? :dustin, could you help me figure out why awsy is not running on mozilla-beta for linux and osx?
Flags: needinfo?(dustin)
Comment 8•7 years ago
|
||
The tasks are in the full-task-graph and not in the target tasks, so I'd look at the target task method. if platform in ( 'linux', 'linux64', 'macosx64', 'win32', 'win64', ): if task.attributes['build_type'] == 'opt' and \ task.attributes.get('unittest_suite') != 'talos': return False looks like we don't run opt on beta?
Flags: needinfo?(dustin)
Assignee | ||
Comment 9•7 years ago
|
||
mozilla-beta is confusion, we do run talos on linux64 opt (which is really a pgo build), and osx (which we do not have pgo builds for). My knowledge is based on what I see on treeherder: https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=02852b5a5b3c6b13029da12b9da5060f88ee2222&filter-searchStr=awsy on windows 7, this is run on nightly opt- but we have these nightly opt builds for linux32/64 as well. In summary I see similar builds for windows and linux (and mostly osx)- I see talos running on all platforms correctly, and the 'gtest' job runs correctly on nightly and debug builds; but the awsy job doesn't seem to run except for on windows 7/10. :dustin, do you have other thoughts on how to debug this?
Flags: needinfo?(dustin)
Assignee | ||
Updated•7 years ago
|
Assignee: nobody → jmaher
Comment 10•7 years ago
|
||
I'm sorry, I don't really understand what you're asking.
Flags: needinfo?(dustin)
Assignee | ||
Comment 11•7 years ago
|
||
Basically I cannot figure out why these tests are not running on mozilla-beta and I don't understand taskcluster well enough to debug it. If you are not the right person to help out, could you help me find someone else on the team to help figure this out?
Flags: needinfo?(dustin)
Comment 12•7 years ago
|
||
I think I answered it in comment 8. Looking at https://dxr.mozilla.org/mozilla-central/source/taskcluster/ci/test/test-platforms.yml?q=file%3Atest-platforms.yml&redirect_type=single I only see awsy set to run on */opt platforms. And in the mozilla-beta target task function, unittests (of which awsy is one) on opt platforms appear to be filtered out. I don't know why either of those things are the case, but those are the assumptions to question. I'm not sure who to ask, either -- I don't know who is in charge of AWSY. I suppose someone from releng may know the background on the beta config. And there's always 'hg blame'.
Flags: needinfo?(dustin)
Assignee | ||
Comment 13•7 years ago
|
||
this will land on nightly, then we need to uplift to beta.
Attachment #8902494 -
Flags: review?(bob)
Comment 14•7 years ago
|
||
Comment on attachment 8902494 [details] [diff] [review] run awsy tests on linux64/osx nightly test-sets looks ok. let's see how it does.
Attachment #8902494 -
Flags: review?(bob) → review+
Comment 15•7 years ago
|
||
diff of before and after output of ./mach taskgraph tasks --json
Comment 16•7 years ago
|
||
Pushed by jmaher@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/bcf177055b33 Enable AWSY for linux64/osx nightly builds. r=bc
Assignee | ||
Comment 17•7 years ago
|
||
:ryanvm, what is needed to uplift this to mozilla-beta? do I need to write a beta specific patch?
Flags: needinfo?(ryanvm)
Comment 18•7 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/bcf177055b33
Status: NEW → RESOLVED
Closed: 7 years ago
status-firefox57:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla57
Updated•7 years ago
|
Flags: needinfo?(ryanvm)
Comment 19•7 years ago
|
||
bugherder uplift |
https://hg.mozilla.org/releases/mozilla-beta/rev/88ba464220c4
status-firefox56:
--- → fixed
Comment 20•7 years ago
|
||
Looks like it worked. https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&filter-searchStr=slim
Assignee | ||
Comment 21•7 years ago
|
||
thanks everyone!
You need to log in
before you can comment on or make changes to this bug.
Description
•