Closed
Bug 1387377
Opened 7 years ago
Closed 7 years ago
Ensure we don't run microbenchmark/awsy performance numbers on devedition builds
Categories
(Testing :: Talos, enhancement)
Tracking
(firefox56 fixed, firefox57 fixed)
RESOLVED
FIXED
mozilla57
People
(Reporter: igoldan, Assigned: rwood)
Details
(Whiteboard: [PI:August])
Attachments
(1 file)
From bug 1385903, it looks like platform micro values from devedition builds are miswired.
Comment 1•7 years ago
|
||
I think we can probably stop running gtest on devedition, at the very least stop reporting numbers from gtest on devedition. I am not sure from inside the gtest how to determine if we are in devedition or not, so that would be something to look into.
Summary: Ensure we don't run performance numbers on devedition builds → Ensure we don't run microbenchmark performance numbers on devedition builds
Whiteboard: [PI:August]
Comment 2•7 years ago
|
||
I see we need to resolve this for awsy as well! Once we successfully land this on trunk, it will need to end up on mozilla-beta to be effective :)
Summary: Ensure we don't run microbenchmark performance numbers on devedition builds → Ensure we don't run microbenchmark/awsy performance numbers on devedition builds
Assignee | ||
Comment 3•7 years ago
|
||
I believe such a check could be added here, dependent upon being able to determine if we are running on devedition somehow: https://dxr.mozilla.org/mozilla-central/rev/a9d372645a32b8d23d44244f351639af9d73b96a/testing/gtest/rungtests.py#177
Assignee | ||
Comment 4•7 years ago
|
||
I'll see if I can get anywhere with this
Assignee: nobody → rwood
Status: NEW → ASSIGNED
Assignee | ||
Comment 5•7 years ago
|
||
Hmm, unfortunately the platform info received via mozinfo when running on Linux DevEdition opt, doesn't indicate DevEdition: [task 2017-08-24T15:53:34.930049Z] 15:53:34 INFO - gtest INFO | * RW * inside rungtests.py main [task 2017-08-24T15:53:34.930528Z] 15:53:34 INFO - gtest INFO | options: [task 2017-08-24T15:53:34.931267Z] 15:53:34 INFO - gtest INFO | {'symbols_path': 'https://queue.taskcluster.net/v1/task/P5KQew1WQrmskOqnsY4lFw/artifacts/public/build/target.crashreporter-symbols.zip', 'xre_path': '/home/worker/workspace/build/application/firefox', 'cwd': '/home/worker/workspace/build/tests/gtest', 'utility_path': 'tests/bin'} [task 2017-08-24T15:53:34.932226Z] 15:53:34 INFO - gtest INFO | mozinfo is: [task 2017-08-24T15:53:34.932467Z] 15:53:34 INFO - gtest INFO | {u'bin_suffix': u'', u'pgo': True, u'sync': True, u'buildapp': u'browser', u'crashreporter': True, u'addon_signing': True, u'require_signing': False, u'release_or_beta': False, u'appname': u'firefox', u'stylo': False, u'mozconfig': u'/home/worker/workspace/build/src/.mozconfig', u'topsrcdir': u'/home/worker/workspace/build/src', u'telemetry': False, 'os_version': StringVersion ('16.04'), 'version': 'Ubuntu 16.04', u'datareporting': True, u'buildtype_guess': u'pgo', 'bits': 32, 'has_sandbox': True, u'artifact': False, u'toolkit': u'gtk3', u'healthreport': True, u'updater': True, u'asan': False, u'platform_guess': u'linux', u'ccov': False, u'ubsan': False, u'tests_enabled': True, 'linux_distro': 'Ubuntu', u'official': True, u'tsan': False, u'nightly_build': True, u'debug': False, u'no_legacy_extensions': False, 'os': u'linux', 'processor': u'x86'} [task 2017-08-24T15:53:34.932520Z] 15:53:34 INFO - gtest INFO | Running gtest
Assignee | ||
Comment 6•7 years ago
|
||
There are DevEdition platforms specified in taskcluster test-platforms, i.e: http://searchfox.org/mozilla-central/rev/48ea452803907f2575d81021e8678634e8067fc2/taskcluster/ci/test/test-platforms.yml#59 The solution may be to create specific test-sets in test-sets.yml, just for devedition runs, and omit gtest from there.
Assignee | ||
Comment 7•7 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=fadb729d349bbda1d4deb30026280f28c1af6c6d
Assignee | ||
Comment 8•7 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=275225300f0f8b62412524d02d2e11385736767f
Comment hidden (mozreview-request) |
Comment 10•7 years ago
|
||
mozreview-review |
Comment on attachment 8900807 [details] Bug 1387377 - Don't run gtest or awsy on devedition builds; https://reviewboard.mozilla.org/r/172252/#review177518 I am concerned that we are adding test sets and the more we have the greater the chance of making mistakes, this is fine if there is not another option, could we edit the tests.yml file and ensure gtest doesn't run on devedition (like it is for win10 right now): http://searchfox.org/mozilla-central/source/taskcluster/ci/test/tests.yml#226
Attachment #8900807 -
Flags: review?(jmaher) → review-
Assignee | ||
Comment 11•7 years ago
|
||
(In reply to Joel Maher ( :jmaher) (UTC-5) from comment #10) > Comment on attachment 8900807 [details] > Bug 1387377 - Don't run gtest or awsy on devedition builds; > > https://reviewboard.mozilla.org/r/172252/#review177518 > > I am concerned that we are adding test sets and the more we have the greater > the chance of making mistakes, this is fine if there is not another option, > could we edit the tests.yml file and ensure gtest doesn't run on devedition > (like it is for win10 right now): > http://searchfox.org/mozilla-central/source/taskcluster/ci/test/tests.yml#226 Ah yes that's better, and I'll do that for awsy also, thanks!
Assignee | ||
Comment 12•7 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=7df0baad73c4d0912f414a0c23a8d234a9cf6893
Comment hidden (mozreview-request) |
Comment 14•7 years ago
|
||
mozreview-review |
Comment on attachment 8900807 [details] Bug 1387377 - Don't run gtest or awsy on devedition builds; https://reviewboard.mozilla.org/r/172252/#review177556 that looks good
Attachment #8900807 -
Flags: review?(jmaher) → review+
Assignee | ||
Comment 15•7 years ago
|
||
Thanks, I'm trying to use ./mach taskgraph to verify the config is correct but I am still seeing gtest on devedition, not sure I am using it correctly and the help is a bit confusing. :wcosta, how can I use ./mach taskgraph to verify that my config changes in tests.yml are correct? I tried ./mach taskgraph target before and after my patch, but the output is the same. I also tried './mach taskgraph tasks -p pointing to the latest parameters.yml on central, running that before and after my patch applied locally and still the output is the same... I must be doing it incorrectly.
Flags: needinfo?(wcosta)
Comment 16•7 years ago
|
||
(In reply to Robert Wood [:rwood] from comment #15) > Thanks, I'm trying to use ./mach taskgraph to verify the config is correct > but I am still seeing gtest on devedition, not sure I am using it correctly > and the help is a bit confusing. > > :wcosta, how can I use ./mach taskgraph to verify that my config changes in > tests.yml are correct? > > I tried ./mach taskgraph target before and after my patch, but the output is > the same. I also tried './mach taskgraph tasks -p pointing to the latest > parameters.yml on central, running that before and after my patch applied > locally and still the output is the same... I must be doing it incorrectly. Hi, sorry for the delay. Usually, what I do is do a try push, grab the mach command line from decision task log and run it locally from now on.
Flags: needinfo?(wcosta)
Assignee | ||
Comment 17•7 years ago
|
||
(In reply to Wander Lairson Costa [:wcosta] from comment #16) > > Hi, sorry for the delay. Usually, what I do is do a try push, grab the mach > command line from decision task log and run it locally from now on. Thanks :wcosta, I did that and I don't see any target tasks for 'awsy' or 'gtest' on devedition, so going to land this
Comment 18•7 years ago
|
||
Pushed by rwood@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/40bc1d4ce54b Don't run gtest or awsy on devedition builds; r=jmaher
Comment 19•7 years ago
|
||
please request uplift to mozilla-beta here.
Comment 20•7 years ago
|
||
This is test-only, no need for an approval request.
Whiteboard: [PI:August] → [PI:August][checkin-needed-beta]
Comment 21•7 years ago
|
||
bugherder uplift |
https://hg.mozilla.org/releases/mozilla-beta/rev/67363a144710
status-firefox56:
--- → fixed
Whiteboard: [PI:August][checkin-needed-beta] → [PI:August]
Comment 22•7 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/40bc1d4ce54b
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
status-firefox57:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla57
You need to log in
before you can comment on or make changes to this bug.
Description
•