Closed
Bug 1364157
Opened 7 years ago
Closed 7 years ago
disable non-e10s talos tests on all platforms for Firefox 55+
Categories
(Release Engineering :: General, enhancement)
Release Engineering
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: jmaher, Assigned: jmaher)
References
Details
Attachments
(4 files)
14.39 KB,
patch
|
rwood
:
review+
|
Details | Diff | Splinter Review |
1.82 KB,
patch
|
kmoir
:
review+
|
Details | Diff | Splinter Review |
17.03 KB,
text/plain
|
Details | |
4.79 KB,
patch
|
chmanchester
:
review+
|
Details | Diff | Splinter Review |
given our load issues and the increase in new tests needing to run, we need to find machine time somewhere. Since in <3 months we will not be running non-e10s talos tests, lets disable them now so we can reduce feedback time for developers on try, not randomize developers with non-e10s only small performance regressions, and allow for a focus on what we want to ship.
Assignee | ||
Comment 1•7 years ago
|
||
odd, I did some initial work here and on osx it ran all the tests as well as duplicate e10s tests: https://treeherder.mozilla.org/#/jobs?repo=try&revision=e017d2c4d557dce3c4801a2c0ec4c9b86c081425 linux did the right thing.
Assignee | ||
Comment 2•7 years ago
|
||
what is more odd is that using the same patch I run the decision task locally and get the right set of jobs: ./mach taskgraph target -p ~/Downloads/parameters.yml 0:02.06 Loading kinds 0:02.38 Generating full task set 0:02.70 Generated 5 tasks for kind android-stuff 0:03.00 Generated 1 tasks for kind artifact-build 0:03.39 Generated 48 tasks for kind build 0:03.48 Generated 7 tasks for kind build-signing 0:03.87 Generated 6 tasks for kind docker-image 0:03.87 Generated 2 tasks for kind hazard 0:03.95 Generated 3 tasks for kind l10n 0:04.15 Generated 26 tasks for kind nightly-l10n 0:04.27 Generated 26 tasks for kind nightly-l10n-signing 0:04.32 Generated 1 tasks for kind push-apk-breakpoint 0:04.42 Generated 1 tasks for kind repackage 0:04.77 Generated 98 tasks for kind repackage-l10n 0:04.92 Generated 10 tasks for kind source-test 0:04.96 Generated 13 tasks for kind spidermonkey 0:04.98 Generated 8 tasks for kind static-analysis 0:05.06 Starting new HTTPS connection (1): s3.amazonaws.com 0:24.76 Generated 2958 tasks for kind test 0:25.46 Generated 14 tasks for kind toolchain 0:25.49 Generated 9 tasks for kind upload-symbols 0:25.49 Generated 1 tasks for kind valgrind 0:25.52 Generated 1 tasks for kind beetmover-repackage 0:26.48 Generated 385 tasks for kind beetmover-l10n 0:26.50 Generated 6 tasks for kind beetmover 0:27.25 Generated 392 tasks for kind balrog 0:27.97 Generated 392 tasks for kind checksums-signing 0:29.08 Generated 392 tasks for kind beetmover-checksums 0:29.14 Generated 1 tasks for kind push-apk 0:29.17 Generating full task graph 0:29.27 Full task graph contains 4806 tasks and 7841 dependencies 0:29.27 Generating target task set 0:29.28 Filter filter_servo pruned 0 tasks (4806 remain) 0:29.29 platform linux64 triggers ridealong builds linux64-l10n, sm-plain, sm-nonunified, sm-arm-sim, sm-arm64-sim, sm-compacting, sm-rootanalysis, sm-package, sm-tsan, sm-asan, sm-mozjs-sys, sm-msan, sm-fuzzing 0:29.33 Filter filter_target_tasks pruned 4753 tasks (53 remain) android-api-15-gradle-dependencies android-checkstyle android-findbugs android-lint android-test build-linux64/opt build-macosx64/opt l10n-linux64-l10n/opt source-test-mozharness source-test-mozlint-eslint source-test-mozlint-flake8 source-test-sphinx source-test-webidl-test source-test-wptlint-gecko spidermonkey-sm-asan/opt spidermonkey-sm-fuzzing/opt spidermonkey-sm-msan/opt spidermonkey-sm-package/opt spidermonkey-sm-plain/opt spidermonkey-sm-tsan/opt test-linux64/opt-talos-chrome-e10s test-linux64/opt-talos-dromaeojs-e10s test-linux64/opt-talos-g1-e10s test-linux64/opt-talos-g2-e10s test-linux64/opt-talos-g3-e10s test-linux64/opt-talos-g4-e10s test-linux64/opt-talos-other-e10s test-linux64/opt-talos-perf-reftest-e10s test-linux64/opt-talos-svgr-e10s test-linux64/opt-talos-tp5o-e10s test-macosx64/opt-talos-chrome-e10s test-macosx64/opt-talos-dromaeojs-e10s test-macosx64/opt-talos-g1-e10s test-macosx64/opt-talos-g2-e10s test-macosx64/opt-talos-g4-e10s test-macosx64/opt-talos-other-e10s test-macosx64/opt-talos-perf-reftest-e10s test-macosx64/opt-talos-svgr-e10s test-macosx64/opt-talos-tp5o-e10s toolchain-linux64-binutils/opt toolchain-linux64-cctools-port/opt toolchain-linux64-clang-tidy/opt toolchain-linux64-clang/opt toolchain-linux64-gcc/opt toolchain-linux64-hfsplus/opt toolchain-linux64-libdmg/opt toolchain-macosx64-cctools-port/opt toolchain-macosx64-clang-tidy/opt toolchain-macosx64-clang/opt toolchain-win32-clang-cl/opt toolchain-win32-clang-tidy/opt toolchain-win64-clang-cl/opt toolchain-win64-clang-tidy/opt and same for the decision task that ran: https://public-artifacts.taskcluster.net/aq9EDP9tS8-8TakGMqxkyA/0/public/target-tasks.json something seems wrong with taskcluster here. :wcosta, do you know why in taskcluster we would schedule talos non-e10s jobs and a second talos-e10s job when the decision task doesn't request it...specifically for osx- maybe there is something special on try for osx for bbb + native worker?
Flags: needinfo?(wcosta)
Comment 3•7 years ago
|
||
(In reply to Joel Maher ( :jmaher) from comment #2) > what is more odd is that using the same patch I run the decision task > locally and get the right set of jobs: > ./mach taskgraph target -p ~/Downloads/parameters.yml > 0:02.06 Loading kinds > 0:02.38 Generating full task set > 0:02.70 Generated 5 tasks for kind android-stuff > 0:03.00 Generated 1 tasks for kind artifact-build > 0:03.39 Generated 48 tasks for kind build > 0:03.48 Generated 7 tasks for kind build-signing > 0:03.87 Generated 6 tasks for kind docker-image > 0:03.87 Generated 2 tasks for kind hazard > 0:03.95 Generated 3 tasks for kind l10n > 0:04.15 Generated 26 tasks for kind nightly-l10n > 0:04.27 Generated 26 tasks for kind nightly-l10n-signing > 0:04.32 Generated 1 tasks for kind push-apk-breakpoint > 0:04.42 Generated 1 tasks for kind repackage > 0:04.77 Generated 98 tasks for kind repackage-l10n > 0:04.92 Generated 10 tasks for kind source-test > 0:04.96 Generated 13 tasks for kind spidermonkey > 0:04.98 Generated 8 tasks for kind static-analysis > 0:05.06 Starting new HTTPS connection (1): s3.amazonaws.com > 0:24.76 Generated 2958 tasks for kind test > 0:25.46 Generated 14 tasks for kind toolchain > 0:25.49 Generated 9 tasks for kind upload-symbols > 0:25.49 Generated 1 tasks for kind valgrind > 0:25.52 Generated 1 tasks for kind beetmover-repackage > 0:26.48 Generated 385 tasks for kind beetmover-l10n > 0:26.50 Generated 6 tasks for kind beetmover > 0:27.25 Generated 392 tasks for kind balrog > 0:27.97 Generated 392 tasks for kind checksums-signing > 0:29.08 Generated 392 tasks for kind beetmover-checksums > 0:29.14 Generated 1 tasks for kind push-apk > 0:29.17 Generating full task graph > 0:29.27 Full task graph contains 4806 tasks and 7841 dependencies > 0:29.27 Generating target task set > 0:29.28 Filter filter_servo pruned 0 tasks (4806 remain) > 0:29.29 platform linux64 triggers ridealong builds linux64-l10n, sm-plain, > sm-nonunified, sm-arm-sim, sm-arm64-sim, sm-compacting, sm-rootanalysis, > sm-package, sm-tsan, sm-asan, sm-mozjs-sys, sm-msan, sm-fuzzing > 0:29.33 Filter filter_target_tasks pruned 4753 tasks (53 remain) > android-api-15-gradle-dependencies > android-checkstyle > android-findbugs > android-lint > android-test > build-linux64/opt > build-macosx64/opt > l10n-linux64-l10n/opt > source-test-mozharness > source-test-mozlint-eslint > source-test-mozlint-flake8 > source-test-sphinx > source-test-webidl-test > source-test-wptlint-gecko > spidermonkey-sm-asan/opt > spidermonkey-sm-fuzzing/opt > spidermonkey-sm-msan/opt > spidermonkey-sm-package/opt > spidermonkey-sm-plain/opt > spidermonkey-sm-tsan/opt > test-linux64/opt-talos-chrome-e10s > test-linux64/opt-talos-dromaeojs-e10s > test-linux64/opt-talos-g1-e10s > test-linux64/opt-talos-g2-e10s > test-linux64/opt-talos-g3-e10s > test-linux64/opt-talos-g4-e10s > test-linux64/opt-talos-other-e10s > test-linux64/opt-talos-perf-reftest-e10s > test-linux64/opt-talos-svgr-e10s > test-linux64/opt-talos-tp5o-e10s > test-macosx64/opt-talos-chrome-e10s > test-macosx64/opt-talos-dromaeojs-e10s > test-macosx64/opt-talos-g1-e10s > test-macosx64/opt-talos-g2-e10s > test-macosx64/opt-talos-g4-e10s > test-macosx64/opt-talos-other-e10s > test-macosx64/opt-talos-perf-reftest-e10s > test-macosx64/opt-talos-svgr-e10s > test-macosx64/opt-talos-tp5o-e10s > toolchain-linux64-binutils/opt > toolchain-linux64-cctools-port/opt > toolchain-linux64-clang-tidy/opt > toolchain-linux64-clang/opt > toolchain-linux64-gcc/opt > toolchain-linux64-hfsplus/opt > toolchain-linux64-libdmg/opt > toolchain-macosx64-cctools-port/opt > toolchain-macosx64-clang-tidy/opt > toolchain-macosx64-clang/opt > toolchain-win32-clang-cl/opt > toolchain-win32-clang-tidy/opt > toolchain-win64-clang-cl/opt > toolchain-win64-clang-tidy/opt > > and same for the decision task that ran: > https://public-artifacts.taskcluster.net/aq9EDP9tS8-8TakGMqxkyA/0/public/ > target-tasks.json > > something seems wrong with taskcluster here. > > :wcosta, do you know why in taskcluster we would schedule talos non-e10s > jobs and a second talos-e10s job when the decision task doesn't request > it...specifically for osx- maybe there is something special on try for osx > for bbb + native worker? From task generation POV it is seems alright, 303 :kmoir
Flags: needinfo?(wcosta) → needinfo?(kmoir)
Comment 4•7 years ago
|
||
I looked at it, from task graph generation it seems fine. Not sure why several non e10s mac jobs were triggered and there were duplicates. The bbb should just trigger the jobs that were requested by tc.
Flags: needinfo?(kmoir)
Assignee | ||
Comment 5•7 years ago
|
||
it appears using --artifact for the osx builds triggers extra talos jobs: https://treeherder.mozilla.org/#/jobs?repo=try&author=jmaher@mozilla.com&exclusion_profile=false&filter-tier=1&filter-tier=2&filter-tier=3&fromchange=ad15a0450f0e4e4229a853ae9098b9ec66600613&tochange=5645ac93f0f9a59105ff7d86c154a6711b285948 Chris, do you have thoughts on why osx would schedule a full duplicate set of talos tests only when using --artifact? Maybe we somehow post two messages that we are done? Although I think that is odd since in my patch I disabled non-e10s talos and that still got scheduled once.
Flags: needinfo?(cmanchester)
Assignee | ||
Comment 6•7 years ago
|
||
after this we will need a buildbot patch for the windows jobs.
Updated•7 years ago
|
Attachment #8867118 -
Flags: review?(rwood) → review+
Assignee | ||
Comment 7•7 years ago
|
||
this turns off all jobs- we would need the taskcluster bits to land and merge before landing this.
Attachment #8867292 -
Flags: review?(kmoir)
Assignee | ||
Comment 8•7 years ago
|
||
for reference to the patch to disable non-e10s.
Updated•7 years ago
|
Attachment #8867292 -
Flags: review?(kmoir) → review+
Comment 9•7 years ago
|
||
The artifact build configs diverged at some point, it looks like we need to flip `enable_talos_sendchange` to false in testing/mozharness/configs/builds/releng_sub_mac_configs/64_artifact.py and/or remove these builders from buildbot so there's nothing to trigger there.
Flags: needinfo?(cmanchester)
Comment 10•7 years ago
|
||
Pushed by jmaher@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/c92e76bad29b disable non-e10s talos tests on all platforms for Firefox 55+; taskcluster patch. r=rwood
Assignee | ||
Comment 11•7 years ago
|
||
looking at the 64_artifacct.py files, I see that we have sendchange enabled for unittests and talos, but looking at the linux artifact config it is also enabled: https://dxr.mozilla.org/mozilla-central/search?q=path%3Atesting%2Fmozharness+enable_talos_sendchange%27%3A+T&redirect=false I wonder if we should set this to false for linux as well? I see linux32 needs to be false and same with Android. Chris- should I make linux32, linux64, mac, and android all false?
Flags: needinfo?(cmanchester)
Keywords: leave-open
Comment 12•7 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/c92e76bad29b
Assignee | ||
Comment 13•7 years ago
|
||
https://hg.mozilla.org/build/buildbot-configs/rev/f51dfdef83c3
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Comment 14•7 years ago
|
||
(In reply to Joel Maher ( :jmaher) from comment #11) > looking at the 64_artifacct.py files, I see that we have sendchange enabled > for unittests and talos, but looking at the linux artifact config it is also > enabled: > https://dxr.mozilla.org/mozilla-central/ > search?q=path%3Atesting%2Fmozharness+enable_talos_sendchange%27%3A+T&redirect > =false > > I wonder if we should set this to false for linux as well? I see linux32 > needs to be false and same with Android. > > Chris- should I make linux32, linux64, mac, and android all false? I'm not entirely sure, but it seems like those would all be scheduled through TC now, so that sounds right.
Flags: needinfo?(cmanchester)
Comment 15•7 years ago
|
||
In production: https://hg.mozilla.org/build/buildbot-configs/rev/f51dfdef83c3
Assignee | ||
Comment 16•7 years ago
|
||
Attachment #8868173 -
Flags: review?(cmanchester)
Comment 17•7 years ago
|
||
In production: https://hg.mozilla.org/build/buildbot-configs/rev/f51dfdef83c3
Comment 18•7 years ago
|
||
Comment on attachment 8868173 [details] [diff] [review] remove talos_sendchange from configurations where it is not needed Review of attachment 8868173 [details] [diff] [review]: ----------------------------------------------------------------- Looks ok to me. I think removing the entries from the configs would have the same effect.
Attachment #8868173 -
Flags: review?(cmanchester) → review+
Comment 19•7 years ago
|
||
Pushed by jmaher@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/3f87d6c1d378 do not run talos via sendchange unnecessarily. r=chmanchester
Comment 20•7 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/3f87d6c1d378
Comment 21•6 years ago
|
||
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
Updated•6 years ago
|
Component: General Automation → General
You need to log in
before you can comment on or make changes to this bug.
Description
•