Closed Bug 1342825 Opened 5 years ago Closed 5 years ago
Don't run mochitest browser screenshots on PGO builds
M(ss) jobs are running both on opt and PGO but the screenshots should be the same so this isn't necessary. Is there an easy way to disable running the job on PGO for both buildbot and TC? If there isn't an easy solution in TC and/or BB then we can do this in the browser.ini but there will still be wasted resources from starting the job and then doing nothing. Taskcluster config: https://dxr.mozilla.org/mozilla-central/rev/a08ec245fa24d573fc99e81210ecc09de734cdd3/taskcluster/ci/test/tests.yml#501 Buildbot config: https://hg.mozilla.org/build/buildbot-configs/annotate/d816f8ed797e49d73755ddcf16f616b153f30ffe/mozilla-tests/config.py#l407
Component: General Automation → Buildduty
QA Contact: catlee → bugspam.Callek
So looking at treeherder, these jobs seem to be limited to running on linux64 on m-c and try via taskcluster. They only seem to run on e10s on linux64 opt and linux64 nightly opt https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-job_type_symbol=ss Looking at taskcluster/ci/test/tests.yml they are defined like this mochitest-browser-screenshots: 531 description: "Mochitest Browser Screenshots" 532 suite: mochitest/browser-chrome-screenshots 533 treeherder-symbol: tc-M(ss) 534 loopback-video: true 535 run-on-projects: 536 by-test-platform: 537 linux64/opt: ['mozilla-central', 'try'] 538 default:  539 e10s: true 540 max-run-time: 3600 541 mozharness: 542 mochitest-flavor: browser 543 script: desktop_unittest.py 544 no-read-buildbot-config: true 545 config: 546 by-test-platform: 547 windows.*: 548 - unittests/win_taskcluster_unittest.py 549 macosx.*: 550 - remove_executables.py 551 - unittests/mac_unittest.py 552 linux.*: 553 - unittests/linux_unittest.py 554 - remove_executables.py 555 extra-options: 556 - --mochitest-suite=browser-chrome-screenshots 557 instance-size: legacy 558 allow-software-gl-layers: false I don't see them running on buildbot, perhaps the buildbot code that references them can be deleted as a cleanup. I don't see the jobs running on both pgo and opt builds now, perhaps this changed recently?
Sebastian From irc kmoir: it seems they are being scheduled... http://buildbot-master109.bb.releng.scl3.mozilla.com:8201/builders/Windows%207%2032-bit%20mozilla-central%20pgo%20test%20mochitest-browser-screenshots/builds/85 I don't know how to turn off pgo tests without turning off the associated opt tests. I looked it it briefly but you could investigate more. If these windows tests aren't showing up in treeherder, are they needed? Perhaps ask the original requestor on this bug.
I noticed these tests are indeed running in buildbot, in Tier 3. I will continue to investigate this issue. https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=f9362554866b327700c7f9b18050d7b7eb3d2b23&filter-searchStr=screenshots&filter-tier=1&filter-tier=2&filter-tier=3
If they are tier3 and hidden, is someone actually looking at the results? Will they need to be migrated to taskcluster eventually or can they be turned off?
> run on e10s on linux64 opt and linux64 nightly opt If that means that there are two sets of linux64 opt screenshots for the same revision can we disable the "linux64 nightly opt"? (In reply to Kim Moir [:kmoir] from comment #4) > If they are tier3 and hidden, is someone actually looking at the results? > Will they need to be migrated to taskcluster eventually or can they be > turned off? The job isn't eligible for higher tiers because it's not a pass/fail job, it simply takes screenshots and uploads them. The results are processed hourly and changes are sent to https://www.mozilla.org/en-US/about/forums/#dev-ui-alerts. They need to continue to run on all opt platforms regardless of BB or TC but we don't need screenshots from both of those for the same platform. See also https://developer.mozilla.org/en-US/docs/Mozilla/QA/Browser_screenshots
This would be an example of parameter.yml: https://public-artifacts.taskcluster.net/KsXhsnIXQkKleP6gVCJRgA/0/public/parameters.yml And these are the commands I tried, all refering to the same parameters.yml file: ./mach taskgraph tasks --json -p parameters.yml > old-tasks.json ./mach taskgraph tasks --json -p url/to/parameters.yml > old-tasks.json ./mach taskgraph tasks --json -p task-id=<task-id> > old-tasks.json
Dustin landed a patch yesterday which requires some new values to the parameters.yml such as include_nightly: false You can look at the the decision graph associated with a recent job on inbound and at the paramaters.yml https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&selectedJob=84020595 https://queue.taskcluster.net/v1/task/etgzxEVDSlWz_kwzSis_jQ/runs/0/artifacts/public/parameters.yml
Looks like it's working now. I don't receive that exception when I generate the graphs, after I did a pull and update from hg.
Diff resulting from applying the above patch. As it can be noticed, test-linux64-nightly/opt-mochitest-browser-screenshots-e10s is being removed.
I guess that patch is just for TC, I hope that BB will be handled too. I'm fine switching jobs from BB to TC but I believe screenshots don't work on some platforms over TC yet so those would have to remain in BB for now.
Comment on attachment 8847671 [details] [diff] [review] bug_1342825_linux64-nightly-opt.patch Like the other comment, I think there is a patch required for buildbot too
Attachment #8847671 - Flags: review?(kmoir) → review+
Did some investigation here along Sebastian and found the following: BB mochitest-browser-screenshots tests are enabled on both m-c and try here: https://dxr.mozilla.org/build-central/source/buildbot-configs/mozilla-tests/config.py#2667 However, the PGO variant only runs on m-c and it's disabled on try: https://dxr.mozilla.org/build-central/source/buildbot-configs/mozilla-tests/config.py#2402 https://dxr.mozilla.org/build-central/source/buildbot-configs/mozilla-tests/config.py#2444 It seems the 'pgo_strategy' parameter is then referenced in misc.py: https://dxr.mozilla.org/build-central/source/buildbotcustom/misc.py#809 meaning that we most likely cannot tell buildbot to do/not to do PGO for a particular test suite. Taking a look at a buildbot-master, there are only two such PGO tests currently running in BB: Windows 8 64-bit mozilla-central pgo test mochitest-browser-screenshots ScriptFactory Windows 7 32-bit mozilla-central pgo test mochitest-browser-screenshots ScriptFactory Not sure what we could do about those though given the above stats.
@Matthew: how would a convenient solution look in this case? Sebastian already provided a patch to remove "linux64-nightly opt" variant in TaskCluster (not landed yet). As for Buildbot, I don't see an easy way to remove the pgo variant of mochitest-browser-screenshots.
Pushed by email@example.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/5402e5e97e21 don't run mochitest browser screenshots on linux64-nightly opt, r=kmoir
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.