Closed Bug 1342825 Opened 5 years ago Closed 5 years ago

Don't run mochitest browser screenshots on PGO builds


(Infrastructure & Operations Graveyard :: CIDuty, task)

Not set


(firefox55 fixed)

Tracking Status
firefox55 --- fixed


(Reporter: MattN, Assigned: spacurar)




(2 files)

M(ss) jobs are running both on opt and PGO but the screenshots should be the same so this isn't necessary. Is there an easy way to disable running the job on PGO for both buildbot and TC? If there isn't an easy solution in TC and/or BB then we can do this in the browser.ini but there will still be wasted resources from starting the job and then doing nothing.

Taskcluster config:
Buildbot config:
Component: General Automation → Buildduty
QA Contact: catlee → bugspam.Callek
Assignee: nobody → spacurar
So looking at treeherder, these jobs seem to be limited to running on linux64 on m-c and try via taskcluster.  They only seem to run on e10s on linux64 opt and linux64 nightly opt

Looking at taskcluster/ci/test/tests.yml 

they are defined like this

 531     description: "Mochitest Browser Screenshots"
 532     suite: mochitest/browser-chrome-screenshots
 533     treeherder-symbol: tc-M(ss)
 534     loopback-video: true
 535     run-on-projects:
 536         by-test-platform:
 537             linux64/opt: ['mozilla-central', 'try']
 538             default: []
 539     e10s: true
 540     max-run-time: 3600
 541     mozharness:
 542         mochitest-flavor: browser
 543         script:
 544         no-read-buildbot-config: true
 545         config:
 546             by-test-platform:
 547                 windows.*:
 548                     - unittests/
 549                 macosx.*:
 550                     -
 551                     - unittests/
 552                 linux.*:
 553                     - unittests/
 554                     -
 555         extra-options:
 556             - --mochitest-suite=browser-chrome-screenshots
 557     instance-size: legacy
 558     allow-software-gl-layers: false

I don't see them running on buildbot, perhaps the buildbot code that references them can be deleted as a cleanup.

I don't see the jobs running on both pgo and opt builds now, perhaps this changed recently?

From irc

kmoir: it seems they are being scheduled...

I don't know how to turn off pgo tests without turning off the associated opt tests.  I looked it it briefly but you could investigate more.  If these windows tests aren't showing up in treeherder, are they needed? Perhaps ask the original requestor on this bug.
If they are tier3 and hidden, is someone actually looking at the results?  Will they need to be migrated to taskcluster eventually or can they be turned off?
> run on e10s on linux64 opt and linux64 nightly opt

If that means that there are two sets of linux64 opt screenshots for the same revision can we disable the "linux64 nightly opt"?

(In reply to Kim Moir [:kmoir] from comment #4)
> If they are tier3 and hidden, is someone actually looking at the results? 
> Will they need to be migrated to taskcluster eventually or can they be
> turned off?

The job isn't eligible for higher tiers because it's not a pass/fail job, it simply takes screenshots and uploads them. The results are processed hourly and changes are sent to They need to continue to run on all opt platforms regardless of BB or TC but we don't need screenshots from both of those for the same platform.

See also
This would be an example of parameter.yml:

And these are the commands I tried, all refering to the same parameters.yml file:

./mach taskgraph tasks --json -p parameters.yml > old-tasks.json
./mach taskgraph tasks --json -p url/to/parameters.yml > old-tasks.json
./mach taskgraph tasks --json -p task-id=<task-id> > old-tasks.json
Dustin landed a patch yesterday which requires some new values to the parameters.yml

such as

include_nightly: false

You can look at the the decision graph associated with a recent job on inbound and at the paramaters.yml
Looks like it's working now. I don't receive that exception when I generate the graphs, after I did a pull and update from hg.
Diff resulting from applying the above patch. As it can be noticed, test-linux64-nightly/opt-mochitest-browser-screenshots-e10s is being removed.
I guess that patch is just for TC, I hope that BB will be handled too.

I'm fine switching jobs from BB to TC but I believe screenshots don't work on some platforms over TC yet so those would have to remain in BB for now.
Comment on attachment 8847671 [details] [diff] [review]

Like the other comment, I think there is a patch required for buildbot too
Attachment #8847671 - Flags: review?(kmoir) → review+
Did some investigation here along Sebastian and found the following:

BB mochitest-browser-screenshots tests are enabled on both m-c and try here:

However, the PGO variant only runs on m-c and it's disabled on try:

It seems the 'pgo_strategy' parameter is then referenced in 
meaning that we most likely cannot tell buildbot to do/not to do PGO for a particular test suite.

Taking a look at a buildbot-master, there are only two such PGO tests currently running in BB:
Windows 8 64-bit mozilla-central pgo test mochitest-browser-screenshots ScriptFactory
Windows 7 32-bit mozilla-central pgo test mochitest-browser-screenshots ScriptFactory

Not sure what we could do about those though given the above stats.
@Matthew: how would a convenient solution look in this case?

Sebastian already provided a patch to remove "linux64-nightly opt" variant in TaskCluster (not landed yet).
As for Buildbot, I don't see an easy way to remove the pgo variant of mochitest-browser-screenshots.
Flags: needinfo?(MattN+bmo)
Pushed by
don't run mochitest browser screenshots on linux64-nightly opt, r=kmoir
Attachment #8847671 - Flags: checked-in+
Closed: 5 years ago
Resolution: --- → FIXED
Flags: needinfo?(MattN+bmo)
Product: Release Engineering → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.