Since enabling PGO on TC build times have become bimodal

NEW
Unassigned

Status

2 years ago
6 months ago

People

(Reporter: automatedtester, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Comment 1

2 years ago
IMO we should change the Perfherder metrics to take into account buildbot vs TC so the numbers are part of separate metrics. Pretty sure this would fall under the domain of mozharness since it is what's outputting the TinderboxPrint messages that get turned into Perfherder data.

FWIW, we have similar problems elsewhere. e.g. "clobber" builds have completely different times from non-clobber builds. For any given job, you end up with N variations. e.g. "clobber+treeherder" versus "non-clobber+buildbot." Trying to shoehorn everything into a string gets difficult because order matters. So what you really want is the ability to associate N tags/annotations against a metric so you group/filter/distinguish data accordingly. That would certainly require changes on Perfherder to support, however.
(In reply to Gregory Szorc [:gps] from comment #1)
> IMO we should change the Perfherder metrics to take into account buildbot vs
> TC so the numbers are part of separate metrics. Pretty sure this would fall
> under the domain of mozharness since it is what's outputting the
> TinderboxPrint messages that get turned into Perfherder data.
> 
> FWIW, we have similar problems elsewhere. e.g. "clobber" builds have
> completely different times from non-clobber builds. For any given job, you
> end up with N variations. e.g. "clobber+treeherder" versus
> "non-clobber+buildbot." Trying to shoehorn everything into a string gets
> difficult because order matters. So what you really want is the ability to
> associate N tags/annotations against a metric so you
> group/filter/distinguish data accordingly. That would certainly require
> changes on Perfherder to support, however.

You can specify extraOptions as part of a perfherder series as of bug 1261922, which we could use to specify things like this:

https://github.com/mozilla/treeherder/blob/master/schemas/performance-artifact.json#L66

:rwood recently made talos jobs generate 'e10s' options. Search for PERFHERDER_DATA in e.g. http://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-inbound-win32/1464444695/mozilla-inbound_win7_ix_test-g2-e10s-bm111-tests1-windows-build109.txt.gz

Updated

6 months ago
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.