Closed
Bug 1265861
Opened 8 years ago
Closed 8 years ago
Reduce threshold for perfherder alerts on APK size changes
Categories
(Firefox Build System :: General, defect)
Firefox Build System
General
Tracking
(firefox48 fixed)
RESOLVED
FIXED
mozilla48
Tracking | Status | |
---|---|---|
firefox48 | --- | fixed |
People
(Reporter: Margaret, Assigned: gbrown)
Details
Attachments
(1 file)
1.16 KB,
patch
|
wlach
:
review+
jmaher
:
feedback+
|
Details | Diff | Splinter Review |
Right now perfherder is configured to alert on performance regressions/improvements of 2% (I think) or more, but this creates a threshold that's too high for APK size changes. Chatting with wlach, it sounds like we can change this to alert on smaller changes. gbrown, can you help make this happen?
Flags: needinfo?(gbrown)
Comment 1•8 years ago
|
||
The magic incantation to make this work is to add alertThreshold to the PERFHERDER_DATA string. If you want something like 0.25%, you would do something like: PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"subtests": [{"name": "libxul.so", "value": 25478543}, {"name": "classes.dex", "value": 5193188}, {"name": "omni.ja", "value": 6037744}], "name": "installer size", "value": 38296509, alertThreshold: 0.25}, ]} Everything is documented here: https://github.com/mozilla/treeherder/blob/master/schemas/performance-artifact.json
Assignee | ||
Comment 2•8 years ago
|
||
The idea here is that the Fennec APK size is about 40 MB, 0.25% = 100K, which seems large enough to be actionable. This change affects all platforms. A 0.25% change may not be as important to desktop...but then again, maybe it should be? Let's try this and dial it back -- perhaps for different platforms -- if it causes too many alerts, or non-actionable alerts. :jmaher - Thought I'd check in with you since you may be triaging these -- any objections?
Assignee: nobody → gbrown
Flags: needinfo?(gbrown)
Attachment #8743013 -
Flags: review?(wlachance)
Attachment #8743013 -
Flags: feedback?(jmaher)
Comment 3•8 years ago
|
||
Comment on attachment 8743013 [details] [diff] [review] use alert threshold of 0.25% for "installer size" Review of attachment 8743013 [details] [diff] [review]: ----------------------------------------------------------------- great stuff!
Attachment #8743013 -
Flags: feedback?(jmaher) → feedback+
Comment 4•8 years ago
|
||
Comment on attachment 8743013 [details] [diff] [review] use alert threshold of 0.25% for "installer size" Review of attachment 8743013 [details] [diff] [review]: ----------------------------------------------------------------- LGTM!
Attachment #8743013 -
Flags: review?(wlachance) → review+
Comment 6•8 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/43534e7fd750
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla48
Updated•6 years ago
|
Product: Core → Firefox Build System
You need to log in
before you can comment on or make changes to this bug.
Description
•