Closed Bug 863061 Opened 9 years ago Closed 9 years ago

No alerts for PGO libxul linker memory regressions under 2%

Categories

(Release Engineering :: General, defect)

x86
Windows 7
defect
Not set
critical

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: mbrubeck, Assigned: mbrubeck)

References

Details

(Whiteboard: [regression-detection])

Attachments

(1 file)

Bug 710840 added libxul link memory data to the graph server.  However, we have not received email alerts for the regressions since then because bug 822249 suppresses email for regressions of less than 2%.  There have been at least 6 since those changes were deployed, but all of theme were between 0.45% and 1.05% and so no one was notified.

Some suggestions:

1) For specific whitelisted datasets where we care about small changes (like libxul memory), ignore the threshold and always send emails.

2) For changes below the 2% threshold, do not send email to individual authors but do send it to the dev-tree-management list.

3) For changes below the 2% threshold, send email only if the t-test score is above an extra-high threshold like 20 or 30, instead of the regular threshold of 9.

4) Get rid of the 2% threshold and try reducing false alarms in other ways, e.g. bug 627860.
The next doomsday is close.  Bumping this to critical.
Severity: normal → critical
I'd say we should go with #1.
Note also that Datazilla means analyze_talos.py will be EOLed soon, so not worth coming up with an elaborate solution for this.
Component: Release Engineering → Release Engineering: Developer Tools
QA Contact: hwine
This implements option 1 from comment 0.  (Note that this will require a change to analysis.cfg during deployment.)
Assignee: nobody → mbrubeck
Status: NEW → ASSIGNED
Attachment #739160 - Flags: review?(catlee)
Attachment #739160 - Flags: review?(catlee) → review+
Comment on attachment 739160 [details] [diff] [review]
ignore percentage for "LibXUL Memory during link" regressions

http://hg.mozilla.org/graphs/rev/640b38d873e0

Leaving open until deployed (including analysis.cfg edit).
Attachment #739160 - Flags: checked-in+
Deployed, with analysis.cfg updated.
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/d1a8bcf8a4d832b99764214c7da13b46b1186a84
Bug 863061 - Ignore percentage_threshold for libxul linker memory [r=catlee]
Component: Tools → General
You need to log in before you can comment on or make changes to this bug.