turn off regression email notifications for irrelevent statistics

RESOLVED WONTFIX

Status

Testing
Talos
RESOLVED WONTFIX
5 years ago
5 years ago

People

(Reporter: Jeff Hammel, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [sheriff-want])

(Reporter)

Description

5 years ago
It appears (though I could be wrong) that there are some talos
regression emails that aren't particularly important to send out/read:

Regression  Trace Malloc Allocs increase 1.23% on ...

Talos Improvement! Number of Constructors decrease 1.18% ...

Do we actually want these emails to be sent out?  They're probably
good statistics to keep around, but are they actually helpful? (or
helpful enough to alert dev-tree-management and [I think] developers
that trigger them with a push?)

Likewise, are there other talos perf emails that are worth turning
off?
Whiteboard: [sheriff-want]

Comment 1

5 years ago
Yes, these are helpful.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → WONTFIX
I am not sure these are helpful.  The general consensus is we ignore these emails and don't take action on them (for a variety of reasons).  Could we change the thresholds on these so we get less and take more action?
(In reply to Ehsan Akhgari [:ehsan] from comment #1)
> Yes, these are helpful.

And are contributing to the reason why myself & others don't have time to look at any dev.tree-management regressions.

It seems premature to close the bug without first having someone assess the full list of tests for which we get emails (rather than just the two examples given in comment 0).
Status: RESOLVED → REOPENED
Resolution: WONTFIX → ---
We'll send out email for any test not explicitly excluded. Right now the only thing we exclude is codesighs.

See also bug 579528 for improving the regression detection for tests that report static numbers (e.g. number num_ctors)

Comment 5

5 years ago
The tests mentioned in comment 0 are useful.  The fact that nobody looks at those emails is our failure, and not reason enough to stop sending the emails.
I have frequently commented or filed bugs about Number of Constructors regressions, and several have been fixed.  One is being actively worked on right now: https://bugzilla.mozilla.org/show_bug.cgi?id=774757

Updated

5 years ago
Status: REOPENED → RESOLVED
Last Resolved: 5 years ago5 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.