Closed
Bug 1481728
Opened 3 years ago
Closed 5 months ago
Improvement alerts aren't generated automatically
Categories
(Tree Management :: Perfherder, defect, P3)
Tree Management
Perfherder
Tracking
(Not tracked)
RESOLVED
INVALID
People
(Reporter: igoldan, Unassigned)
References
Details
I noticed this happens on the build_metrics framework, for at least the compiler warnings and installer size tests. Recent examples, were I had to manually create the alerts: https://bugzilla.mozilla.org/show_bug.cgi?id=1474860#c10 https://bugzilla.mozilla.org/show_bug.cgi?id=1476194#c8 https://bugzilla.mozilla.org/show_bug.cgi?id=1476194#c9 https://bugzilla.mozilla.org/show_bug.cgi?id=1476194#c10
Comment 1•3 years ago
|
||
we are getting improvement alerts for the other frameworks still, so this seems like an edge case. Would we expect to see improvement alerts for <2%, that is our default threshold for an alert unless otherwise specified. The 13% improvement should have been alerted on. Maybe there are messages in the log file? Were data points backfilled here and we got the notification >2 weeks later because we were backfilling 2 weeks later?
Comment 2•2 years ago
|
||
This is marked as a P1 but isn't assigned to anyone. Can this be marked down to P2 or P3?
Flags: needinfo?(igoldan)
Reporter | ||
Comment 3•2 years ago
|
||
(In reply to Sarah Clements [:sclements] from comment #2)
This is marked as a P1 but isn't assigned to anyone. Can this be marked down to P2 or P3?
Yes, I'd mark it as P3.
Flags: needinfo?(igoldan)
Updated•2 years ago
|
Priority: P1 → P3
Reporter | ||
Updated•1 year ago
|
Type: enhancement → defect
Reporter | ||
Comment 4•5 months ago
|
||
Reason for this issue is that these tests have an absolute threshold, not a percentage based one.
Because of this, there's a visual confusion that makes us think an alert should have been generated.
Status: NEW → RESOLVED
Closed: 5 months ago
Resolution: --- → INVALID
You need to log in
before you can comment on or make changes to this bug.
Description
•