Closed Bug 1034243 Opened 10 years ago Closed 8 years ago

[UI] Alert Manager should generate a report for uplift statistics

Categories

(Testing :: General, defect)

Product:
Testing
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: jmaher, Unassigned, Mentored)

References

Details

(Whiteboard: [good next bug][lang=js]) 

Alert Manager provides a lot of value, but we have a lot of data that can be extrapolated from it to track how useful the tool is, the tests we run are, and how healthy Mozilla is an an organization when it comes to managing performance alerts.

Here is an example of some hand crafted metrics:
http://elvis314.wordpress.com/2014/05/05/performance-alerts-by-the-numbers/

I would like to see when we uplift to aurora:
* how many regressions, improvements do we have
** compare this to previous uplifts
* break regressions, improvements down by platform

To determine which is an uplift, we should mark them in the database, but we could just look for what is on mozilla-aurora and has >20 alerts for a given revision to get this going.

these could live on the same report.html page as bug 1034242 would use for bug level reporting.


To see what alert manager does and how to get started, here is a good place to start:
https://wiki.mozilla.org/Auto-tools/Projects/AlertManager

Please ask questions in this bug or on irc!
Whiteboard: [next_bug][lang=javascript] → [good next bug][lang=javascript]
Whiteboard: [good next bug][lang=javascript] → [good next bug][lang=js]
This is still a valid issue with the latest alert manager.
alert manager is has been replaced by perfherder
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.