Bugzilla

(In reply to Mike Hommey [:glandium] from comment #3)
> The graph shows a regression in May that doesn't appear to have an associated regression. Similarly, I randomly picked another job (Linux64 debug on autoland), checked the num_static_constructors graph for it, and there are plenty of visible regressions that are not marked with alerts.
> https://treeherder.mozilla.org/perf.html#/graphs?highlightAlerts=1&series=autoland,1468482,1,2&timerange=1209600

This series alerts when its baseline changes its **absolute** *(not percentage)* value by 3. Those baseline changes increased just by 1. Thus, they didn't alert.
Problem here is the zoomed in graph is confusing: those regressions seem serious, when in fact they're hardly noticeable.

(In reply to Mike Hommey [:glandium] from comment #3)
> The graph shows a regression in May that doesn't appear to have an associated regression. Similarly, I randomly picked another job (Linux64 debug on autoland), checked the num_static_constructors graph for it, and there are plenty of visible regressions that are not marked with alerts.
> https://treeherder.mozilla.org/perf.html#/graphs?highlightAlerts=1&series=autoland,1468482,1,2&timerange=1209600

This series alerts when its baseline changes its **absolute** *(not percentage)* value by 3. Those baseline changes increased just by 1. Thus, they didn't alert.
Problem here is the zoomed in graph is confusing: those regressions seem serious, when in fact they're hardly noticeable. But I guess this is the risk when a test like **compiler_metrics num_static_constructors** applies for absolute changes.