984894 - Do not alert <30ms cold_load_time

Status: RESOLVED FIXED

(Datazilla Graveyard :: Metrics, defect)

Description

[Kyle Lahnakoski [:ekyle]]

In general, all potential alerts should have a parameter to filter by some absolute difference, along with sign, and percentage.  This will prevent minor regressions from occupying our attention.

Comment 1

[Kyle Lahnakoski [:ekyle]]

Each of the tests (cold_load_time, fps, etc) can have a minimum difference set (absolute or percentage, but not both).  fps, cold_load_time, communications/contacts_memory, startup_time are currently set to 1%.  Sign is handled by marking each test as higher/lower is better.  Finally, the measurement units can also be included.

Example:

    "cold_load_time": {
        "units": "ms",
        "better": "lower",
        "min_regression": "1%"
    }

Comment 2

[Kyle Lahnakoski [:ekyle]]

Confirmed fixed