Closed
Bug 1013298
Opened 10 years ago
Closed 10 years ago
tp4m has only two samples - median is not a good statistic
Categories
(Datazilla Graveyard :: Metrics, defect)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: ekyle, Unassigned)
References
Details
dzAlerts uses median value from each test. tp4m has only two samples per test, which results in the mean == median. This is a problem because first page load time is significantly different than the second (in most cases it seems). Either parametrize dzAlerts so we can declare the statistic to run regression analysis on, or split each tp4m test into two sub-tests with one run each. The latter will benefit us by comparing first load times and subsequent load times. The former may be of benefit to other tests where median is not appropriate.
Reporter | ||
Comment 1•10 years ago
|
||
Now that we set per-test parameters, I have changed tp4m to use last (last replicate) instead of median.
> "tp4m": {
> "units": "ms",
> "select": {
> "value": {
> "name": "value",
> "value": "result.stats.last"
> }
> }
> }
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•