Speedometer test wrong score after name change
Categories
(Testing :: Raptor, defect, P1)
Tracking
(firefox77 fixed)
Tracking | Status | |
---|---|---|
firefox77 | --- | fixed |
People
(Reporter: alexandru.irimovici, Assigned: marauder)
Details
(Keywords: perf-alert)
Attachments
(1 file)
There are some big differences in score speedometer tests after the name changes:
For example:
Old score (from 3rd of april): 25.93
job: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&selectedJob=295887733&group_state=expanded&revision=1d96f4423d1622370f0dac09d475f8ea76b144a6
json file: https://firefoxci.taskcluster-artifacts.net/OqWc-IBkQLKwaNhKMLvBTA/0/public/test_info/perfherder-data.json
New score (from 8th of april): 118.45
https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&selectedJob=296750317&group_state=expanded&revision=6663d3dc883b6ad0d0dfa9346f9ceabf2b2c7967
json file: https://firefoxci.taskcluster-artifacts.net/HAWyzT_jQ72P3qPjFIsGiQ/0/public/test_info/perfherder-data.json
The problem can be fixed here:
https://dxr.mozilla.org/mozilla-central/source/testing/raptor/raptor/output.py#303
Updated•4 years ago
|
Assignee | ||
Updated•4 years ago
|
Assignee | ||
Comment 1•4 years ago
|
||
Pushed by mraiciof@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/b3e61f709ef5 Speedometer test wrong score after name change r=perftest-reviewers,alexandru.irimovici
Comment 3•4 years ago
|
||
bugherder |
Comment 4•4 years ago
|
||
== Change summary for alert #25595 (as of Sat, 11 Apr 2020 14:57:16 GMT) ==
Improvements:
194% speedometer windows7-32-shippable opt 32.72 -> 96.19
190% speedometer linux64-shippable opt 32.80 -> 94.95
189% speedometer linux64-shippable-qr opt 32.84 -> 94.97
173% speedometer windows10-64-shippable-qr opt 33.73 -> 92.21
172% speedometer windows10-64-shippable opt 33.90 -> 92.19
105% speedometer linux64-qr opt 38.85 -> 79.66
105% speedometer linux64 opt 38.82 -> 79.55
99% speedometer windows7-32 opt 39.65 -> 78.78
85% speedometer windows10-64-qr opt 41.08 -> 76.02
27% speedometer macosx1014-64-shippable opt 52.01 -> 65.89
For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=25595
Comment 5•4 years ago
|
||
That perf alert is valid, the values weren't being calculated correctly before this patch (now they match raptor-webext numbers).
Description
•