perfherder_data.json is being overwritten by android OS baseline data in power tests
Categories
(Testing :: Raptor, defect, P2)
Tracking
(firefox70 fixed)
| Tracking | Status | |
|---|---|---|
| firefox70 | --- | fixed |
People
(Reporter: sparky, Assigned: sparky)
References
Details
Attachments
(1 file)
Currently, when running android power tests (and soon any resource tests) the perfherder_data.json file that is output as an artifact overwrites previous saved JSONs. In power tests, the OS baseline data is overwriting the test power results.
This could be solved by outputting multiple perfherder_data.json files (with different names). We could also solve this issue by solving bug 1550587, although we will still need to output multiple artifacts once we have multiple resources running in the same task.
|
Assignee | |
Comment 1•5 years ago
|
||
This patch fixes three things at once:
(1) Consolidate supporting raptor measurements into one PERFHERDER_DATA output per type. For example, with this change, all power data will be grouped into one PERFHERDER_DATA output.
(2) Output perfherder-data-<DATA_TYPE>.json for each supporting measurement instead of overwriting perfherder-data.json which contains the regular raptor test results.
(3) Take an average of the supporting measurements when particular unit's are specified. In this case, the '%' unit makes us take the average instead of the sum of the measurements.
|
|
Assignee | |
Comment 2•5 years ago
|
||
Try run with the patch (linting errors have since been fixed): https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=8db5c16bef07bdf784af7e598ec56856c12eec37&selectedJob=259960215
|
|
Assignee | |
Updated•5 years ago
|
|
||
Updated•5 years ago
|
|
|
Assignee | |
Comment 3•5 years ago
|
||
Subtest names in power tests will be changed by this patch to 'power-<MEASURE>' where MEASURE is cpu, wifi, screen, or proportional instead of '<RAPTOR_TEST>-power-<MEASURE>'.
|
|
||
Comment 4•5 years ago
|
||
(In reply to Greg Mierzwinski [:sparky] from comment #3)
Subtest names in power tests will be changed by this patch to 'power-<MEASURE>' where MEASURE is cpu, wifi, screen, or proportional instead of '<RAPTOR_TEST>-power-<MEASURE>'.
We'll need to follow up with an update to health.graphics to plot the data associated with the new test names.
Pushed by gmierz2@outlook.com: https://hg.mozilla.org/integration/autoland/rev/50687ba1d4c7 Consolidate supporting raptor measurements into one perfherder output and artifact per type. r=perftest-reviewers,rwood
|
||
Comment 6•5 years ago
|
||
| bugherder | ||
Description
•