Improve organization of Raptor power and resource test jobs on treeherder
Categories
(Testing :: Raptor, defect, P1)
Tracking
(firefox69 fixed)
| Tracking | Status | |
|---|---|---|
| firefox69 | --- | fixed |
People
(Reporter: rwood, Assigned: krubin)
Details
Attachments
(1 file)
We need to better organize the job symbols/groups currently being used for the Raptor android tests that are generating supplementary data like power, cpu, and memory info. Currently some tests are in the "Rap-P" treeherder group (i.e. for geckoview example) but for other platforms they're aligned with the other tests i.e. "Rap-fenix(idl)", "Rap-fennec(idl)".
The plan:
Instead of having a separate treeherder group symbol (i.e. Rap-P), align all the test jobs within the other jobs on the same platform (i.e. Rap, Rap-fenix, Rap-refbrow, Rap-fennec, etc) but use unique job symbols/names.
i.e. for any Raptor test that uses '--power-test', use the group name with the other test jobs, but append "-p" to the symbol. So for speedometer power running on geckoview the job would be: Rap(sp-p).
Do the same for tests that are collecting cpu (--cpu-test) and memory (--memory-test) but append "-r" for resource. So a Raptor speedometer test collecting cpu info and/or memory info on Fennec would be: Rap-fennec(sp-r).
Follow suit for the scenario tests that are collecting power info i.e. Rap-fenix(idl-p).
|
Reporter | |
Comment 1•5 years ago
|
||
Hi Ken,
This would be a good first bug for you to take, so you can get familiar with the Raptor test and taskcluster job configs. Once your build/dev env is all setup and running, could you take this one please? I can give you an overview of the Raptor and taskcluster configs first to help you get started.
Thanks!
|
||
Comment 2•5 years ago
|
||
(In reply to Robert Wood [:rwood] from comment #0)
Do the same for tests that are collecting cpu (--cpu-test) and memory (--memory-test) but append "-r" for resource. So a Raptor speedometer test collecting cpu info and/or memory info on Fennec would be: Rap-fennec(sp-r).
If we had a job gathering cpu and another of the same job gathering memory, how might that look? I don't anticipate this, but it's worth giving it some thought. Also, if we decide to gather cpu or memory on a test that's measuring power usage, would that job be suffixed with "-p-r"?
bc: as you were involved in the initial power test work, do you have some suggestions here?
|
||
Comment 3•5 years ago
|
||
Removing Rap-P as the group symbol and going with -p suffix for job symbols for power tests seems reasonable. We just need to make sure that we keep in mind that power tests require the 'batt' worker types of which we only have 2 p2 and 2 g5 up to a maximum of 8 total without adding additional hubs.
I'm not sure about the -r suffix however. If you had a cpu test and a separate memory test how would a single -r suffix distinguish them?
Would -c and -m be better or worse?
For combinations, I would suggest keep them sorted. Is it better to separate them? Is -p-r better than -pr ?
|
|
||
Comment 4•5 years ago
|
||
(In reply to Bob Clary [:bc:] from comment #3)
Removing Rap-P as the group symbol and going with -p suffix for job symbols for power tests seems reasonable. We just need to make sure that we keep in mind that power tests require the 'batt' worker types of which we only have 2 p2 and 2 g5 up to a maximum of 8 total without adding additional hubs.
Yes, this is good to keep in mind as we add power tests.
I'm not sure about the -r suffix however. If you had a cpu test and a separate memory test how would a single -r suffix distinguish them?
Would -c and -m be better or worse?
For combinations, I would suggest keep them sorted. Is it better to separate them? Is -p-r better than -pr ?
I like the idea of using a single character per resource measured, and keeping them sorted. It means we'll have somewhat cryptic job names, but at least all the information would be present. Some examples might be: 'sp-p', 'sp-cm', 'sp-cmp'. This also allows us to expand what we measure, for example 't' could represent temperature.
Pushed by rwood@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/7966936b3c86 Improve organization of Raptor power and resource test jobs on treeherder r=perftest-reviewers,AlexandruIonescu,rwood
|
||
Comment 7•5 years ago
|
||
| bugherder | ||
Description
•