1553335 - Add and run Android-OS baseline power-usage measurement test, in CI

Status: RESOLVED FIXED

(Testing :: Raptor, task, P1)

Description

[Stephen Donner [:stephend] Not actively reading bugmail]

After synching up with both Dave and Greg, we conferred we still need and want an OS-baseline-measurement test, which will run periodically in CI.

Greg can provide additional, fine-grained detail, since he's graciously looking into this.

Comment 1

[Greg Mierzwinski [:sparky]]

The plan is to collect power usage data for 1 minute before raptor starts the tests (and before it starts the browser). The original plan was to subtract the OS baseline data from the test data but this won't work since the test data is already app-specific. Instead, we will need to submit the OS baseline results alongside the perfherder data (merged).

Comment 2

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1553335 - Run Android OS baseline test before all power tests. r?#perftest

This patch adds an Android OS baseline test which runs before all tests when '--power-test' is supplied. The data is then added to the test's PERFHERDER_DATA blob as power data that is suffixed with '-baseline'.

Comment 3

[Greg Mierzwinski [:sparky]]

Here's a try run for this patch: https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=d81ff5f312599d029b9389b7ccd42df3d6c1d7c9

Note that perfherder adds all the data together (including the baseline data) so we get an artificial increase in the power usage. :rwood, is it possible to submit two PERFHERDER_DATA entries to perfherder from one test?

Comment 4

[Robert Wood [:rwood]]

(In reply to Greg Mierzwinski [:sparky] from comment #3)

Here's a try run for this patch: https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=d81ff5f312599d029b9389b7ccd42df3d6c1d7c9

Note that perfherder adds all the data together (including the baseline data) so we get an artificial increase in the power usage. :rwood, is it possible to submit two PERFHERDER_DATA entries to perfherder from one test?

Yes you can submit multiple PERFHERDER_DATA json blobs in any test run. Raptor can already do that i.e. submit one blob for the power info and another blob for the actual test results i.e. if running speedometer-power; however currently we have that turned off and just submitting power data as that was the initial decision.

Comment 5

[Robert Wood [:rwood]]

Is this actively being worked on? Looks like it's been idle for 20 days - setting to P2 until it is active again, thanks.

Comment 6

[Greg Mierzwinski [:sparky]]

Try run for latest patch: https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=3a923c2df79d5a7a911ab58c3a2f9f3ffb33f0cc

Comment 7

[Greg Mierzwinski [:sparky]]

Last patch failed, good one is here (phab was updated as well): https://treeherder.mozilla.org/#/jobs?repo=try&revision=9b5709745a046bc634e52dff73c825c14f38dc07

Comment 8

[Greg Mierzwinski [:sparky]]

Try run for the latest patch (integrated re fixes, and removed the "multiple by 20" step): https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=dcd0588c26b26de71984f6b3e93320d4b3b6889e

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/45d6004b3194
Run Android OS baseline test before all power tests. r=perftest-reviewers,rwood

Comment 10

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/45d6004b3194