Closed Bug 1546741 Opened 5 years ago Closed 5 years ago

Land initial idle-power-usage measurement test for Android

Categories

(Testing :: Raptor, enhancement, P1)

Version 3
enhancement

Tracking

(firefox68 fixed)

RESOLVED FIXED
mozilla68
Tracking Status
firefox68 --- fixed

People

(Reporter: stephend, Assigned: stephend)

References

Details

Attachments

(1 file, 7 obsolete files)

To get things started, Greg and I plan to land a "browser-idle" test -- without any specific background colors(*), so likely just about:blank, and sit for 20 minutes -- across the many browser variants. We'll be collecting battery stats (battery-%-levels, charge counters, perhaps others) just before test run, and shortly thereafter, in the post-test cleanup.

  • Bug 1511350 concerns testing a "dark mode" proxy, as a start, and will have us move soon into enabling and measuring the real, implemented feature (in at least Fenix).
Type: defect → enhancement
Priority: -- → P1

After conferring and confirming with Dave, today, we'll focus on, first and foremost, these three (3) "idle" tests (and their results -> CI -> Firefox perf dashboard[*]):

  1. Idle phone - this is/should be the hard work Greg's done to baseline and vet, across models & Android versions
  2. Idle browser, running/displayed in the foreground
  3. Idle browser, running as a background app/in the background

As far as "dark mode" goes -- both as a proxy for the feature, and in Fenix, the one app in which it's available now -- we'll leave that work until further down the line, a bit.

Dave can fact-check me on that +/or expound further.

  • Note to self: look further at CI work (scoping).
Flags: needinfo?(dave.hunt)

Rob and I tried to get #2, "Idle browser, running/displayed in the foreground" working. We started with Greg's patch (https://www.irccloud.com/pastebin/GI83y0lT/rap-power.patch) -- for which I did an hg import; trying to hg diff isn't yielding the full changes, perhaps because of that as a staring point? -- but haven't gotten that case working, yet.

Works:

  • Using Raptor, starting with about:blank -> injecting/passing in a URL https://www.mozilla.org via a resource.js content script[0]
./mach raptor-test --test raptor-res-power-dark --app fennec --binary org.mozilla.firefox

Doesn’t work:

  • Using Raptor, starting with about:blank -> injecting about:blank (to get a “white”-page proxy quickly) via a resource.js content script
./mach raptor-test --test raptor-res-power-idle --app geckoview --binary org.mozilla.geckoview_example

04-28 01:14:28.490 24882 24898 E GeckoConsole: [JavaScript Error: "Unchecked lastError value: Error: Illegal URL: about:blank" {file: "moz-extension://8e1cb86a-9750-48b1-909e-e4f2f1c3ddaf/runner.js" line: 425}] 04-28 01:14:28.490 24882 24898 E GeckoConsole: nextCycle/</<@moz-extension://8e1cb86a-9750-48b1-909e-e4f2f1c3ddaf/runner.js:425:18

[0] https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/content_scripts

You'll still need 1) resource.js 2) runner.js and 3) manifest.json, which I'll attach, next

Attached file resource.js (obsolete) —
Attached file runner.js (obsolete) —

Rob/Sparky, have I missed/gotten anything wrong, from particularly https://bugzilla.mozilla.org/show_bug.cgi?id=1546741#c2?

Flags: needinfo?(rwood)

(In reply to Stephen Donner [:stephend] from comment #7)

Rob/Sparky, have I missed/gotten anything wrong, from particularly https://bugzilla.mozilla.org/show_bug.cgi?id=1546741#c2?

For some reason with everything we tried, we weren't able to get your 'resource.js' injected into the web page content, whether using 'about:blank' or a site like 'mozilla.org'. I'll apply your patch and try to figure out what's going on.

Flags: needinfo?(rwood)
Attachment #9061281 - Attachment is obsolete: true
Attachment #9061285 - Attachment is obsolete: true
Attachment #9061283 - Attachment is obsolete: true
Attachment #9061284 - Attachment is obsolete: true

Having as of yet no (consistent) success with about:blank-replacing & injecting test content, our next (and current) approach to try to achieve an MVP is thus:

  • create and upload a minimal-ish HTML (blank) test page, which we'll...
  • upload (blank-power-test-page.html, from above) to tooltool
  • after which, we'll (re)play that blank HTML test page back via mitmproxy, like the other pages
    • runner.js will open that test page
    • add a new function in runner.js that just waits x minutes (20min, for our idle tests)
    • then closes open tab(s)
    • finish up by sending the "shutdown" message to the control server
Attachment #9061482 - Attachment is obsolete: true
Attachment #9061815 - Attachment is obsolete: true

Comment on attachment 9061983 [details]
Bug 1546741. Land initial idle-power-usage-measurement test for Android. r?rwood,stephendonner

Marking as obsolete in lieu of my patch which incorporates yours, and fixes the lint issues <3

https://phabricator.services.mozilla.com/D29654

Attachment #9061983 - Attachment is obsolete: true
Attachment #9062177 - Attachment description: Bug 1546741. Land initial idle-power-usage-measurement test for Android. r?rwood → Bug 1546741. Land initial idle-power-usage-measurement test for Android. r?rwood,sparky
Attachment #9061983 - Attachment is obsolete: false
Attachment #9061983 - Attachment is obsolete: true
Attachment #9062177 - Attachment is obsolete: true

(In reply to Stephen Donner [:stephend] from comment #1)

Dave can fact-check me on that +/or expound further.

Yep, this all sounds good to me.

Flags: needinfo?(dave.hunt)
Attachment #9062177 - Attachment is obsolete: false
Pushed by sdonner@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/751aaa1009d3
Land initial idle-power-usage-measurement test for Android. r=rwood
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: