1836281 - Gather cpu usage from Chrome browser tests

Status: NEW

(Testing :: Raptor, task, P3)

Description

[Greg Mierzwinski [:sparky]]

This bug is for gathering the cpu usage information for chrome browsers during pageload tests.

Comment 1

[Greg Mierzwinski [:sparky]]

:florian, would you know if there's a way that we could gather the CPU time from the resource usage profile during the test?
I'm wondering if the CPU from the resource usage profile could be used for this instead since it's gathered on Firefox, and Chrome tests/tasks in the same way.

Currently, CPU time on chrome is gathered in a very different way from on Firefox which makes comparing the numbers to each other difficult: https://github.com/sitespeedio/browsertime/blob/465f1de3de9b4e59b8ad076ce9317c829de35af3/lib/chrome/parseCpuTrace.js#L14

Comment 2

[Florian Quèze [:florian]]

From the resource usage profile it would be (mostly) straightforward to measure how much CPU was used by the entire system while running the test. I assume what we are trying to measure is the CPU time used by the processes of the browsers. I don't see a way to capture that information from the resource use profile. Recording CPU use for the entire system could be interesting (it would count work done by the graphics driver or the operating system as a consequence of the browser running for example) but I think it would be a new metric.

Comment 3

[Greg Mierzwinski [:sparky]]

Ok, I agree it would be a new metric, and it should be more comparable between browsers vs the browsertime techniques.

Do you know what we would need to do to access that info while the test step is running? I would need to grab that data while we're in a JS file like this: https://searchfox.org/mozilla-central/source/testing/raptor/browsertime/speedometer3.js

Comment 4

[Florian Quèze [:florian]]

Ah, I thought you had the resource-usage profile JSON file already. If you need to access it while the test is running (and from JS), I'm not sure. It's all done in python in https://searchfox.org/mozilla-central/source/testing/mozbase/mozsystemmonitor/mozsystemmonitor/resourcemonitor.py

Accessing it from within the existing python process might be harder than trying to access it from the operating system (hoping some node libraries exist to help with that).

Comment 5

[Greg Mierzwinski [:sparky]]

Ok, so what I did was use the os module in node to gather the times, and it seems to be working for both Firefox, and Chrome: https://treeherder.mozilla.org/jobs?repo=try&tier=1%2C2%2C3&revision=92c11a58865d42b3aefb850a627948ae418de481

I think it's close to what you're doing with the resource monitor. Here's the code for that: https://hg.mozilla.org/try/rev/20335217dbecb09105066d4d83b9cd896a4b5dc2

There's an additional field called idle, and I'm not sure if we would want to include that in the cpu time measurements. It's the amount of time that the CPUs spent in idle.

Comment 6

[Markus Stange [:mstange]]

This looks perfect. You're right, we are not interested in the idle time, just in user + sys.

Comment 7

[Greg Mierzwinski [:sparky]]

Great, thanks! I added a few retriggers, and the data looks pretty stable too: https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&replicates=0&series=try,195698,1,13&series=try,127847,1,13&timerange=1209600&zoom=1725966201475,1725993513458,66128.85835514442,116167.76129632091

Comment 8

[Greg Mierzwinski [:sparky]]

Something to note is this method will only work on desktop platforms. We'll need to do something different for Android (I'll probably copy from the background resource test for this).