Closed Bug 1550702 Opened 5 months ago Closed 4 months ago

Improve handling of Gecko profiler data

Categories

(Testing :: Raptor, defect, P2)

Version 3
defect

Tracking

(firefox69 fixed)

RESOLVED FIXED
mozilla69
Tracking Status
firefox69 --- fixed

People

(Reporter: whimboo, Assigned: whimboo)

References

(Blocks 2 open bugs)

Details

Attachments

(1 file)

When running tests for the upcoming Youtube playback performance benchmark (bug 1548845) the generated gecko profiles are huge! As such the current handling of its data causes lots of problems, because it is way to slow. Here some ideas:

  1. Don't call console.log() for the profile data before sending the data to the Python harness. This could add a hundreds of MB entry to the console, and causes a slowdown.

  2. The Raptor web extension tries to send the full profile data via a single packet over the local / remote (android) socket connection. Imagine what could happen when you do that with >100 MB or more data.

The best way to improve the performance would be to let the Raptor web extension directly save the profile to disk, and only inform the harness of the path. Sadly this is restricted for web extensions beside storing data via local storage.

Given that the profiler is a Gecko only thing anyway, I will check if I can build an addition via WebExtensions Experiments, which would allow us to access the profiler directly. That way we would simply call dumpProfileToFileAsync():

https://searchfox.org/mozilla-central/rev/b9da45f63cb567244933c77b2c7e827a057d3f9b/tools/profiler/gecko/nsIProfiler.idl#67-72

Assignee: nobody → hskupin
Status: NEW → ASSIGNED
Priority: -- → P1
No longer blocks: 1548845
See Also: → 1548845
See Also: → 1544463
Blocks: 1544544

For a description of Webextensions Experiments see:
https://webextensions-experiments.readthedocs.io/en/latest/

See Also: → 1551992
Blocks: 1552738
See Also: → 1552845

The dumping of the profile directly to disk will be implemented in the profiler experimental API directly, which is covered by bug 1552845. Once that has been done Raptor can make use of it.

Depends on: 1552845
Blocks: 1553131

Improvements can be seen eg. for the speedometer tests which have profiles of about 33MB.

With the unpatched version it takes 5s:
https://treeherder.mozilla.org/logviewer.html#?job_id=248951432&repo=mozilla-central

With the patched version it takes 2s:
https://treeherder.mozilla.org/logviewer.html#?job_id=248966448&repo=try

Note that we also have a synchronous handling before shutting down the browser.

Priority: P1 → P2
Pushed by hskupin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e90b1260461b
[raptor] Switch to dumpProfileToFile() for saving the Gecko profile. r=perftest-reviewers,sparky
Status: ASSIGNED → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla69
You need to log in before you can comment on or make changes to this bug.