1628073 - Improve the workflow of using the profiler in mochitests

Status: RESOLVED FIXED

(Core :: Gecko Profiler, enhancement, P2)

Description

[Greg Tatum [:gregtatum] (REO Fx132)]

This bug is for tracking some work I'm doing on attempting to improve the workflow for using the profiler with mochitests and our testing infrastructure.

Comment 1

[Greg Tatum [:gregtatum] (REO Fx132)]

Attachment: Bug 1628073 - Make shutdown profiling nicer for mochitests

Comment 2

[Greg Tatum [:gregtatum] (REO Fx132)]

Attachment: Bug 1628073 - Rewrite view_gecko_profile.py to open a URL; r?sparky

Depends on D70088

Comment 3

[Greg Tatum [:gregtatum] (REO Fx132)]

Attachment: Bug 1628073 - Add CLI flags to mochitests for the profiler; r?sparky

Depends on D70089

Comment 4

[Greg Tatum [:gregtatum] (REO Fx132)]

Attachment: Bug 1628073 - Make the defaults for the profiler settings nicer for mochitests; r?gerald

The defaults should serve our users. I did some profiling with mochitests
and the defaults weren't ideal. It would be nice if the profiler "just worked"
for most of the situations. It would be great to teach the environment variables
about presets, but for now this will make the profiler easier to use for our
users in the general case.

This patch adds both screenshots and updates the entries count. This causes
additional overhead, but people on low-end devices can still tweak the settings
to remove that overhead.

Depends on D70090

Comment 5

[Greg Tatum [:gregtatum] (REO Fx132)]

Attachment: Bug 1628073 - Document some of the symbolication python code; r?sparky

Depends on D70090

Comment 6

[Greg Tatum [:gregtatum] (REO Fx132)]

Attachment: Bug 1628073 - Move gecko profiler testing files to mozgeckoprofiler; r?sparky

I need to add symbolication support for the mochitest Gecko Profiler command line
option. These profiles also need to be symbolicated. Unfortunately, there is not
a common place where I could use these files. Talos and Raptor each had their
own copy of the snappy symbolication server.

This commit consolidates these packages into a re-usable mozbase package that can
be used in mochitests, and eventually in other places like xpcshell tests.

I stubbed out a test file, but it doesn't do anything quite yet. This commit makes
it so that the tests still work in Raptor and Talos, but doesn't add any features.
It also doesn't try too hard to make the files look like a mozbase package.

Depends on D74287

Comment 7

[Greg Tatum [:gregtatum] (REO Fx132)]

Attachment: Bug 1628073 - Hook up symbolication to mochitests; r?sparky

Depends on D74289

Comment 8

[Greg Tatum [:gregtatum] (REO Fx132)]

Attachment: Bug 1628073 - Use mozlog in the mozgeckoprofiler; r?sparky

This patch changes over all of the logging to use the structured mozlog package.
I also ensured that the messages were logged in a way to be useful for end users
of mochitests. This means there were a few small changes in methods in order to
accommodate the logging. The behavior outside of logging should be the same.

Depends on D74290

Comment 9

[Greg Tatum [:gregtatum] (REO Fx132)]

Attachment: Bug 1628073 - Add documentation to mozgeckoprofiler; r?sparky

Depends on D74291

Comment 10

[Greg Tatum [:gregtatum] (REO Fx132)]

Attachment: Bug 1628073 - Remove the view_gecko_profile.py tool; r?sparky

Depends on D74292

Comment 11

[Greg Tatum [:gregtatum] (REO Fx132)]

Attachment: Bug 1628073 - Add a test for the view gecko profile tool; r?sparky

Depends on D74294

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/619d4f31f851
Make the defaults for the profiler settings nicer for mochitests; r=gerald

Comment 13

[Alexandru Michis [:malexandru]]

https://hg.mozilla.org/mozilla-central/rev/619d4f31f851

Comment 14

[Greg Tatum [:gregtatum] (REO Fx132)]

Re-opening as this has more patches to land.

Comment 15

[Greg Tatum [:gregtatum] (REO Fx132)]

Attachment: Bug 1628073 - Move the view_gecko_profile_from_mochitest function; r?gbrown

Depends on D74295

Comment 16

[Greg Tatum [:gregtatum] (REO Fx132)]

Attachment: Bug 1628073 - Run black autoformatting on mozgeckoprofiler; r?sparky

Depends on D75577

Comment 17

[Greg Tatum [:gregtatum] (REO Fx132)]

Attachment: Bug 1628073 - Add python 3 support to mozgeckoprofiler; r?sparky

Depends on D75578

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/448ea7e8873c
Rewrite view_gecko_profile.py to open a URL; r=perftest-reviewers,sparky
https://hg.mozilla.org/integration/autoland/rev/ed2404865290
Add CLI flags to mochitests for the profiler; r=gbrown
https://hg.mozilla.org/integration/autoland/rev/3223007370b6
Document some of the symbolication python code; r=sparky,perftest-reviewers
https://hg.mozilla.org/integration/autoland/rev/bfb980a9199b
Move gecko profiler testing files to mozgeckoprofiler; r=sparky,perftest-reviewers,gbrown
https://hg.mozilla.org/integration/autoland/rev/699e0e8cd072
Hook up symbolication to mochitests; r=gbrown
https://hg.mozilla.org/integration/autoland/rev/e2f1ffe1f769
Use mozlog in the mozgeckoprofiler; r=gbrown
https://hg.mozilla.org/integration/autoland/rev/0c606644f4e3
Add documentation to mozgeckoprofiler; r=sparky
https://hg.mozilla.org/integration/autoland/rev/de3e4dec928c
Remove the view_gecko_profile.py tool; r=sparky,perftest-reviewers
https://hg.mozilla.org/integration/autoland/rev/e99da2a9254f
Add a test for the view gecko profile tool; r=sparky
https://hg.mozilla.org/integration/autoland/rev/49ed1a5fc265
Move the view_gecko_profile_from_mochitest function; r=gbrown
https://hg.mozilla.org/integration/autoland/rev/c14463b2f3e8
Run black autoformatting on mozgeckoprofiler; r=sparky
https://hg.mozilla.org/integration/autoland/rev/6f4edb094e0d
Add python 3 support to mozgeckoprofiler; r=sparky

Comment 19

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/448ea7e8873c
https://hg.mozilla.org/mozilla-central/rev/ed2404865290
https://hg.mozilla.org/mozilla-central/rev/3223007370b6
https://hg.mozilla.org/mozilla-central/rev/bfb980a9199b
https://hg.mozilla.org/mozilla-central/rev/699e0e8cd072
https://hg.mozilla.org/mozilla-central/rev/e2f1ffe1f769
https://hg.mozilla.org/mozilla-central/rev/0c606644f4e3
https://hg.mozilla.org/mozilla-central/rev/de3e4dec928c
https://hg.mozilla.org/mozilla-central/rev/e99da2a9254f
https://hg.mozilla.org/mozilla-central/rev/49ed1a5fc265
https://hg.mozilla.org/mozilla-central/rev/c14463b2f3e8
https://hg.mozilla.org/mozilla-central/rev/6f4edb094e0d