1631919 - Get the Firefox Profiler working in Thunderbird

Status: RESOLVED WORKSFORME

(Thunderbird :: General, task, P2)

Description

[Paul Morris [:pmorris]]

The performance profiler for Firefox no longer uses an add-on to get performance data out of the app, instead the UI (e.g. for starting and stopping performance profiling sessions) is built in. So changes are needed for this to work in Thunderbird. There may be other issues besides the UI.

Profiling is enabled via a pref (devtools.performance.enabled). When that is true a toolbar button for the UI is added to the toolbar or toolbar customization area on Firefox. In TB we are using the older toolbar code than the CustomizableUI code used in Firefox, so this doesn't work. The button does not appear. We do have the CustomizableUI code for the app menu, so we may be able to tweak it to add the button to the TB toolbar customization area instead. That's the first step here.

Other options mentioned by Florian via chat:

"if you are just trying to get a single profile right now, you might start the profiler using the nsIProfiler interface from the console, but that won't be convenient"

"another way to get a profile quickly would be capture a profile of the whole duration of the process, using MOZ_PROFILER_STARTUP=1 and MOZ_PROFILER_SHUTDOWN="path/filename.json" in the environment"

My initial attempts at the MOZ_PROFILER_ option have not worked. (The file did not contain any profiler data after running the app.)

Edit: It's working for me now.

See https://profiler.firefox.com/

Comment 1

[Paul Morris [:pmorris]]

Sigh... despite devoting some hours to it, I haven't been able to get the new profiler UI button and popup code to work with Thunderbird's older toolbar code. I see a few ways forward:

1. Copy the profiler UI code into comm-central, then modify it to work with Thunderbird's toolbar code. (As we did with the app menu.)
2. Migrate Thunderbird's toolbars to use the CustomizableUI code so it is more compatible with the profiler UI code.
3. Expose the functionality in another way, maybe a lazily loadable module that exposes functions, that can be used from the developer console.

In the long run, we may be better off putting effort toward #2 rather than #1. (Even if we ended up having to copy the profiler UI code anyway, many fewer changes to it would be needed.) Maybe #3 could work as a short term solution? Happy to discuss in more detail.

Comment 2

[Wayne Mery (:wsmwk)]

I would be most in favor of #2, because over the life of profiler which I have followed closely, it hasn't worked / been broken at least four times. And most of the time has been broken. It has been a never ending battle

Comment 3

[Wayne Mery (:wsmwk)]

And that said I'm glad you list a short term solution. If #2 is hard then we might need #3 first because with address book and many other backend and front end changes coming we could really need a perf tool.

Comment 4

[Florian Quèze [:florian]]

Another possible option would be to use remote profiling, ie use Firefox as the capturing UI. This is how we profile Firefox for Android. This would be less convenient, but likely easier to maintain once you get it working.

If you are looking for another way to expose the profiler (your option 3 I guess), you could create a simple button that would just start the profiler when clicked the first time, and capture the profile and stop the profiler when clicked a second time. If you find a way to load about:profiling in Thunderbird so that profiler settings can still be edited, that might be enough.

Comment 5

[Paul Morris [:pmorris]]

Thanks Florian and Wayne for your thoughts. I spoke with Magnus about this in a meeting today.

The short term plan is to use the remote profiling that Florian suggested. I'll give it a try and, assuming everything works fine, document the steps on developer.thunderbird.net.

A potential beyond-short-term plan (likely after TB 78 is out) would be to create a simple add-on (using a WebExtension experiment) that added a button to turn profiling on/off, and possibly also offer a way to open about:profiling in Thunderbird. (Or maybe instead have a separate TB add-on for opening various pages like about:profiling in Thunderbird. That could come in handy for other uses too. Maybe it already exists?) Making it an add-on is a simple way to avoid adding it to the UI for regular users who won't need to do profiling.

Comment 6

[Greg Tatum [:gregtatum]]

I think remote profiling will be the most consistent non-breaking solution, as the UI has lots of tight integrations with Firefox. I'd be a little wary of integrating with the WebExtension API, as I'm not sure how long that will stick around.

Comment 7

[Paul Morris [:pmorris]]

(In reply to Greg Tatum [:gregtatum] from comment #6)

I think remote profiling will be the most consistent non-breaking solution, as the UI has lots of tight integrations with Firefox. I'd be a little wary of integrating with the WebExtension API, as I'm not sure how long that will stick around.

That makes sense. If and when we went down the add-on path, we would use a WebExtensions experiment to provide the API for the profiling code, rather than using the official WebExtensions API. Basically, we could port the official WebExtensions API to a WebExtensions experiment and adapt as needed.

Comment 8

[Paul Morris [:pmorris]]

I've looked into remote profiling, but without success so far. Any pointers on how to get this to work with a Thunderbird build?

I've used remote debugging to debug Thunderbird xpcshell tests with mach xpcshell-test --jsdebugger ... over a localhost connection. I assume this would be similar, so there must be a way to launch Thunderbird to allow that kind of connection. Maybe mach run --debugger is the right idea, but if so, what to pass as the --debugger?

Comment 9

[Paul Morris [:pmorris]]

I found this post with instructions from 2013: https://kewisch.wordpress.com/2013/06/13/the-thunderbird-remote-debugger-is-alive/
Edit: and this https://developer.mozilla.org/en-US/docs/Tools/Remote_Debugging/Thunderbird

I have "Allow Remote Debugging" checked in local build of Thunderbird and in FF, and "devtools.debugger.remote-enabled" as true in FF, restarted FF, etc. In FF's about:debugging I click connect to localhost:6000 (and 6080) and get a "Connection Failed" message. Maybe Philipp knows what to try next?

Comment 10

[alta88]

(In reply to Paul Morris [:pmorris] from comment #1)

3. Expose the functionality in another way, maybe a lazily loadable module that exposes functions, that can be used from the developer console.

I must be missing something big here, but aren't we talking about Performance tab in the Developer Toolbox, which is enabled in Settings? While the Fx style controls button would be nice on a toolbar, it's not like major hackery is needed to get a Tb profile file generated and uploaded...

Comment 11

[Magnus Melin [:mkmelin] (away, back June 30)]

I forgot that's there, thx! After a few tries, the setting to enable it agreed to be set.

That seems to work - but not if I enable the "Use new performance recorder ". Then if I trt trying to record something, I get
console.error: "Failed to get the active BrowsingContext ID while starting the profiler."
And on stopping
JavaScript error: resource://devtools/client/performance-new/browser.js, line 77: Error: No browser window

Comment 12

[Paul Morris [:pmorris]]

Thanks alta88! I tried it and had a similar results as mkmelin. Things worked unless I checked "Use new performance recorder". Same errors with that checked.

Comment 13

[Greg Tatum [:gregtatum]]

The "Failed to get the active BrowsingContext ID while starting the profiler." is just a warning really, which should be fine in the case of profiling Thunderbird.

The second error on "No browser window" is the real issue.

The error is caused because the browser expects to be able to open a new tab to inject the profile into profiler.firefox.com. A work around would be to handle this error by offering to save the profile with some alternate method. Perhaps opening up a save dialog so the profile can be saved locally, then manually open it in profiler.firefox.com? I'm happy to review a patch that does this:

https://searchfox.org/mozilla-central/rev/9f074fab9bf905fad62e7cc32faf121195f4ba46/devtools/client/performance-new/browser.js

Comment 14

[Paul Morris [:pmorris]]

Late last week I tried to create a startup-and-shutdown profile using environment variables (for the folders sidebar changes Alessandro is working on), but the profile file was not generated for some reason. I haven't had a chance to look into this further. Next step is to follow this tip from gerald from the Firefox Profiler chat:

If you're still having troubles, try to add MOZ_LOG=prof:4 (assuming Thunderbird has the same logging system as Firefox), and you should see logs from the profiler. The shutdown operation to look for is "locked_profiler_save_profile_to_file(...)".

Comment 15

[Wayne Mery (:wsmwk)]

Any news?

(I think Philipp's NI is now obsolete?)

Comment 16

[Paul Morris [:pmorris]]

No news. I haven't been able to get to this (since my last comment) due to other priorities.

Comment 17

[Wayne Mery (:wsmwk)]

Per standard8 on chat "the environment options for startup/shutdown seem to work in 78".

In that case, is there anything left to do here?

Comment 18

[Paul Morris [:pmorris]]

There's still the "Use new performance recorder" feature to get working (comment 11 and comment 13).

Comment 19

[Wayne Mery (:wsmwk)]

Mark has instructions for profiler at https://github.com/thunderbird-conversations/thunderbird-conversations/wiki/Profiling-Conversation's-Performance

Comment 20

[Wayne Mery (:wsmwk)]

We now have instructions at https://support.mozilla.org/en-US/kb/profiling-thunderbird-performance

Comment 21

[Alessandro Castellani [:aleca]]

The profile works but it doesn't record all the things that the Firefox profiler does.
IIRC, Rob talked about it in a meeting mentioning that he might be able to properly wire things up and make it on par.

Comment 22

[Wayne Mery (:wsmwk)]

Rob, can this fit into your list of things to look at? This would help with some current issues. Bug 1720807 is one of them.

Comment 23

[Rob Lemley [:rjl]]

I'm not sure that there's anything I can do here. To me, it looks like everything is working . Aleca, can you be more specific as to what is missing from Thunderbird profiles?

Comment 24

[Alessandro Castellani [:aleca]]

The profiler seems to record everything correctly and I'm not seeing any missing data.

Comment 25

[Mark Banner (:standard8)]

I just commented on this in Matrix:

The toolbox UI is obviously out-of-date and has to be enabled each time. I don't expect it to be updated, but maybe it would be possible to port the profiler toolbar button that Firefox now has?

Also using the toolbox UI to save profiles doesn't seem to save screenshots. If I profile startup & shutdown using the env vars, then I do get screenshots.

Comment 26

[Florian Quèze [:florian]]

(In reply to Mark Banner (:standard8) from comment #25)

Also using the toolbox UI to save profiles doesn't seem to save screenshots. If I profile startup & shutdown using the env vars, then I do get screenshots.

Until bug 1742722 which was fixed a week ago, the 'Renderer' thread needed to be included in the profile for screenshots to be saved. Could that be the reason why you were missing screenshots?

Comment 27

[Mark Banner (:standard8)]

(In reply to Florian Quèze [:florian] from comment #26)

(In reply to Mark Banner (:standard8) from comment #25)

Also using the toolbox UI to save profiles doesn't seem to save screenshots. If I profile startup & shutdown using the env vars, then I do get screenshots.

Until bug 1742722 which was fixed a week ago, the 'Renderer' thread needed to be included in the profile for screenshots to be saved. Could that be the reason why you were missing screenshots?

I tried it on the latest mozilla-central + comm-central combination.

Comment 28

[Wayne Mery (:wsmwk)]

(In reply to Alessandro Castellani [:aleca] from comment #24)

The profiler seems to record everything correctly and I'm not seeing any missing data.

And comment 11 and comment 13 are resolved?

mkm, who do you want to assign?

Comment 29

[Markus Stange [:mstange]]

For a fully-featured workaround with all profiler data, you can do the following:

1. Run Thunderbird Daily for best results. (Release Thunderbird doesn't have full stacks on macOS.)
2. Open the error console, for example using Cmd+Shift+J.
3. Run the following command to start the profiler:

Services.profiler.StartProfiler(10000000, 1.0, ["stackwalk", "js", "cpu", "screenshots"], ["GeckoMain", "Compositor", "Renderer", "DOM Worker"])
// Parameters: buffer size in bytes, sampling interval in float milliseconds, profiler features, thread filter

4. Reproduce the slow interaction.
5. Save the recording to a file, by executing the following in the error console (adjust the path):

await Services.profiler.dumpProfileToFileAsync("/Users/mstange/Desktop/thunderbird-profile.json")
// (on Windows, make sure to escape any backslashes)

6. Stop the profiler:

Services.profiler.StopProfiler()

7. In Firefox, go to profiler.firefox.com and load the file into it (using drag and drop or using the file picker).

Comment 30

[Rob Lemley [:rjl]]

It looks like bug 1747879 is relevant here with regard to C++ code.

Comment 31

[Magnus Melin [:mkmelin] (away, back June 30)]

Working atm (bug 1771304).