Make AddonManager startup less expensive in a brand new profile or after an update
Categories
(Toolkit :: Add-ons Manager, enhancement, P3)
Tracking
()
| Performance Impact | low |
People
(Reporter: aswan, Unassigned)
References
(Blocks 1 open bug)
Details
(Keywords: perf:frontend, perf:startup)
The function XPIProvider.checkForChanges() does a lot of work whenever we start in a new profile. It also does a bunch of work when the browser is first started after an upgrade, but that's not the focus of this bug. When starting in a new profile, most of this work is synchronously reading, parsing, validating, etc. system addons (https://searchfox.org/mozilla-central/rev/75294521381b331f821aad3d6b60636844080ee2/toolkit/mozapps/extensions/internal/XPIDatabase.jsm#2408)
I did some very crude and unscientific measurements and it looks like on an old-ish mac laptop, all of checkForChanges() takes a few hundred milliseconds. I would expect it to be quite a bit more on a machine without an SSD, so I think there's at least a potential for some significant improvement here.
I have two ideas:
-
Validating system addons is something we can easily do once in automation rather than over and over in the field. This is a little hand-wavy but we could pre-compute the information that's stored in addonsStartup and extensions.json so that those can be created in a new profile more cheaply.
-
Since system addons authors already can't rely on addon code running early in startup, and since we're generally moving away from system addons, we could defer the initial installation of system addons until later in startup (ie at least after we've painted the first browser window). This has the potential to be a little jarring if system addons that create UI elements are slow to install and their buttons suddenly appear some time after it appears that the browser has already started. I haven't really thought it out but perhaps we could piggyback on the existing webextension startupcache machinery to pre-populate UI elements from system addons to avoid that problem.
This is obviously still pretty conceptual, looking for feedback at this point. rhelmer, any thoughts about option 2 in particular? (feedback from anybody on either point or on other options I didn't think of is of course welcome as well)
|
||
Comment 1•5 years ago
•
|
||
(In reply to Andrew Swan [:aswan] from comment #0)
The function XPIProvider.checkForChanges() does a lot of work whenever we start in a new profile. It also does a bunch of work when the browser is first started after an upgrade, but that's not the focus of this bug. When starting in a new profile, most of this work is synchronously reading, parsing, validating, etc. system addons (https://searchfox.org/mozilla-central/rev/75294521381b331f821aad3d6b60636844080ee2/toolkit/mozapps/extensions/internal/XPIDatabase.jsm#2408)
I did some very crude and unscientific measurements and it looks like on an old-ish mac laptop, all of checkForChanges() takes a few hundred milliseconds. I would expect it to be quite a bit more on a machine without an SSD, so I think there's at least a potential for some significant improvement here.
Nice! Thanks for finding that, I wish this sort of thing showed up on Talos.
I have two ideas:
- Validating system addons is something we can easily do once in automation rather than over and over in the field. This is a little hand-wavy but we could pre-compute the information that's stored in addonsStartup and extensions.json so that those can be created in a new profile more cheaply.
I think it'd be pretty easy to do this at build time, we could for instance store a file inside the omni jar and addons manager could refer to it only when creating new profiles.
- Since system addons authors already can't rely on addon code running early in startup, and since we're generally moving away from system addons, we could defer the initial installation of system addons until later in startup (ie at least after we've painted the first browser window).
I am fine with this in principle, but I worry that it'll cause problems with system add-ons that we're currently shipping built-in, taking a quick look I see:
Would that be affected? I don't think this is the sort of thing we're encouraging moving forward (browser code expecting add-on code to be registered), so fixing the add-on in question / moving this into tree / etc is fine, bit more work than #1 I'd guess.
This has the potential to be a little jarring if system addons that create UI elements are slow to install and their buttons suddenly appear some time after it appears that the browser has already started. I haven't really thought it out but perhaps we could piggyback on the existing webextension startupcache machinery to pre-populate UI elements from system addons to avoid that problem.
I'm wondering if we even need to solve this problem, given that none of the current system add-ons appear in the default UI (Screenshots is hidden in the pageAction menu), I don't know of any plans to build such a thing.
This is obviously still pretty conceptual, looking for feedback at this point. rhelmer, any thoughts about option 2 in particular? (feedback from anybody on either point or on other options I didn't think of is of course welcome as well)
Do you have any thoughts on how big of an improvement you expect these to be? Neither one strikes me as super difficult, but I'd expect to have to potentially deal with more fallout from existing system add-ons with #2.
|
Reporter | |
Comment 2•5 years ago
|
||
(In reply to Robert Helmer [:rhelmer] from comment #1)
Nice! Thanks for finding that, I wish this sort of thing showed up on Talos.
Yes, talos has deliberately not tried to measure first startup. Measuring it faithfully is tricky (ie, the real scenario we want to measure is where the browser is starting with an empty directory, but to instrument it with talos we need to set preferences, install test extensions etc.) We've talked about what we can do to approximate it and how useful data measured that way would be, stay tuned...
- Since system addons authors already can't rely on addon code running early in startup, and since we're generally moving away from system addons, we could defer the initial installation of system addons until later in startup (ie at least after we've painted the first browser window).
I am fine with this in principle, but I worry that it'll cause problems with system add-ons that we're currently shipping built-in, taking a quick look I see:
Would that be affected? I don't think this is the sort of thing we're encouraging moving forward (browser code expecting add-on code to be registered), so fixing the add-on in question / moving this into tree / etc is fine, bit more work than #1 I'd guess.
My recollection from talking with MattN about this some time ago is that formautofill is implemented as a system addon so that heuristics for individual sites can be rolled out on the fly (a la webcompat). The whole thing was initially implemented as an extension, but at this point, everybody would be happy to turn the core bits into a built-in browser component while maintaining a method to ship improved heuristics out of cycle. I thought there was actually a bug for that, but I can't seem to find it.
This is obviously still pretty conceptual, looking for feedback at this point. rhelmer, any thoughts about option 2 in particular? (feedback from anybody on either point or on other options I didn't think of is of course welcome as well)
Do you have any thoughts on how big of an improvement you expect these to be? Neither one strikes me as super difficult, but I'd expect to have to potentially deal with more fallout from existing system add-ons with #2.
My quick and dirty test was:
- Start the browser with a new empty profile
- Open the browser console and enter
{ let i = Services.startup.getStartupInfo(); i.firstPaint - i.process; } - Repeat a few times to get an average startup time
- Comment out this line and re-build https://searchfox.org/mozilla-central/rev/b756e6d00728dda4121f8278a744381d8643317a/toolkit/mozapps/extensions/internal/XPIProvider.jsm#2265
- Repeat steps 1-3, compare
Step 4 above gives you a browser that's basically functional but with no screenshots, no formautofill, etc. but this approximates how much faster first paint would be if all the work done from that line moved to some time after first paint. As I mentioned earlier, I was seeing a consistent difference of a few hundred milliseconds which is pretty significant.
Another thought is that we don't ever seem to ship out-of-cycle system addon updates and we have other methods (ie Normanday) to ship extensions if necessary, so we could simply migrate our existing system addons to either builtin addons that can't be updated out-of-cytcle but that we don't install during startup or to builtin components. That would essentially give the same benefits as #2.
|
|
||
Comment 3•5 years ago
|
||
(In reply to Andrew Swan [:aswan] from comment #2)
(In reply to Robert Helmer [:rhelmer] from comment #1)
My recollection from talking with MattN about this some time ago is that formautofill is implemented as a system addon so that heuristics for individual sites can be rolled out on the fly (a la webcompat). The whole thing was initially implemented as an extension, but at this point, everybody would be happy to turn the core bits into a built-in browser component while maintaining a method to ship improved heuristics out of cycle. I thought there was actually a bug for that, but I can't seem to find it.
Cool.
Another thought is that we don't ever seem to ship out-of-cycle system addon updates and we have other methods (ie Normanday) to ship extensions if necessary, so we could simply migrate our existing system addons to either builtin addons that can't be updated out-of-cytcle but that we don't install during startup or to builtin components. That would essentially give the same benefits as #2.
Yes, I think that in general we want to move towards built-in components that can pull updates (static data from Remote Settings etc), or if necessary expose internal webextension APIs for mozilla-signed extensions to call.
I don't think Normandy is ready to take over hotfix extensions yet (:mythmon can correct me here if that's wrong), but I think with the changes you're proposing we'll still be able to ship hotfixes via balrog/system add-ons until it is ready.
The only built-in system add-ons that have had updates in the last 6 months are:
- fx-monitor
- webcompat
CC:'ing folks so they are aware.
|
||
Updated•5 years ago
|
|
||
Updated•5 years ago
|
|
||
Comment 4•5 years ago
|
||
It appears we can fall into this after an update too:
|
|
Reporter | |
Comment 5•5 years ago
|
||
(In reply to Mike Conley (:mconley) (:⚙️) (Wayyyy behind on needinfos) from comment #4)
It appears we can fall into this after an update too:
See the second sentence of comment 0 :)
|
|
||
Comment 6•5 years ago
|
||
Hi, aswan! :)
I thought it might make sense to combine the first start and upgrade scenarios into a single bug, which is why I updated the summary. Or do you think they're different enough to warrant separate bugs?
|
|
Reporter | |
Comment 7•5 years ago
|
||
Hey! Sorry, my comment was confusing. I think that if we can proceed on the goal of moving system addons to built-in addons that don't need to be synchronously installed/updated during startup, that would address both scenarios. I've been making some slow progress on screenshots, I haven't had a chance to look closely at webcompat and formautofill to figure out if there are serious barriers for doing the same with those. But yes, I think a single bug for both scenarios sounds good and if the current plan turns out to be infeasible, this could always be split up later.
|
||
Comment 9•3 years ago
|
||
As my gecko profile shows we spend about 300ms for each and every first startup of a fresh profile in that method:
https://share.firefox.dev/3dlVh9n
Especially for automation this is a deal-breaker. Both Selenium and Puppeteer sequentially start a lot of new Firefox instances with a fresh profile each time. Compared to Chrome (see startup times as given by former Chrome developers on bug 1618354) our startup time is terrible, and it would be great to get these extra 300ms nuked.
As long as we do not have a solution, is there a workaround possible e.g preventing system add-ons to be installed? We would greatly appreciate a hint for that - even if temporary - to make test systems and users happier.
|
|
Reporter | |
Comment 10•3 years ago
|
||
If you're creating a brand new profile, then any preference-based solutions aren't going to work. I'm pretty certain there are no command-line switches or environment variables that do you what you're asking.
Though I would expect that if you're creating a new profile on every startup and that's slow, addon manager is a relatively small piece of a much larger problem....
In any case, you're probably more likely to get help by needinfoing somebody who actively works in this area (i.e., not me) Unless something changed recently, :mixedpuppy is the module owner for the addons manager.
|
|
||
Comment 11•3 years ago
|
||
Ok, thank you. Shane, could you help with the above question regarding a possible workaround for this problem?
|
||
Comment 12•3 years ago
|
||
Some of this stuff has to run early (e.g. langpacks), but if the improvement would be significant enough to make a dent in automation:
I'd move this to another bug.
We might be able to do something to allow turning it off for automation. Do you know if Cu.isInAutomation is true when running under selenium/puppeteer? If so, you could hack in a check[1] on that to see if those test run fine and the perf improvement is seen, something like this (below) should catch first run and updates and skip the startup overhead for new profiles:
if (aAppChanged) {
if (Cu.isInAutomation) return;
updateReasons.push("appChanged");
}
If that does work for you, I'd probably recommend something involving a pref in combination with isInAutomation added to AddonSettings.jsm, where by default this will continue to run, but you can pref off for sets of tests.
|
|
||
Comment 13•3 years ago
|
||
I actually moved the discussion around automation over to bug 1692875.
|
|
||
Updated•2 years ago
|
|
|
||
Updated•2 years ago
|
|
||
Updated•2 years ago
|
|
||
Updated•2 years ago
|
Description
•