Progressively limit preferences to 4kB per field

NEW
Unassigned

Status

()

Core
Preferences: Backend
4 years ago
3 years ago

People

(Reporter: Yoric, Unassigned)

Tracking

(Depends on: 2 bugs, {addon-compat})

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [Snappy])

According to a discussion on dev-platform, numerous add-ons seem to abuse preferences to store non-trivial data. Given the architecture of our preference service, this is costly and we want to avoid this.

The objective of this bug is to get rid of such behaviors.
Depends on: 872981
Keywords: addon-compat

Comment 1

4 years ago
As a note, the initial patch to do a limit at all (which was set to 1MB for now) was done in bug 836263.

Comment 2

4 years ago
The warning in bug 872981 was set to 4kb, so I'm adjusting the summary here for the same limit.
Summary: Progressively limit preferences to 16kb per field → Progressively limit preferences to 4kb per field
Please note bug 915838, which aims to give add-on developers an alternate to Mozilla's preferences.
Some preferences, not necessarily created by extensions, tend to be very long: for instance in my profile, lightweightThemes.usedThemes (the metadata for all my lightweight themes, IIUC) is 20029 characters long. My prefs.js is 233530 bytes long, but AFAICT any slowness of reaction is due more to my high number of tabs (101 at the moment) or of extensions (74, not counting the disabled ones) than to use (or, some would say, "overuse" or "abuse") of the preferences subsystem.

So please do not make those 4 KiB a hard limit (also not the 32 KiB for the whole preferences file, which I saw elsewhere in a supporting comment about this bug).
(In reply to Tony Mechelynck [:tonymec] from comment #4)
> Some preferences, not necessarily created by extensions, tend to be very
> long: for instance in my profile, lightweightThemes.usedThemes (the metadata
> for all my lightweight themes, IIUC) is 20029 characters long.

Well, that's an interesting piece of information, and it shows that we should redesign this bit of lightweight themes. I have just filed this as bug 919459.

> My prefs.js
> is 233530 bytes long, but AFAICT any slowness of reaction is due more to my
> high number of tabs (101 at the moment) or of extensions (74, not counting
> the disabled ones) than to use (or, some would say, "overuse" or "abuse") of
> the preferences subsystem.

[over]use of preferences can slow down the system in subtle and unexpected manners, especially during startup and shutdown. But yes, if you have 74 addons, you can expect all sorts of other subtle and unexpected slowdowns :)

> So please do not make those 4 KiB a hard limit (also not the 32 KiB for the
> whole preferences file, which I saw elsewhere in a supporting comment about
> this bug).

Well, there is already a 4kb soft limit (bug 872981). We would like to eventually turn this into a 4kb hard limit, eventually, but of course, we are going to wait until we have a convenient replacement for this.

Comment 6

4 years ago
(In reply to Tony Mechelynck [:tonymec] from comment #4)
> My prefs.js
> is 233530 bytes long, but AFAICT any slowness of reaction is due more to my
> high number of tabs (101 at the moment) or of extensions (74, not counting
> the disabled ones) than to use (or, some would say, "overuse" or "abuse") of
> the preferences subsystem.

Slow reaction is mostly other things than prefs (we have moved a lot of things in Firefox to be done async and in the background, but more needs to be done - and AFAIK Thunderbird and SeaMonkey have barely done anything in that area). The main things that's becoming slow with large prefs.js is startup, as we need to read this file synchronously early in the process, given that some things we do within startup can probably be influenced by prefs.
Depends on: 919459

Updated

3 years ago
Summary: Progressively limit preferences to 4kb per field → Progressively limit preferences to 4kB per field

Comment 7

3 years ago
Has telemetry been collected on how common large prefs are? (and how big they can get) It'd be really nice to have some real numbers on how bad a problem this is. (seeing as Firefox itself uses JSON blobs in prefs, plenty of addons have followed the leader on this topic) My guess is that it would be a really good idea to shrink the hard limit in steps, maybe first going down to 500KiB then only after that bakes in a release or two go down to 100KiB before eventually thinking about more ideal limits. Go down too fast and you'd risk dataloss in addons doing this before their developers notice. In all likelihood, there are some addons producing massive prefs due to extreme use-cases not expected by their developers, so it would be nice if there was a way to gather some stats on major cases to give to developers.

By the way, I do know the warning does work to get addon developers to reduce their pref size, seeing as it did work on me. ;)
Filed as Bug 1011416.
Depends on: 1011416
You need to log in before you can comment on or make changes to this bug.