Very bad memory efficiency of browser.storage.local

NEW
Unassigned

Status

()

Toolkit
WebExtensions: General
3 months ago
a month ago

People

(Reporter: Wladimir Palant, Unassigned)

Tracking

(Blocks: 2 bugs)

Trunk
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

3 months ago
I just had a remote debugging session with a user who complains about Adblock Plus 2.9.1 being completely broken for them. Turns out, this user misconfigured Adblock Plus by adding way too many filters. However, while Adblock Plus 2.8.2 (using direct file access) wasn't dealing too badly with a single file containing 34 MB of data, Adblock Plus 2.9.1 (using browser.extension.storage) errors out or crashes with an "out of memory" error when saving data. This being a 32-bit Firefox build (Firefox 52 ESR and Firefox 55 Nightly on Windows) on a machine having plenty of swap memory, my conclusion is that there is a memory spike going beyond 3 GB. At the same time, as long as the data merely stays in memory the memory usage stays below 400 MB (that's the total memory usage of the Firefox process, not merely Adblock Plus).

Note on the data structure: Adblock Plus uses browser.extension.storage as a replacement for plain files, so the data is a plain array containing all the lines that would have been written to the file.

Comment 1

2 months ago
Let's re-investigate this once bug 1371255 is done.

Updated

2 months ago
Flags: needinfo?(amckay)

Comment 2

a month ago
Created attachment 8888097 [details]
storage-local-set.PNG

Crap comment 1 wasn't helpful was it, sorry about that. 

I'd be curious to see how this is now that bug 1370752 has landed. That's significantly improved the time spent on storage.local.set. Telemetry is at https://mzl.la/2tglhcl, but I've attached an image of it this work.
Flags: needinfo?(amckay)
You need to log in before you can comment on or make changes to this bug.