Deleting simple-storage data freezes ui

RESOLVED INVALID

Status

P2
normal
RESOLVED INVALID
6 years ago
5 years ago

People

(Reporter: dethe, Assigned: jsantell)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

6 years ago
Using SDK version 1.13.2 and master, when I receive an OverQuota event from simple-storage, I delete old data (kept in array, removed using shift()) until storage is down to 90%. During this time, Firefox shows the spinning beachball and appears to have crashed. Once the data is cleared, it resumes working normally.

There are several things I can (and will) do to improve this. I can use splice instead of shift(). I may be able to move the operation to a worker thread (no idea if workers can access the same simple-storage, but worth a try).

The main bug is that browser shouldn't become unresponsive from a single add-on, and that if there were tools for managing simple-storage asynchronously I could easily avoid this.
this sounds important

Updated

6 years ago
Depends on: 781486

Updated

6 years ago
Priority: -- → P2
(Assignee)

Updated

6 years ago
Assignee: nobody → jsantell
Dethe, are you just looping through, removing elements from the array until it's under quota? The simple storage saves every 5 minutes, or something else if the user changes their config, so this may just be synchronous code hanging the thread, regardless of SS writing to disk.

Since this data is just discarded from the front of the array, could you start discarding it slower, earlier? Depending on the size of the array, and array methods being synchronous without callbacks, even using setTimeout could cause backup in your thread.
Flags: needinfo?(dethe)
(Reporter)

Comment 3

6 years ago
Sorry, thought I'd answered this awhile back. Maybe I only replied by email.

I was initially looping through the array. I now figure out what 10% of the number of entries are and Array.splice() them out of the array in one go, which is faster and reduces how often the quota is exceeded. There is still a delay, but it is much less of a problem now. If there is a way to move this off the main thread, I'd be happy to test that out.
Flags: needinfo?(dethe)
Dethe, unloading to a web worker would still involve serializing the large storage you have currently stored in memory, do operations there, and have to reserialize it back. I don't think removing large chunks of data synchronously is a good strategy for staying under quota (if it's being removed, is it arbitrary? on every new data entry, maybe remove an oldest entry?) -- also, check out the indexed-db storage solution, it's a bit more robust and can handle data manipulation better:
https://addons.mozilla.org/en-US/developers/docs/sdk/latest/modules/sdk/indexed-db.html
Is this still an issue?
Flags: needinfo?(dtownsend+bugmail)
(Reporter)

Comment 6

5 years ago
Not for Collusion, we've implemented a variety of workarounds and have much less dependence on simpleStorage.
Flags: needinfo?(dtownsend+bugmail)
Awesome, good to hear. Running that many consecutive sync operations on anything will hold up anything. Let me know if you run to any other issues!
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.