Firefox shares LocalStorage quota across all sub-domains of a domain name
Categories
(Core :: Storage: localStorage & sessionStorage, defect, P3)
Tracking
()
People
(Reporter: kaldari, Unassigned)
References
(Depends on 1 open bug, Blocks 1 open bug)
Details
(Whiteboard: [platform-rel-Wikipedia])
![]() |
||
Comment 1•11 years ago
|
||
Updated•11 years ago
|
Comment 3•11 years ago
|
||
Comment 4•11 years ago
|
||
Comment 5•11 years ago
|
||
![]() |
||
Comment 7•11 years ago
|
||
Comment 8•11 years ago
|
||
Comment 9•11 years ago
|
||
Reporter | ||
Comment 10•11 years ago
|
||
Comment 11•11 years ago
|
||
Reporter | ||
Comment 12•9 years ago
|
||
Comment 13•9 years ago
|
||
Comment 14•9 years ago
|
||
Comment 15•9 years ago
|
||
Comment 16•9 years ago
|
||
Updated•7 years ago
|
Assignee | ||
Updated•6 years ago
|
Comment 17•6 years ago
|
||
This is starting to become an issue for other major sites as well, such as Reddit, which has quite a number of subdomains nowadays and all subdomains (new/old/mod/etc.) share the same LocalStorage quota.
Comment 18•6 years ago
|
||
We are going to ship a new LocalStorage implementation in FF 68 or 69 which doesn't have this problem.
See bug 1539835.
Comment 19•6 years ago
•
|
||
(In reply to Jan Varga [:janv] from comment #18)
We are going to ship a new LocalStorage implementation in FF 68 or 69 which doesn't have this problem.
See bug 1539835.
I'm running Aurora/DevEdition 68.0b5 (64-bit) with dom.storage.next_gen
set to true
, though.
Comment 20•6 years ago
|
||
What does that mean ?
Does it work well for you in 68.0b5 ?
You should have 2GB of quota for entire eTDL+1 domain.
Comment 21•6 years ago
•
|
||
(In reply to Jan Varga [:janv] from comment #20)
What does that mean ?
Does it work well for you in 68.0b5 ?
You should have 2GB of quota for entire eTlD+1 domain.
Then it doesn't seem to be working in 68.0b5. LocalStorage on reddit reports being capable of writing a total of 5243043 bytes (±5MB)
Comment 22•6 years ago
|
||
There's still the 5MB limit for each origin. However, if there are multiple subdomains they are no longer limited by the 5MB limit as a group.
You can verify that on http://www.filldisk.com/
Comment 23•6 years ago
|
||
(In reply to Jan Varga [:janv] from comment #22)
There's still the 5MB limit for each origin. However, if there are multiple subdomains they are no longer limited by the 5MB limit as a group.
You can verify that on http://www.filldisk.com/
There seems to be something not entirely right with that though. here is what I saw yesterday running this testscript I made with the same firefox version (68.0b5, though it is possible it has since updated of course) and the same flag enabled.
Comment 24•6 years ago
|
||
LS now shares quota with IndexedDB and DOM cache, so you may have too much data stored using these two APIs.
You can check "Cookies and Site Data" in preferences to see how much space is used by your origin.
You can also use the estimate() method to see how much space is available.
See https://storage.spec.whatwg.org/
Comment 25•6 years ago
|
||
(In reply to Jan Varga [:janv] from comment #24)
LS now shares quota with IndexedDB and DOM cache, so you may have too much data stored using these two APIs.
You can check "Cookies and Site Data" in preferences to see how much space is used by your origin.
You can also use the estimate() method to see how much space is available.
See https://storage.spec.whatwg.org/
Executing
navigator.storage.estimate().then(info => {
console.log(info)
});
gives me
quota: 2147483648
usage: 3073184
checking firefox settings shows me this.
Comment 26•6 years ago
|
||
Then, it's a bug. Are you able to reproduce it, or it happened only once ?
Comment 27•6 years ago
|
||
Pretty consistently, in fact rctgamer3 was having the same issue today which is how I arrived here as he was talking with me about it as I helped him troubleshoot it.
Comment 28•6 years ago
|
||
Do you have time to create a test case for this ?
Or provide steps to reproduce.
Comment 29•6 years ago
|
||
I'll try to write them up, what I meant is that I have had it happen consistently but the steps itself aren't precise as it basically amounts to browsing reddit.com from a variety of domains for a little while. I'll see if I can make it a bit less abstract and more useful.
Updated•6 years ago
|
Comment 30•6 years ago
|
||
Btw, does it happen with private browsing only ?
Comment 31•6 years ago
|
||
Also, does it happen if you set the pref "dom.storage.snapshot_reusing" to false ?
Comment 32•6 years ago
|
||
I have had it happen one time today, unfortunately making reliable reproduction steps is proofing to be more difficult than I thought. I'll continue to try.
(In reply to Jan Varga [:janv] from comment #30)
Btw, does it happen with private browsing only ?
It happens with regular browsing, no private browsing involved.
(In reply to Jan Varga [:janv] from comment #31)
Also, does it happen if you set the pref "dom.storage.snapshot_reusing" to false ?
I'll have to try that out.
Comment 33•6 years ago
|
||
(In reply to Jan Varga [:janv] from comment #31)
Also, does it happen if you set the pref "dom.storage.snapshot_reusing" to false ?
It appears that it doesn't. I have been using this flag at home for a week now and did not experience the issue. My firefox installation at work does not have the flag set I did run into the same issue today under otherwise identical circumstances.
Comment 34•6 years ago
|
||
Ok, that narrows down it a bit. Thank you!
Comment 35•5 years ago
|
||
What is the status of this? It looks like it was enabled in Firefox Beta briefly, but then reverted. Is that correct?
Comment 36•5 years ago
|
||
IIUC, https://bugzilla.mozilla.org/show_bug.cgi?id=1064466#c18 implies this was fixed, then reverted in https://bugzilla.mozilla.org/show_bug.cgi?id=1592136
Do Mozilla engineers have any plans to approach this task again rather sooner than later?
Comment 37•5 years ago
|
||
LocalStorage NextGen (aka LSNG, which mitigates the problem) is still present in the code-base and it's a high priority goal to re-enable it and have that ride the release trains to release. The issue is that LocalStorage NextGen depends on the QuotaManager subsystem that also underpins IndexedDB/Cache API/ServiceWorkers and we encountered high rates of breakage in the wild for a variety of reasons. The breakage has been greatly reduced through a series of fixes and enhancements, but we're being conservative about re-enabling until our telemetry indicates that the problem has been dealt with.
Once LSNG has been released for a few cycles we will also undertake the eviction of old LocalStorage data from origins that are no longer used.
And as an aside about the use-case of caching JS in LocalStorage, we'd advise only using the Cache API or just letting the Firefox HTTP cache store JS, since those mechanisms allow the JS engine to cache the parsed bytecode and won't jank cold pageloads for the site, etc. Obviously, for users whose QuotaManager storage is broken, this won't work, but as those issues are corrected, it will start working. (Also, users can "fix" their installation by going to about:support
at any time and choosing "Refresh Firefox...")
Updated•5 years ago
|
Comment 38•5 years ago
|
||
This is a big problem for our company as every web app we use runs on the same main domain. Even the openshift web "console" fills up the default 5 MB quota, leaving no space for other apps. And we cant say our users to change their FF configuration.
Comment 39•4 years ago
|
||
(In reply to deriathan@gmail.com from comment #38)
This is a big problem for our company as every web app we use runs on the same main domain. Even the openshift web "console" fills up the default 5 MB quota, leaving no space for other apps. And we cant say our users to change their FF configuration.
W have the same problem. For example in addition to main site and domain, we have three subdomains for email, test and jira. So it's a really big annoying bug that all of them grouped as one origin and can't work properly.
Comment 40•3 years ago
•
|
||
This has been resolved as of Firefox 92, https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Releases/92.
(In reply to Jan Varga [:janv] from bug 742822 comment #46)
LSNG has been enabled in bug 1599979, so this can be finally resolved as fixed.
Here are some LSNG features relevant for this bug:
- Quota checks and usage reporting for LS is shared with IDB and Cache API (and will be shared with any future quota clients like OPFS)
- There’s no 5MB for entire eTLD+1 group as in the old implementation, the group limit is now 10GB (given/managed by QM).
- The size of LS data is included in navigator.storage.estimate()
- The logical size of LS data is tracked (this is different from all other quota clients like IDB and Cache API which track the physical size of files on disk)
- LS data is evicted along with IDB and Cache API (the new caching behavior with LRU eviction, no prompts asking for a permission are required)
- Alll data is stored under $PROFILE/storage (under the same directory hierarchy)
Commits:
93a: https://hg.mozilla.org/mozilla-central/rev/e165d93f022ce3ff7a26f700098cfa64430f710e
92.0 beta/stable backport: https://hg.mozilla.org/releases/mozilla-beta/rev/098ba31fd111
Comment 41•3 years ago
|
||
Boldly resolving per Krinkle's previous comment.
Description
•