eTLD+1 grouping for permanent storage

RESOLVED WONTFIX

Status

()

P5
normal
RESOLVED WONTFIX
6 years ago
3 months ago

People

(Reporter: janv, Assigned: janv)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

Comment hidden (empty)
(Assignee)

Comment 1

6 years ago
Created attachment 721214 [details] [diff] [review]
backup
Assignee: nobody → Jan.Varga
Status: NEW → ASSIGNED
(Assignee)

Updated

6 years ago
Depends on: 767944
I haven't looked at what policy the current patch implements, but figured I should jolt down some thoughts.

I don't think we should use the same eTLD+1 policy for permanent storage as for temporary storage.

The goal of the eTLD+1 policy for temporary storage is to ensure that a single website doesn't take over your whole temporary storage quota.

However there is no such concern for permanent storage since we put up a prompt. I.e. the user knows that he/she authorized a website to use X MB or storage and it doesn't really how the website uses that. It won't affect any other websites and it won't affect the user more than the user authorized.

Instead I think the more important concern for permanent storage is if something like https://myblog.tumblr.com is authorized 50MB of permanent storage from the user, and then website https://funnycatpictures.tumblr.com uses all of that up.

I think an awesome solution would be if https://myblog.tumblr.com could request 50MB of storage, and when it does so, indicate if that storage should be granted to https://myblog.tumblr.com, or *.tumblr.com or *.myblog.tumblr.com etc. This would be really cool if it was part of the quota API.

In the meantime, I think we should simply grant permanent storage on a per-origin basis. I.e. when https://myblog.tumblr.com starts using permanent storage through indexedDB we'll put up a prompt, and if the user accepts that prompt we'll grant storage to only the https://myblog.tumblr.com origin.

Let me know what you think.
(Assignee)

Updated

6 years ago
No longer blocks: 785884
(Assignee)

Updated

6 years ago
Depends on: 785884
(Assignee)

Updated

6 years ago
No longer depends on: 767944
(Assignee)

Comment 3

6 years ago
Yeah, that completely makes sense.

Comment 4

6 years ago
I sent this mail to Jan Varga in March regarding this topic, I think it fits here now: 


#### snip ####

Hi Jan, 

I was wondering about the Quota limit for (Sub)Domains (in the following, WebApp means any website requiring local storage, app cache, iDB etc.): 

The current implementation (with scope eTLD+1, whatever that means*) allows for (e. g.) 5MB for google.com including docs.google.com and mail.google.com, iirc. If google.com needs 5MB, there's no storage left for *.google.com. That's both good (see recent storage-fill-up attack) and bad (WebApps cannot share one domain). 

* I guess an eTLD+1 is domain.com and domain.co.uk but not s.domain.com? If so, when I say "domain" I mean eTLD+1 and when I say "subdomain" I mean eTLD+2 (or even higher). 

What I'd like to see is: 
1. Each individual domain (google.com) and subdomain (docs.google.com, mail.google.com) has a quota of 5MB.
2. But each domain (group) has a total limit of e. g. 30MB for both the domain (google.com) and subdomains (docs|mail.google.com). This effectively allows for 5 WebApps on different subdomains as well as one on the (root) domain or 6 WebApps on different subdomains.
3. If a WebApp gets higher storage privileges (however this is achieved – e. g. a prompt if it reaches its quota, or installing a WebApp from the Marketplace), its quota is excluded from the group quota and gets its own 30MB (50? 100? In several steps each with a prompt?) quota. Thus docs.google.com can store e. g. 30MB, while mail.google.com and google.com each store 5MB, leaving 20MB for other subdomains. 
4. There should be a watchdog monitoring if the quota is reached on several subdomains (of the same domain) and possibly even across domains in a short timeframe (within tens of seconds, escalating to minutes with an exponential algorithm) to prevent the storage-fill-up attack and any non-optimal behavior of apps spanning across multiple subdomains. 
5. All those mechanisms need to be documented clearly with a high-level overview and maybe more in-depth information, possibly involving PR explaining the "clever" combination of mechanisms when (if) it's released. 


Is any of this implemented, or can be implemented by an add-on (however I'd like to see this as a standard functionality)? Either way, what do you think about my ideas?
(Assignee)

Comment 5

6 years ago
(In reply to Jonas Sicking (:sicking) from comment #2)
> Instead I think the more important concern for permanent storage is if
> something like https://myblog.tumblr.com is authorized 50MB of permanent
> storage from the user, and then website https://funnycatpictures.tumblr.com
> uses all of that up.

hm, https://myblog.tumblr.com and https://funnycatpictures.tumblr.com are separate origins, no ?
so they have separate 50 MB limit too
(Assignee)

Comment 6

6 years ago
Florian, we plan to implement a quota API, but it is a lower priority at the moment.
See https://wiki.mozilla.org/WebAPI/Storage2013
(Assignee)

Comment 7

6 years ago
(In reply to Jan Varga [:janv] from comment #5)
> (In reply to Jonas Sicking (:sicking) from comment #2)
> > Instead I think the more important concern for permanent storage is if
> > something like https://myblog.tumblr.com is authorized 50MB of permanent
> > storage from the user, and then website https://funnycatpictures.tumblr.com
> > uses all of that up.
> 
> hm, https://myblog.tumblr.com and https://funnycatpictures.tumblr.com are
> separate origins, no ?
> so they have separate 50 MB limit too

or you mean that we will change our implementation for permanent storage and give 50 MB for entire eTLD+1 domain, and then we would have the problem you described ?
(In reply to Jan Varga [:janv] from comment #5)
> (In reply to Jonas Sicking (:sicking) from comment #2)
> > Instead I think the more important concern for permanent storage is if
> > something like https://myblog.tumblr.com is authorized 50MB of permanent
> > storage from the user, and then website https://funnycatpictures.tumblr.com
> > uses all of that up.
> 
> hm, https://myblog.tumblr.com and https://funnycatpictures.tumblr.com are
> separate origins, no ?
> so they have separate 50 MB limit too

Yes, they are separate origins, but they are not separate eTLD+1s.

What I'm saying is that I think that when a page on https://myblog.tumblr.com uses permanent storage, we should only grant the origin "https://myblog.tumblr.com" 50MB of storage.

If later, a page from https://funnycatpictures.tumblr.com tries to use permanent storage, we should not let it use space from the 50MB that the first page was granted. Instead we should put a prompt to the user. If the user says "yes", then we should grant the origin "https://funnycatpictures.tumblr.com" 50MB of storage.

Once we have a quota API then it might make sense to enable the page that is asking for permanent storage to define which origins or some such that it wants to share quota with, but that's something we can look at when we do the quota API.

Makes sense?
(Assignee)

Comment 9

6 years ago
Yeah, makes sense, thanks

Comment 10

5 years ago
Jan, what's the current status and plan here, now?
Flags: needinfo?(Jan.Varga)
(Assignee)

Comment 11

5 years ago
Well, we haven't started working on public quota API yet. Quota manager is currently being refactored to work on PBackground and to provide simpler internal API for clients (IDB, asm.js caching, LocalStorage) plugged into it.
Flags: needinfo?(Jan.Varga)

Comment 12

5 years ago
How does this bug depend on a *public* quota API? IMHO, this should be transparent and just work automatically. Waiting for refactored internal APIs is good, though.
(Assignee)

Comment 13

5 years ago
It seems that we will need to implement [1] for this. See requestPersistentQuota() method.

[1] http://www.w3.org/TR/quota-api/

Comment 14

5 years ago
(In reply to Jan Varga [:janv] from comment #13)
> It seems that we will need to implement [1] for this. […]
> 
> [1] http://www.w3.org/TR/quota-api/

Is there a bug on file for this? Haven't found one.
Depends on: 961049, 942542
(Assignee)

Updated

5 years ago
No longer depends on: 942542

Updated

4 months ago
Priority: -- → P5
(Assignee)

Comment 15

3 months ago
Quota API has been deprecated, we implemented the Storage standard instead:
https://storage.spec.whatwg.org/

The spec mentions "schemeless origin group" which is something like eTLD+1, but the method for persisting data always work with given origin.

For now, there are no plans to have persist permissions for entire groups like it was suggested in comment 2.
Status: ASSIGNED → RESOLVED
Last Resolved: 3 months ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.