Closed Bug 847913 Opened 9 years ago Closed 3 years ago

eTLD+1 grouping for permanent storage

Categories

(Core :: Storage: IndexedDB, defect, P5)

defect

Tracking

()

RESOLVED WONTFIX

People

(Reporter: janv, Assigned: janv)

References

Details

Attachments

(1 file)

No description provided.
Attached patch backupSplinter Review
Assignee: nobody → Jan.Varga
Status: NEW → ASSIGNED
Depends on: 767944
I haven't looked at what policy the current patch implements, but figured I should jolt down some thoughts.

I don't think we should use the same eTLD+1 policy for permanent storage as for temporary storage.

The goal of the eTLD+1 policy for temporary storage is to ensure that a single website doesn't take over your whole temporary storage quota.

However there is no such concern for permanent storage since we put up a prompt. I.e. the user knows that he/she authorized a website to use X MB or storage and it doesn't really how the website uses that. It won't affect any other websites and it won't affect the user more than the user authorized.

Instead I think the more important concern for permanent storage is if something like https://myblog.tumblr.com is authorized 50MB of permanent storage from the user, and then website https://funnycatpictures.tumblr.com uses all of that up.

I think an awesome solution would be if https://myblog.tumblr.com could request 50MB of storage, and when it does so, indicate if that storage should be granted to https://myblog.tumblr.com, or *.tumblr.com or *.myblog.tumblr.com etc. This would be really cool if it was part of the quota API.

In the meantime, I think we should simply grant permanent storage on a per-origin basis. I.e. when https://myblog.tumblr.com starts using permanent storage through indexedDB we'll put up a prompt, and if the user accepts that prompt we'll grant storage to only the https://myblog.tumblr.com origin.

Let me know what you think.
No longer blocks: 785884
Depends on: 785884
No longer depends on: 767944
Yeah, that completely makes sense.
I sent this mail to Jan Varga in March regarding this topic, I think it fits here now: 


#### snip ####

Hi Jan, 

I was wondering about the Quota limit for (Sub)Domains (in the following, WebApp means any website requiring local storage, app cache, iDB etc.): 

The current implementation (with scope eTLD+1, whatever that means*) allows for (e. g.) 5MB for google.com including docs.google.com and mail.google.com, iirc. If google.com needs 5MB, there's no storage left for *.google.com. That's both good (see recent storage-fill-up attack) and bad (WebApps cannot share one domain). 

* I guess an eTLD+1 is domain.com and domain.co.uk but not s.domain.com? If so, when I say "domain" I mean eTLD+1 and when I say "subdomain" I mean eTLD+2 (or even higher). 

What I'd like to see is: 
1. Each individual domain (google.com) and subdomain (docs.google.com, mail.google.com) has a quota of 5MB.
2. But each domain (group) has a total limit of e. g. 30MB for both the domain (google.com) and subdomains (docs|mail.google.com). This effectively allows for 5 WebApps on different subdomains as well as one on the (root) domain or 6 WebApps on different subdomains.
3. If a WebApp gets higher storage privileges (however this is achieved – e. g. a prompt if it reaches its quota, or installing a WebApp from the Marketplace), its quota is excluded from the group quota and gets its own 30MB (50? 100? In several steps each with a prompt?) quota. Thus docs.google.com can store e. g. 30MB, while mail.google.com and google.com each store 5MB, leaving 20MB for other subdomains. 
4. There should be a watchdog monitoring if the quota is reached on several subdomains (of the same domain) and possibly even across domains in a short timeframe (within tens of seconds, escalating to minutes with an exponential algorithm) to prevent the storage-fill-up attack and any non-optimal behavior of apps spanning across multiple subdomains. 
5. All those mechanisms need to be documented clearly with a high-level overview and maybe more in-depth information, possibly involving PR explaining the "clever" combination of mechanisms when (if) it's released. 


Is any of this implemented, or can be implemented by an add-on (however I'd like to see this as a standard functionality)? Either way, what do you think about my ideas?
(In reply to Jonas Sicking (:sicking) from comment #2)
> Instead I think the more important concern for permanent storage is if
> something like https://myblog.tumblr.com is authorized 50MB of permanent
> storage from the user, and then website https://funnycatpictures.tumblr.com
> uses all of that up.

hm, https://myblog.tumblr.com and https://funnycatpictures.tumblr.com are separate origins, no ?
so they have separate 50 MB limit too
Florian, we plan to implement a quota API, but it is a lower priority at the moment.
See https://wiki.mozilla.org/WebAPI/Storage2013
(In reply to Jan Varga [:janv] from comment #5)
> (In reply to Jonas Sicking (:sicking) from comment #2)
> > Instead I think the more important concern for permanent storage is if
> > something like https://myblog.tumblr.com is authorized 50MB of permanent
> > storage from the user, and then website https://funnycatpictures.tumblr.com
> > uses all of that up.
> 
> hm, https://myblog.tumblr.com and https://funnycatpictures.tumblr.com are
> separate origins, no ?
> so they have separate 50 MB limit too

or you mean that we will change our implementation for permanent storage and give 50 MB for entire eTLD+1 domain, and then we would have the problem you described ?
(In reply to Jan Varga [:janv] from comment #5)
> (In reply to Jonas Sicking (:sicking) from comment #2)
> > Instead I think the more important concern for permanent storage is if
> > something like https://myblog.tumblr.com is authorized 50MB of permanent
> > storage from the user, and then website https://funnycatpictures.tumblr.com
> > uses all of that up.
> 
> hm, https://myblog.tumblr.com and https://funnycatpictures.tumblr.com are
> separate origins, no ?
> so they have separate 50 MB limit too

Yes, they are separate origins, but they are not separate eTLD+1s.

What I'm saying is that I think that when a page on https://myblog.tumblr.com uses permanent storage, we should only grant the origin "https://myblog.tumblr.com" 50MB of storage.

If later, a page from https://funnycatpictures.tumblr.com tries to use permanent storage, we should not let it use space from the 50MB that the first page was granted. Instead we should put a prompt to the user. If the user says "yes", then we should grant the origin "https://funnycatpictures.tumblr.com" 50MB of storage.

Once we have a quota API then it might make sense to enable the page that is asking for permanent storage to define which origins or some such that it wants to share quota with, but that's something we can look at when we do the quota API.

Makes sense?
Yeah, makes sense, thanks
Jan, what's the current status and plan here, now?
Flags: needinfo?(Jan.Varga)
Well, we haven't started working on public quota API yet. Quota manager is currently being refactored to work on PBackground and to provide simpler internal API for clients (IDB, asm.js caching, LocalStorage) plugged into it.
Flags: needinfo?(Jan.Varga)
How does this bug depend on a *public* quota API? IMHO, this should be transparent and just work automatically. Waiting for refactored internal APIs is good, though.
It seems that we will need to implement [1] for this. See requestPersistentQuota() method.

[1] http://www.w3.org/TR/quota-api/
(In reply to Jan Varga [:janv] from comment #13)
> It seems that we will need to implement [1] for this. […]
> 
> [1] http://www.w3.org/TR/quota-api/

Is there a bug on file for this? Haven't found one.
Depends on: 961049, 942542
No longer depends on: 942542
Priority: -- → P5
Quota API has been deprecated, we implemented the Storage standard instead:
https://storage.spec.whatwg.org/

The spec mentions "schemeless origin group" which is something like eTLD+1, but the method for persisting data always work with given origin.

For now, there are no plans to have persist permissions for entire groups like it was suggested in comment 2.
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.