1457973 - Enable navigator.storage API on Firefox Android (Fennec), enabling navigator.storage() and navigator.persist()

Status: RESOLVED INCOMPLETE

(Firefox for Android Graveyard :: Web Apps (PWAs), defect, P2)

Description

[formatkaka]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0
Build ID: 20180426170554

Steps to reproduce:

As per MDN Docs for Storage Manager API at https://developer.mozilla.org/en-US/docs/Web/API/StorageManager#Browser_compatibility, the Storage API is available and enabled by default after firefox v 57 on mobile. 

I am using firefox nightly v61 / stable v59 on android and both return `navigator.storage` as undefined.

Check out this codepen - https://codepen.io/formatkaka/pen/zWQbvr

Works perfectly for firefox dev edition on laptop.
Also works for chrome/edge on android.


Actual results:

returns undefined


Expected results:

should return StorageManager Object

Comment 1

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

[triage] Core API isn't working: P1. Susheel, this is a platform issue: maybe someone from the desktop side can take a look?

Comment 2

[:sdaswani]

Snorp, can your team take a look? Thanks!

Comment 3

[James Willcox (:snorp) ([e-mail]) (he/him)]

Marion, can your team take a look?

Comment 4

[Marion Daly [:mdaly]]

Andrew: can you give your two cents here?

Comment 5

[Andrew Sutherland [:asuth] (he/him)]

I can reproduce.  It's possible this is related to bug 1458320 which kinda-sorta broke IndexedDB.  The IndexedDB breakage at least has broken the WebIDE remote debugging mechanism documented at https://developer.mozilla.org/en-US/docs/Tools/Remote_Debugging/Debugging_Firefox_for_Android_over_Wifi in my nightly profile, although I also feel like I heard things about WebIDE going away, so I'm a little concerned it may still be broken after addressing that.  In any event, I'll clean up the profile and investigate a bit more.

Comment 6

[Andrew Sutherland [:asuth] (he/him)]

We do this on purpose.  Enabled everywhere but android in bug 1399038.

https://searchfox.org/mozilla-central/rev/8837610b6c999451435695e800f38d4acbc0a644/modules/libpref/init/all.js#5703

// Enable Storage API for all platforms except Android.
#if !defined(MOZ_WIDGET_ANDROID)
pref("dom.storageManager.enabled", true);
#else
pref("dom.storageManager.enabled", false);
#endif

Comment 7

[Andrew Sutherland [:asuth] (he/him)]

While we should almost certainly have a bug for turning this on for Android/Mobile, in the interim the MDN docs should be updated to reflect that the API is not and never was enabled for Fennec.

Comment 8

[formatkaka]

Do I need to file a new bug report to turn this on for Android ?

Comment 9

[formatkaka]

(In reply to Andrew Sutherland [:asuth] from comment #7)
> While we should almost certainly have a bug for turning this on for
> Android/Mobile, in the interim the MDN docs should be updated to reflect
> that the API is not and never was enabled for Fennec.

Hello, I see that you are working on this bug. 
How can I track the current status of this bug ? 
Thanks

Comment 10

[Andrew Sutherland [:asuth] (he/him)]

:snorp, I think historically we've been afraid of filling up mobile storage, which is presumably why the decision in bug 1399038 was made to not turn navigator.storage on for Android.  Unless it was a prompting issue... Is leaving navigator.storage disabled still the right decision?

Comment 11

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to Andrew Sutherland [:asuth] from comment #10)
> :snorp, I think historically we've been afraid of filling up mobile storage,
> which is presumably why the decision in bug 1399038 was made to not turn
> navigator.storage on for Android.  Unless it was a prompting issue... Is
> leaving navigator.storage disabled still the right decision?

We can just set a smaller quota if filling up storage is a concern.

If there is a prompting issue, myself or someone else on the team should be able to help out. If you use nsIPromptService it should Just Work.

Comment 12

[formatkaka]

(In reply to Andrew Sutherland [:asuth] from comment #10)
> :snorp, I think historically we've been afraid of filling up mobile storage,
> which is presumably why the decision in bug 1399038 was made to not turn
> navigator.storage on for Android.  Unless it was a prompting issue... Is
> leaving navigator.storage disabled still the right decision?

How is turning on navigator.storage related to filling up mobile storage? 

As per https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Browser_storage_limits_and_eviction_criteria , Firefox already has eviction criteria in place. So, I reckon, filling up mobile storage shouldn't be a problem

Also, the navigator.storage API is used only for managing persistence permissions and estimating available storage. If I use service worker and cache API, how am I to know if I am nearing eviction limits?

Thanks

Comment 13

[Andrew Sutherland [:asuth] (he/him)]

(In reply to formatkaka from comment #12)
> How is turning on navigator.storage related to filling up mobile storage? 

By that I mean that navigator.storage.persist() causes the eviction mechanism to no longer automatically evict data for a site.  And while Firefox desktop has UI under about:preferences, "Cookies and Site Data", "Manage Data..." which lets a user figure out where their storage is going and which origins are marked persistent, Fennec doesn't have similar UI that I can find which means that the eviction mechanism becomes particularly important.  The only way the user can get rid of the excess data is by manually navigating to the site and bringing up the permissions for the page and removing the persistent permission or using clear private data to clear all their stored data, which is a pretty broad granularity.

That said, I realize your interest is in navigator.storage.estimate().  We could alter things so that persist() simply fails on mobile and allow estimate() to work.

> Also, the navigator.storage API is used only for managing persistence
> permissions and estimating available storage. If I use service worker and
> cache API, how am I to know if I am nearing eviction limits?

To clarify: There's no eviction limit.  There's just the quota limit.  When you hit your origin/group's quota limit, you can't write any more data.  Eviction, per the page, happens based on recency of usage.  It doesn't matter how little data you've used if the user hasn't accessed your site in a long time.

Comment 14

[Andrew Sutherland [:asuth] (he/him)]

Dale, as triage owner, it'd be great if you could re-triage this.  Here's the situation as I understand it:
- navigator.storage is disabled by pref for Fennec.
- Fennec does not have UI to surface per-origin data usage and allow clearing it, so it makes sense for navigator.storage.persist() to be forbidden (see comment 13).
- navigator.storage.estimate() doesn't make as much sense to hide because that potentially helps sites be better about self-moderation of data usage.  However, it's quite possible our quota values are overly generous and might encourage counter-productive behavior like over-caching.  At least on desktop, Nightly claims I can use 2 gigs of data for localhost because of this logic:
  // To avoid one group evicting all the rest, limit the amount any one group
  // can use to 20%. To prevent individual sites from using exorbitant amounts
  // of storage where there is a lot of free space, cap the group limit to 2GB.
  uint64_t x = std::min<uint64_t>(mTemporaryStorageLimit * .20, 2 GB);
- We currently enable navigator.storage as all-or-nothing based on the dom.storageManager.enabled pref.

It seems like there are a few possible paths, these are the ones that make the most sense to me:
1. Wait until we have UI in place in Fennec to manage origin data usage before doing anything.
2. Do enable the pref, but have navigator.storage.persist() always throw until we have the data usage UI.  Also, maybe change the group limit thresholds on Fennec to avoid suggesting to sites that they can use 2 gigs of data?

All of these seem like product decisions, however.

Comment 15

[Dale Harvey (:daleharvey)]

> 1. Wait until we have UI in place in Fennec to manage origin data usage before doing anything.

Especially if exposing this is already on the cards it seem like the ideal solution, however yeh while I am triage owner I am not involved in the mobile team at the moment so I dont know if it is on the cards or can be put on them (I think Andreas was going to switch the triage owner to someone actively working on mobile) sorry I couldnt be more help

Comment 16

[Andrew Sutherland [:asuth] (he/him)]

:snorp, please see comment 14 (and 13), I'm going by the module system here, but it seems like that might be outdated, please redirect as appropriate.

Comment 17

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to Andrew Sutherland [:asuth] from comment #14)
> It seems like there are a few possible paths, these are the ones that make
> the most sense to me:
> 1. Wait until we have UI in place in Fennec to manage origin data usage
> before doing anything.

This is very unlikely to ever happen.

> 2. Do enable the pref, but have navigator.storage.persist() always throw
> until we have the data usage UI.  Also, maybe change the group limit
> thresholds on Fennec to avoid suggesting to sites that they can use 2 gigs
> of data?

This sounds reasonable.

> 
> All of these seem like product decisions, however.

I agree. I choose you, Andreas Bovens!

Comment 18

[Andreas Bovens [:abovens]]

Let's go for option 2 for the time being.

Comment 19

[formatkaka]

(In reply to Andrew Sutherland [:asuth] from comment #14)
> Dale, as triage owner, it'd be great if you could re-triage this.  Here's
> the situation as I understand it:
> - navigator.storage is disabled by pref for Fennec.
> - Fennec does not have UI to surface per-origin data usage and allow
> clearing it, so it makes sense for navigator.storage.persist() to be
> forbidden (see comment 13).
> - navigator.storage.estimate() doesn't make as much sense to hide because
> that potentially helps sites be better about self-moderation of data usage. 
> However, it's quite possible our quota values are overly generous and might
> encourage counter-productive behavior like over-caching.  At least on
> desktop, Nightly claims I can use 2 gigs of data for localhost because of
> this logic:
>   // To avoid one group evicting all the rest, limit the amount any one group
>   // can use to 20%. To prevent individual sites from using exorbitant
> amounts
>   // of storage where there is a lot of free space, cap the group limit to
> 2GB.
>   uint64_t x = std::min<uint64_t>(mTemporaryStorageLimit * .20, 2 GB);
> - We currently enable navigator.storage as all-or-nothing based on the
> dom.storageManager.enabled pref.

What is the value of mTemporaryStorageLimit ? Is it the value of storage space available on mobile device.

When you say group, does it mean a specific domain ?

Comment 20

[Andrew Sutherland [:asuth] (he/him)]

(In reply to formatkaka from comment #19)
> What is the value of mTemporaryStorageLimit ? Is it the value of storage
> space available on mobile device.
> 
> When you say group, does it mean a specific domain ?

mTemporaryStorageLimit is the global limit documented at https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Browser_storage_limits_and_eviction_criteria#Storage_limits which also covers the definition of group.

Comment 21

[Andrew Sutherland [:asuth] (he/him)]

(In reply to Andrew Sutherland [:asuth] from comment #7)
> While we should almost certainly have a bug for turning this on for
> Android/Mobile, in the interim the MDN docs should be updated to reflect
> that the API is not and never was enabled for Fennec.

I've now filed bug 1469876 to track correcting MDN since I misunderstood how dev-doc-needed works.  It gets a bug looked at once it's fixed/closed.

Comment 22

[Firefox Bug Husbandry Bot]

Closing per https://bugzilla.mozilla.org/show_bug.cgi?id=1473195

Contact :susheel if you think this bug should be re-opened

Comment 23

[Marion Daly [:mdaly]]

Non-GeckoView bug, see it's been reopened. is this still a priority?

Comment 24

[:sdaswani]

Putting it on P1 list, we will triage as we burn down.

Comment 25

[Marion Daly [:mdaly]]

(In reply to :sdaswani only needinfo from comment #24)
> Putting it on P1 list, we will triage as we burn down.

Still want us to handle it?

Comment 26

[:sdaswani]

Sorry Marion - I am not sure. NI'ing Andreas to make a decision.

Comment 27

[Andreas Bovens [:abovens]]

I suggested going for option 2, earlier, and that still seems like the best option forward (unless we decide to do nothing at all). Andrew, is fixing this a lot of work for your team? Or should it be handled by the Fennec team?

Snorp, is this something we should somehow implement or otherwise document in GeckoView as well?

Comment 28

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to Andreas Bovens [:abovens] from comment #27)
> I suggested going for option 2, earlier, and that still seems like the best
> option forward (unless we decide to do nothing at all). Andrew, is fixing
> this a lot of work for your team? Or should it be handled by the Fennec team?
> 
> Snorp, is this something we should somehow implement or otherwise document
> in GeckoView as well?

For GV, I think we'd want to expose some kind of interface for querying and clearing the storage used on a per-site basis.

Comment 29

[Andrew Sutherland [:asuth] (he/him)]

As a different Andrew involved in these things, option 2 as proposed in comment 14 isn't that much work, it's just that the risk/reward ratio is bad for the change until https://wiki.mozilla.org/DOM/Workers-Storage/projects/WSIF is landed and happy and we have more telemetry, and/or until Fennec/GeckoView has the resources to help focus on the PWA aspects and ensure that Fennec/GV aren't filling up the user's disk, etc.  We are planning to have better QuotaManager telemetry in the next quarter.  And once WSIF is done, we will have more bandwidth/ability to deftly pivot on such issues.

(Not that the change is particularly risky.  It's just that if we expose an API and it shifts the behavior of a mobile website to use more data and it turns out that interacts badly with the 2gig max limit and other sites or other apps on the device... that would be bad.  It's easier to make the change to justify the change when we have more eyes on things, more data, and are targeting for specific PWA-style use-case support, like making sure gmail works reliably offline or some other specific rich PWA we can focus on as a driving use-case.)

Comment 30

[Tom Tung [:tt, :ttung]]

(In reply to Andreas Bovens [:abovens] from comment #27)
> I suggested going for option 2, earlier, and that still seems like the best
> option forward (unless we decide to do nothing at all). Andrew, is fixing
> this a lot of work for your team? Or should it be handled by the Fennec team?

I am also involving in Storage API. As :asuth said, it's not too much work. I don't think that needs to take much time. Finding a suitable global limit for Fennec might take time, but if we don't need to do that, it should even take less time.

Comment 31

[Chris Peterson [:cpeterson]]

Downgrading priority from P1 to P2

Comment 32

[BMO Automation]

We have completed our launch of our new Firefox on Android. The development of the new versions use GitHub for issue tracking. If the bug report still reproduces in a current version of [Firefox on Android nightly](https://play.google.com/store/apps/details?id=org.mozilla.fenix) an issue can be reported at the [Fenix GitHub project](https://github.com/mozilla-mobile/fenix/). If you want to discuss your report please use [Mozilla's chat](https://wiki.mozilla.org/Matrix#Connect_to_Matrix) server https://chat.mozilla.org and join the [#fenix](https://chat.mozilla.org/#/room/#fenix:mozilla.org) channel.