Full disk warnings are not displayed after clearing data from "Preferences"

VERIFIED FIXED

Status

()

Core
DOM: Quota Manager
P1
normal
VERIFIED FIXED
5 months ago
3 months ago

People

(Reporter: Ioana Crisan, Assigned: shawnjohnjr)

Tracking

(Blocks: 1 bug)

56 Branch
x86_64
Linux
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox57 verified)

Details

(Whiteboard: [storage-v1])

Attachments

(4 attachments)

(Reporter)

Description

5 months ago
Created attachment 8881384 [details]
index.html

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:56.0) Gecko/20100101 Firefox/56.0
Build ID: 20170627093404

Steps to reproduce:
1. Run the "python -m SimpleHTTPServer" command in terminal to create a http server
2. Add the attached HTML file in the folder where the server is running
3. Open the following url: http://localhost:8000
4. Click the "Persist me" button in order to persist the site
5. Click the "Start adding data" button to start storing data via indexeddb
6. Continue to store data until the disk runs out of space(e.g: 96%)

Expected results:
2 notifications should be displayed, one when the firefox-used space < 5GB and one when the firefox-used space >= 5GB

Actual results:
No notification is displayed.
(Assignee)

Comment 1

5 months ago
(In reply to Ioana Crisan from comment #0)
> Created attachment 8881384 [details]
> index.html
> 
> User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:56.0) Gecko/20100101
> Firefox/56.0
> Build ID: 20170627093404
> 
> Steps to reproduce:
> 1. Run the "python -m SimpleHTTPServer" command in terminal to create a http
> server
> 2. Add the attached HTML file in the folder where the server is running
> 3. Open the following url: http://localhost:8000
If this is just http, I wonder that persist() can work properly, you might not be able to get navigator.storage.
Can you confirm this step?

> 4. Click the "Persist me" button in order to persist the site
> 5. Click the "Start adding data" button to start storing data via indexeddb
> 6. Continue to store data until the disk runs out of space(e.g: 96%)
> 
> Expected results:
> 2 notifications should be displayed, one when the firefox-used space < 5GB
> and one when the firefox-used space >= 5GB
> 
> Actual results:
> No notification is displayed.
Flags: needinfo?(icrisan)
(Reporter)

Comment 2

5 months ago
Created attachment 8881725 [details]
firstTry.png
Flags: needinfo?(icrisan)
(Reporter)

Comment 3

5 months ago
Created attachment 8881726 [details]
secondTry.png
(Reporter)

Comment 4

5 months ago
(In reply to Shawn Huang [:shawnjohnjr] from comment #1)

> > Steps to reproduce:
> > 1. Run the "python -m SimpleHTTPServer" command in terminal to create a http
> > server
> > 2. Add the attached HTML file in the folder where the server is running
> > 3. Open the following url: http://localhost:8000
> If this is just http, I wonder that persist() can work properly, you might
> not be able to get navigator.storage.
> Can you confirm this step?


It is possible to access the navigator.storage.persist() with http but only with localhost. You can try this or directly the bmoattachment linked to this issue.
After further investigation I noticed that I can get one of the warnings(when the firefox-used space >= 5GB) but only with a clean profile. Even though I cleared the stored site data from "Preferences" this only works once per profile. Please see the attached screenshots for more details(I run it directly from bmoattachment so it's https secured).
(Assignee)

Comment 5

5 months ago
(In reply to Ioana Crisan from comment #4)
> (In reply to Shawn Huang [:shawnjohnjr] from comment #1)
> 
> > > Steps to reproduce:
> > > 1. Run the "python -m SimpleHTTPServer" command in terminal to create a http
> > > server
> > > 2. Add the attached HTML file in the folder where the server is running
> > > 3. Open the following url: http://localhost:8000
> > If this is just http, I wonder that persist() can work properly, you might
> > not be able to get navigator.storage.
> > Can you confirm this step?
> 
> 
> It is possible to access the navigator.storage.persist() with http but only
> with localhost. You can try this or directly the bmoattachment linked to
> this issue.
> After further investigation I noticed that I can get one of the
> warnings(when the firefox-used space >= 5GB) but only with a clean profile.
> Even though I cleared the stored site data from "Preferences" this only
> works once per profile. Please see the attached screenshots for more
> details(I run it directly from bmoattachment so it's https secured).

I see. I remembered we do have a timer to not disturb users to show notification often (i guess 20 mins). Fischer, can you confirm that?
Flags: needinfo?(fliu)

Comment 6

5 months ago
(In reply to Shawn Huang [:shawnjohnjr] from comment #5)
> (In reply to Ioana Crisan from comment #4)
> > (In reply to Shawn Huang [:shawnjohnjr] from comment #1)
> > 
> > > > Steps to reproduce:
> > > > 1. Run the "python -m SimpleHTTPServer" command in terminal to create a http
> > > > server
> > > > 2. Add the attached HTML file in the folder where the server is running
> > > > 3. Open the following url: http://localhost:8000
> > > If this is just http, I wonder that persist() can work properly, you might
> > > not be able to get navigator.storage.
> > > Can you confirm this step?
> > 
> > 
> > It is possible to access the navigator.storage.persist() with http but only
> > with localhost. You can try this or directly the bmoattachment linked to
> > this issue.
> > After further investigation I noticed that I can get one of the
> > warnings(when the firefox-used space >= 5GB) but only with a clean profile.
> > Even though I cleared the stored site data from "Preferences" this only
> > works once per profile. Please see the attached screenshots for more
> > details(I run it directly from bmoattachment so it's https secured).
> 
> I see. I remembered we do have a timer to not disturb users to show
> notification often (i guess 20 mins). Fischer, can you confirm that?
Yes, in order to not bother users too much, the 2nd warning notification would be allowed after 20 mins of the 1st notification.
There is a pref controlling this time: "browser.storageManager.pressureNotification.minIntervalMS"
So it could be set to 1000 which will lower the warning notification interval to 1 sec.
Flags: needinfo?(fliu)
(Reporter)

Comment 7

5 months ago
I am not able to receive the 2nd warning even though I expect more than 20 minutes from the 1st warning before breaching the global limit again.

Comment 8

5 months ago
(In reply to Ioana Crisan from comment #7)
> I am not able to receive the 2nd warning even though I expect more than 20 minutes from the 1st warning before breaching the global limit again.
Please try to store something again. That should trigger the space-checking mechanism again then trigger the warning notification. Thank you.
(Reporter)

Comment 9

5 months ago
(In reply to Fischer [:Fischer] from comment #8)
> (In reply to Ioana Crisan from comment #7)
> > I am not able to receive the 2nd warning even though I expect more than 20 minutes from the 1st warning before breaching the global limit again.
> Please try to store something again. That should trigger the space-checking
> mechanism again then trigger the warning notification. Thank you.

Yes it indeed re-triggers the notification the second time I try to store something else, using the pref you provided (setting it to 1000ms) and not clearing the data from preferences.

What I was trying to describe is a behavior involving clearing data and then trying to store it again.
So, not only the notification is not triggered, but it also permits the persisted site add data over the global limit, it may even take 100% of the available usage(system) on disk. Please see the linked video where the issue is reproduced: 
https://drive.google.com/open?id=0B_L6mNdlzO5TQndtT0pkcnBIclk

Note: I'm using the BuildID: 20170701181359

Also I'm not receiving The Full Disk warning notification described by this slide from UX specs:
https://mozilla.invisionapp.com/share/4Y87EJO39#/screens/218412145
(Reporter)

Updated

5 months ago
Summary: Full disk warnings are not displayed → Full disk warnings are not displayed after clearing data from "Preferences"

Updated

4 months ago
Assignee: nobody → shuang
Status: NEW → ASSIGNED
Priority: -- → P1
Whiteboard: [storage-v1][triage] → [storage-v1]
(Assignee)

Comment 10

4 months ago
Thank you so much for reporting this bug. I will investigate this week.
(Assignee)

Comment 11

4 months ago
This issue seems to be related to a unknown issue.
(Assignee)

Comment 12

4 months ago
Per discussion with Tom, this seems to be related to QuotaManager known issue.
(Assignee)

Comment 13

4 months ago
(In reply to Shawn Huang [:shawnjohnjr] from comment #11)
> This issue seems to be related to a unknown issue.

s:/unknown/known

Updated

4 months ago
Depends on: 1372116
(Assignee)

Comment 14

4 months ago
Created attachment 8886056 [details] [diff] [review]
Tuning storage limit so we can easily see storage pressure notification
(Assignee)

Comment 15

4 months ago
Bug 1372116 Patch1 and Patch2 fix will be landed first but without landing test at the same time. Please retest again after fix landed in mozilla-central (should be tomorrow).
(Assignee)

Comment 16

4 months ago
Set ni? for Comment 15.
Flags: needinfo?(icrisan)
(Reporter)

Comment 17

4 months ago
After bug #1372116 was fixed the persisted sites don't allow you to store more than 50% from the available disk space, but I still don't receive the warning when the Firefox reaches 90% from the available space.

Please confirm if I'm doing something wrong or this is indeed a bug.
 - My available disk space is 74G so the Firefox can use up to 37G to store data.
 - (90/100 *37G) = 33.3G, so I expect to receive the first warning when the the available disk space is 74G - 33.3G = 40.7G(~=41G)
 - second warning I expect to get when the available space is 37G
Flags: needinfo?(icrisan)

Comment 18

4 months ago
(In reply to Ioana Crisan from comment #17)
Hi Ioana,

>  - (90/100 *37G) = 33.3G, so I expect to receive the first warning when the
> the available disk space is 74G - 33.3G = 40.7G(~=41G)

Could you check the usage used by Firefox through storage.estimate() or preference rather than observing the rest of overall usage? Besides, could you confirm the following things? Thanks!

- Firefox should warn users when it stores data more than 90% (33.3 G) from its space (50% from the available disk space, 37G in your case).
- Firefox checks its space only when the first write/store happen. Once it checks how much space it can use, it won't check the available space again until it is closed and opened in the next time. 

Note: the warning should happen while Firefox store data make its usage greater than 90% form its space. The warning won't happen when you write things on your available space though other applications.
(Reporter)

Comment 19

4 months ago
(In reply to Tom Tung [:tt] from comment #18)

Hi Tom,

> Could you check the usage used by Firefox through storage.estimate() or
> preference rather than observing the rest of overall usage? Besides, could
> you confirm the following things? Thanks!
I checked the usage from Preferences(via storage.estimate() I couldn't since the web page is persisted). 

> - Firefox should warn users when it stores data more than 90% (33.3 G) from
> its space (50% from the available disk space, 37G in your case).
Even though I increased the usage only with Firefox I received the warning only when the usage reached 100% from the allocated disk space for Firefox(in my case 37G)

> - Firefox checks its space only when the first write/store happen. Once it
> checks how much space it can use, it won't check the available space again
> until it is closed and opened in the next time.
From my tests this seems to be like you mentioned. 

> Note: the warning should happen while Firefox store data make its usage
> greater than 90% form its space. The warning won't happen when you write
> things on your available space though other applications.
(Assignee)

Comment 20

4 months ago
I will dig a bit more. Thanks.
(Assignee)

Comment 21

3 months ago
(In reply to Ioana Crisan from comment #19)
> (In reply to Tom Tung [:tt] from comment #18)
> 
> Hi Tom,
> 
> > Could you check the usage used by Firefox through storage.estimate() or
> > preference rather than observing the rest of overall usage? Besides, could
> > you confirm the following things? Thanks!
> I checked the usage from Preferences(via storage.estimate() I couldn't since
> the web page is persisted). 
> 
> > - Firefox should warn users when it stores data more than 90% (33.3 G) from
> > its space (50% from the available disk space, 37G in your case).
> Even though I increased the usage only with Firefox I received the warning
> only when the usage reached 100% from the allocated disk space for
> Firefox(in my case 37G)
I thing to be added.
As I investigated in Bug 1384492 Comment 6. If you tested using pref "browser.storageManager.pressureNotification.testStorageLimitKB" which indirectly set "dom.quotaManager.temporaryStorage.fixedLimit" in gecko, that leads a fixed value instead of using 50% available space. So pressure event only fired once, when the current usage space is full.
(Assignee)

Comment 22

3 months ago
(In reply to Ioana Crisan from comment #19)
> (In reply to Tom Tung [:tt] from comment #18)
> 
> Hi Tom,
> 
> > Could you check the usage used by Firefox through storage.estimate() or
> > preference rather than observing the rest of overall usage? Besides, could
> > you confirm the following things? Thanks!
> I checked the usage from Preferences(via storage.estimate() I couldn't since
> the web page is persisted). 
> 
> > - Firefox should warn users when it stores data more than 90% (33.3 G) from
> > its space (50% from the available disk space, 37G in your case).
> Even though I increased the usage only with Firefox I received the warning
> only when the usage reached 100% from the allocated disk space for
> Firefox(in my case 37G)
> 

One thing to be added.
As I investigated in Bug 1384492 Comment 6. If you tested using pref "browser.storageManager.pressureNotification.testStorageLimitKB" which indirectly set "dom.quotaManager.temporaryStorage.fixedLimit" in gecko, that leads a fixed value instead of using 50% available space. So pressure event only fired once, when the current usage space is full.

Comment 23

3 months ago
(In reply to Tom Tung [:tt] from comment #18)
> (In reply to Ioana Crisan from comment #17)
> - Firefox should warn users when it stores data more than 90% (33.3 G) from
Just discuss this with Shawn, we find out I somehow mix up the UX spec v1 and v1.5/v2. 

We should warn the user only when the disk is full in v1. It means Firefox cannot store any data by evicting the old temporary storage. In this case, it's 37G, since there is only a persistent storage. 

Sorry for the misunderstanding. Warning when usage > 90% is on UX spec v1.5/v2.

Besides, the message should suggest user cleaning their storage since their usage is greater than 5G.

Comment 24

3 months ago
(In reply to Ioana Crisan from comment #19)
> Even though I increased the usage only with Firefox I received the warning
> only when the usage reached 100% from the allocated disk space for
> Firefox(in my case 37G)

So, I think it's reasonable as I said in comment 23.
(Assignee)

Comment 25

3 months ago
(In reply to Ioana Crisan from comment #0)
> Created attachment 8881384 [details]
> index.html
> 
> User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:56.0) Gecko/20100101
> Firefox/56.0
> Build ID: 20170627093404
> Expected results:
> 2 notifications should be displayed, one when the firefox-used space < 5GB
> and one when the firefox-used space >= 5GB
If firefox-used space < 5GB, then warn user to free some disk space.
This is because this usage is small and not the main cause for space issue.
In order to avoid the bad and wrong impression among users that
firefox consumes disk space a lot, indicate users to clean up other disk space.


If firefox-used space >= 5GB, then guide users to about:preferences
to clear some data stored on firefox by websites.

The notification is triggered when the disk is "full" and the difference is that firefox occupied usage.

So I think the expectation is not correct. But can you confirm it again, Fischer, since I'm not the one implemented this.
Flags: needinfo?(fliu)

Comment 26

3 months ago
(In reply to Shawn Huang [:shawnjohnjr] from comment #25)
> (In reply to Ioana Crisan from comment #0)
> > Created attachment 8881384 [details]
> > index.html
> > 
> > User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:56.0) Gecko/20100101
> > Firefox/56.0
> > Build ID: 20170627093404
> > Expected results:
> > 2 notifications should be displayed, one when the firefox-used space < 5GB
> > and one when the firefox-used space >= 5GB
> If firefox-used space < 5GB, then warn user to free some disk space.
> This is because this usage is small and not the main cause for space issue.
> In order to avoid the bad and wrong impression among users that
> firefox consumes disk space a lot, indicate users to clean up other disk
> space.
> 
> 
> If firefox-used space >= 5GB, then guide users to about:preferences
> to clear some data stored on firefox by websites.
> 
Yes, this is why 2 types of warnings, < 5G and >= 5GB. 

> The notification is triggered when the disk is "full" and the difference is
> that firefox occupied usage.
> 
> So I think the expectation is not correct. But can you confirm it again, Fischer, since I'm not the one implemented this.
Also by default, we don't want to bother user too much so within 20 mins only prompt once. If the 1st warning type was the type of < 5G, we would only see 1 warning for this type.
Flags: needinfo?(fliu)
(Assignee)

Comment 27

3 months ago
In summary, I think it's normal to see one notification. If everyone agree the behavior is correct, I will let the reporter decide whether this bug should be closed.
Flags: needinfo?(icrisan)
(Reporter)

Comment 28

3 months ago
(In reply to Shawn Huang [:shawnjohnjr] from comment #21)
> (In reply to Ioana Crisan from comment #19)
> > (In reply to Tom Tung [:tt] from comment #18)
> > 
> > Hi Tom,
> > 
> > > Could you check the usage used by Firefox through storage.estimate() or
> > > preference rather than observing the rest of overall usage? Besides, could
> > > you confirm the following things? Thanks!
> > I checked the usage from Preferences(via storage.estimate() I couldn't since
> > the web page is persisted). 
> > 
> > > - Firefox should warn users when it stores data more than 90% (33.3 G) from
> > > its space (50% from the available disk space, 37G in your case).
> > Even though I increased the usage only with Firefox I received the warning
> > only when the usage reached 100% from the allocated disk space for
> > Firefox(in my case 37G)
> I thing to be added.
> As I investigated in Bug 1384492 Comment 6. If you tested using pref
> "browser.storageManager.pressureNotification.testStorageLimitKB" which
> indirectly set "dom.quotaManager.temporaryStorage.fixedLimit" in gecko, that
> leads a fixed value instead of using 50% available space. So pressure event
> only fired once, when the current usage space is full.

Hi, sorry for the late response but I was on PTO.
I did not test using pref "browser.storageManager.pressureNotification.testStorageLimitKB". The pref I used is "browser.storageManager.pressureNotification.minIntervalMS" to lower the warning notification interval to 1 sec by setting it to 1000.
(Assignee)

Comment 29

3 months ago
(In reply to Ioana Crisan from comment #28)
> (In reply to Shawn Huang [:shawnjohnjr] from comment #21)
> > (In reply to Ioana Crisan from comment #19)
> > > (In reply to Tom Tung [:tt] from comment #18)
> > > 
> > > Hi Tom,
> > > 
> > > > Could you check the usage used by Firefox through storage.estimate() or
> > > > preference rather than observing the rest of overall usage? Besides, could
> > > > you confirm the following things? Thanks!
> > > I checked the usage from Preferences(via storage.estimate() I couldn't since
> > > the web page is persisted). 
> > > 
> > > > - Firefox should warn users when it stores data more than 90% (33.3 G) from
> > > > its space (50% from the available disk space, 37G in your case).
> > > Even though I increased the usage only with Firefox I received the warning
> > > only when the usage reached 100% from the allocated disk space for
> > > Firefox(in my case 37G)
> > I thing to be added.
> > As I investigated in Bug 1384492 Comment 6. If you tested using pref
> > "browser.storageManager.pressureNotification.testStorageLimitKB" which
> > indirectly set "dom.quotaManager.temporaryStorage.fixedLimit" in gecko, that
> > leads a fixed value instead of using 50% available space. So pressure event
> > only fired once, when the current usage space is full.
> 
> Hi, sorry for the late response but I was on PTO.
> I did not test using pref
> "browser.storageManager.pressureNotification.testStorageLimitKB". The pref I
> used is "browser.storageManager.pressureNotification.minIntervalMS" to lower
> the warning notification interval to 1 sec by setting it to 1000.

I think even you did not set "browser.storageManager.pressureNotification.testStorageLimitKB", the notifcation only fires once.
(Reporter)

Comment 30

3 months ago
Hi Francis,

Could you please confirm what Tom said in comment #23? I'm confused because the v1 UX spec displays TBD for both notifications. Please see the link(https://mozilla.invisionapp.com/share/4Y87EJO39#/screens/179635747_1-0_Cover). Thank you!
Flags: needinfo?(icrisan) → needinfo?(frlee)

Comment 31

3 months ago
To be clear, the difference between [1] and [2] is that the content of notifications should be different between the current usage > 5G or <= 5G, when the full disk warning is presenting. Thus, only one notification shows up should be reasonable.

[1] https://mozilla.invisionapp.com/share/4Y87EJO39#/screens/213203147
[2] https://mozilla.invisionapp.com/share/4Y87EJO39#/screens/218412145

Comment 32

3 months ago
Hi Ioana,

Tom is right, in Storage V1, we only deal with V1 spec. some further improvement will be handled in V1.5 or V2.

thank you
Flags: needinfo?(frlee)
(In reply to Ioana Crisan from comment #30)
> Hi Francis,
> 
> Could you please confirm what Tom said in comment #23? I'm confused because
> the v1 UX spec displays TBD for both notifications. Please see the
> link(https://mozilla.invisionapp.com/share/4Y87EJO39#/screens/179635747_1-
> 0_Cover). Thank you!

Sorry for the confusion, the "TBD" was for early stage UX explorations, I just remove them and updated the spec.

Thanks.
(Reporter)

Comment 34

3 months ago
Thank you all for the clarifications!
Status: ASSIGNED → RESOLVED
Last Resolved: 3 months ago
Resolution: --- → FIXED
57.0a1 (2017-08-25)
The full disk warning is now properly displayed after clearing the site data in about:preferences#privacy. 
The problem is I'm seeing different notification's content across platforms:
Ubuntu 14.04: http://imgur.com/a/N0hdT
Win 10, OS X 10.12: http://imgur.com/a/JZxUe
Thoughts?
Flags: needinfo?(shuang)
(Assignee)

Comment 36

3 months ago
(In reply to Paul Silaghi, QA [:pauly] from comment #35)
> 57.0a1 (2017-08-25)
> The full disk warning is now properly displayed after clearing the site data
> in about:preferences#privacy. 
> The problem is I'm seeing different notification's content across platforms:
> Ubuntu 14.04: http://imgur.com/a/N0hdT
> Win 10, OS X 10.12: http://imgur.com/a/JZxUe
> Thoughts?

Because usage is less than 5GB and an another is larger than 5GB, as Comment 25, 31 mentioned.
Flags: needinfo?(shuang)
Thanks, Shawn.
Verified fixed based on comment 36.
Status: RESOLVED → VERIFIED
status-firefox57: --- → verified
You need to log in before you can comment on or make changes to this bug.