1515682 - Crashing Nightly while it is running out of storage space will not display the correct value of saved Cookies and Site Data

Status: NEW

(Core :: Storage: localStorage & sessionStorage, defect, P3)

Description

[Timea Cernea [:tbabos][inactive]]

[Affected versions]
66.0a1 (2018-12-20)

[Affected platforms]
Windows 10 x64 (only tested on this so far)

[Steps to reproduce]
1. Launch Firefox
2. Open 3-4 tabs with sites like youtube, facebook, reddit etc.
3. Go to https://bug1377104.bmoattachments.org/attachment.cgi?id=8882153 in a different tab
4. Click on Persist Me and start adding data until the yellow bar notification appears (Nightly is running out of disk space...)
5. Check the value of saved Cookies and Site Data (from Clear Data list) on the about:preferences#privacy
6. Crash Nightly with about:crashparent
7. Restart Nightly and restore/reload all the previously opened pages
8. Check again the saved Cookies and Site Data

Expected result]
The saved Cookies and Site data amount should be the same as before crashing Nightly

[Actual result]
The are no Saved Cookies and Site Data ( 0 bytes ). 

Recording of the issue: https://streamable.com/m1na5

Browser Console log: https://imgur.com/lYaNFG9

[Notes]:
It can be reproduced with NextGen local storage enabled and disabled as well.
Restarting the browser solves the issue as the correct value will be displayed. 

Further browsing sites after reproducing this issue might cause twitch live stream to not be displayed (black screen)and other page loading issues. <- I had like 20 tabs opened at this point, this part still needs to be investigated.

Comment 1

[Andrew Overholt [:overholt]]

Thanks for the bug report, Timea!

Andrew Sutherland kindly pointed out to me that the cookie stuff is bug 1237527.

Jan, can you think about whether or not this is particularly relevant to next-gen localStorage (i.e. if we have to block on this)?

Comment 2

[Jan Varga [:janv]]

(In reply to Andrew Overholt [:overholt] (Back Jan 2) from comment #1)
> Jan, can you think about whether or not this is particularly relevant to
> next-gen localStorage (i.e. if we have to block on this)?

If it can be reproduced with LSNG disabled, then I don't think we need to block on this.
Anyway, cookies are stored separately, LSNG doesn't interact with it at all.

Comment 3

[Timea Cernea [:tbabos][inactive]]

Thanks Jan and Andrew for reviewing this issue.
I'm still quite confused due to the "Cookies and Site Data" label given that Cookies and Site Data are being placed together in the Clear Data UI. 

Since I stored persistent storage (1.2 GB) which goes to local storage (Site Data?) and all of it "disappeared" after crashing Nighlty, made me totally forgot that there are also cookies in that label. 

Please note that in my case even if cookies and site data look like being deleted, I am still logged in on Facebook and Twitch but Reddit page doesn't want to load anymore at all(even after several restarts!), here is a screenshot of the browser console: https://imgur.com/YlyTbhr
In this case, I am not sure if this is indeed the mentioned Cookies bug or some alteration of it. 

In any case it does affect page rendering (reddit) and I don't understand why deleted Cookies bug affects the displayed value for the stored Site Data given that they are stored separately. I would expect to see for example 200 KB less (as for the deleted cookies) and the 1.2 GB persistent storage to still be there. 

I know there are a lot of questions here...but the Reddit page not being rendered (unless I delete all Cookies and Site Data) is not a good sign. Note that Reddit was working just fine while I had full storage space reached and without starting to crash Nightly.

Comment 4

[Andrew Sutherland [:asuth] (he/him)]

For the erroneous data usage, on about:preferences, it looks like the amount of data used by cookies isn't actually reflected.  On the Privacy & Data page, the total is `siteDataUsage + cacheUsage`.  Inside the dialog, those two numbers are split out.

`siteDataUsage` comes exclusively from QuotaManager and in the event of an error[2] 0 will be returned.  It seems like this is the most likely explanation for why you saw zero.  (The control flow in SiteDataManager.jsm is a little suspect, but if the QM getUsage call wasn't made, I think we'd expect the dialog to fail to initialize rather than show a 0.)

Based on the recording, it appears that when you first brought up the browser after triggering the crash that QuotaManager came up in a broken state, but that it recovered after that.  Very interestingly, it doesn't look like QuotaManager actually performed any eviction after you used up that disk space or after the restarts.  I wonder if that means the problem might instead be related to crashing the parent process rather than the disk usage.  (That is, perhaps there is an edge case due to the IndexedDB database not being normally closed/checkpointed or something like that?)

1: https://searchfox.org/mozilla-central/rev/9528360768d81b1fc84258b5fb3601b5d4f40076/browser/components/preferences/in-content/privacy.js#1040
2: https://searchfox.org/mozilla-central/rev/9528360768d81b1fc84258b5fb3601b5d4f40076/browser/modules/SiteDataManager.jsm#136

Comment 5

[Timea Cernea [:tbabos][inactive]]

I somehow managed to reproduce this without crashing the browser. It occurs randomly and I couldn't figure it out what really triggers this behavior. (on Windows 7)
If it is of any help, I uploaded the profile on which it can be reproduced easily to the mozilla drive (https://drive.google.com/drive/u/1/folders/1FnrSDJc6ZpfVDgezqyTerNyhqynI528l).
It's a profile with full allowed storage reached (simulated via dom.quotaManager.temporaryStorage.fixedLimit) and with a bunch of tabs opened.

STR:

1. Launch Nightly with this profile
2. Check the Cookies and Site Data value -> it is around 12.4 GB
3. Reload all tabs
4. Check again the Cookies and Site data value -> It will be around 750 MB or so

It can be reproduced on Firefox Release as well using the same profile.
Given that its not actually a NextGen local storage issue, but a more weird edge case as mentioned by Andrew, is the engineering team taking into account to debug/fix this as part of enabling LSNG or not?

Comment 6

[Andrew Sutherland [:asuth] (he/him)]

I don't think this will block LSNG landing. But this does fall under the "War on Storage Initialization Failures" umbrella which is a high priority for us as well. Setting this bug to block the meta-bug bug 1482662.

Can you elaborate on what specific series of actions you are talking about for step 3 "Reload all tabs"? Do you mean doing "Select all tabs" and then "Reload Tabs", or are you referring to the extension of the same name? (Or is that menu options still available on release?)

Comment 7

[Timea Cernea [:tbabos][inactive]]

Its the classic right click on a tab -> select all tabs -> right click again -> reload all tabs. Same steps applies on Firefox Release version as on Nightly.
I was able to reproduce this on Windows 7 specifically with that profile. Truth is I enabled/disabled LSNG a couple of times on it so I am not actually sure if it is a "generic" firefox issue or LSNG has something to do with it.

Comment 8

[Jari Jalkanen [:jari]]

This is reproducible at least when starting from a fresh, clean profile.

Comment 9

[Andrew Sutherland [:asuth] (he/him)]

This is the call to nsIQuotaManagerService.getUsage in the UI for population. But an interesting complication here is that, at least at the time the bug was filed, it's quite likely that QuotaManager would almost certainly have wiped out tons of data stored in QuotaManager because we would be experiencing dramatic storage pressure. So it would not be surprising for the amount of storage space reported to decrease massively. The only exceptions would be persisted origins, but note that site usage data is reported based on the site (eTLD+1) but persistence is only tracked on a per-origin basis, although the UI will report any site that has at least one persisted origin in it as a persisted site. (This is in the SiteDataManager.sys.mjs logic.) It's also possible that if the disk was sufficiently full, it might not even be possible for QuotaManager to initialize or properly clear data, which could have also caused 0 values and/or a failure to even respond to the callback which could then still also look a lot like a 0 value report from QuotaManager.