1755206 - wordle game not working for some users after move to nytimes (page remains empty/white/blank); console: Uncaught DOMException: The quota has been exceeded.

Status: RESOLVED FIXED

(Core :: Storage: localStorage & sessionStorage, defect, P2)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

As reported by several users - e.g. :bz - the Wordle game page doesn't load any content for them and the web console reports: "Uncaught DOMException: The quota has been exceeded."

I have no older profile with previous Wordle usage at hand to reproduce the issue, attempts with a new profile work as expected here.

Edit: Wordle moved very recently to its new home at https://www.nytimes.com/games/wordle/index.html - it had been hosted at https://www.powerlanguage.co.uk/wordle/ before.

Comment 1

[Andrew Sutherland [:asuth] (he/him)]

Can you copy and paste the following into devtools (for the resulting nytimes page) and see what the LocalStorage usage gets reported as?

lsUsage = 0 ; for (let iKey = 0; iKey < localStorage.length; iKey++) { const key = localStorage.key(iKey); lsUsage += key.length + localStorage[key].length; } ; console.log("LS Usage", lsUsage);

I locally see a value of 1508705 and the devtools storage tab shows a bunch of usages like cached CSS font-face directives including data URIs so it seems possible others might have additional cached data. Also, there were reports that the initial data migration seemed to naively propagate the contents of LocalStorage from the old site to the new site, so it's possible more might have been propagated than intended?

Comment 2

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Boris, could you provide the data requested in comment 1, please?

Comment 3

[Boris Zbarsky [:bzbarsky]]

The above command line gives me 5242490

Note that I deleted some optimizely localstorage crud for that site via the "Storage" tab in devtools, and then Wordle worked fine. So the above value is after I did that and Wordle is working.

Comment 4

[Andrew Sutherland [:asuth] (he/him)]

Thank you! So you were definitely at/near the LocalStorage limit (as opposed to this being highly impacted by multi-process snapshot quota pre-allocations which is something being fixed actively in another bug).

This[1] seems like a general indication that probably need to think about mitigations for LocalStorage breakage like we've done for ServiceWorkers (as motivated by slack breakage). That is, if we throw a LocalStorage quota limit exception and it ends up propagating without being caught and we run a heuristic check that suggests the page might be broken and/or there's no indication that the site is otherwise thinking about quota concerns, we should think about expiring some or all of LocalStorage. We did add a last_access_time column[2] in our schema, but we haven't hooked it up to anything yet.

By spec, the granularity would be "clear the whole origin", but in practice an intervention that clears old data that has not been read or written in a long time seems reasonable. And it would be reasonable to treat running code like I provided in comment 1 as the site seeming to take responsibility for clearing its data. The one thing would be to make sure devtools doesn't accidentally impact this data. Presumably we can check the principal performing access and if it's an expanded principal (for web extensions) or a system principal (for devtools), we can disable the checks having an impact.

1: Breakage of a significant site that primarily uses a single origin and that has a lot of engineering behind it.

2: Note that last_access_time was introduced for determining whether we can be more clever about our initial reads of the LocalStorage database and/or what we pre-emptively send down to the content processes in a snapshot. If we can just read 64K from disk in a first pass instead of 5M in order to reduce the initial blocking phase, that lets us help reduce the amount of time the content process has to block. That such a field could be used for partial data-clearing was something we were aware of, but it's definitely not clear that clearing at a granularity less than the origin is appropriate.

Comment 5

[Kevin Brosnan [Ex-Mozilla]]

I have a family member with this problem.

LS Usage 5242875

Comment 6

[Randell Jesup [:jesup] (needinfo me)]

Given the ideas from asuth in comment 4, how easy would it be to introduce some type of mitigation for this? Having a very popular site like wordle break is bad, especially if general users don't have any obvious way to resolve it. If their storage is simply accumulating, it sounds like most users of the site will eventually hit this and be stuck.

Even the most simplistic (clear everything) sounds like it would be better than doing nothing. (I presume this causes no issues with the functionality of the site, correct? bz?)

Comment 7

[Andrew Sutherland [:asuth] (he/him)]

The mitigation I was thinking of is basically:

• If we throw a quota limit hit for a given LocalStorage origin, start a timer.
• If the usage hasn't decreased when the timer expires, clear the LocalStorage for the origin.
  • In the future, we could try and partially clear LocalStorage if we start tracking when specific keys/values are read/written, but we don't have that data now. This would be a larger effort and we also would need to really think through the privacy implications of this. Specifically, for privacy, one might be able to infer specific URL visits, so we wouldn't want to capture this data in cases where we know that history is disabled/cleared at shutdown. (In theory the history clearing would also clear the LocalStorage? But maybe not.)
  • The intent would be that if the site catches the exception and handles it in a timely fashion, we wouldn't clear its storage.

Comment 8

[Boris Zbarsky [:bzbarsky]]

I don't know what effect clearing everything would have, though I agree in this specific case it would likely not break anything, since all the data that matters should really be stored on the server, attached to the account. I only cleared the optimizely bits, because I happen to know that that's only used for A/B testing, so was exteremely unlikely to lose any actual state anyone cared about.

Comment 10

[simons]

Gret workling website for wordle game https://playwordle.app/ give it a try

Comment 11

[Jari Jalkanen]

Timewise, the occurrance of this issue coincided with the landing of the patch which extended the support of localstorage values from valid UTF to all DOM strings. In the code, one of the places where the error NS_ERROR_FILE_NO_DEVICE_SPACE may originate is right in the middle of the related migration. It is already known, that this migration takes up a lot of space.

To mitigate this in the spirit of releasing more space, we could just not do the migration but make a fresh start if we end up here.

The other source for the issue is when attempt to increase peak usage returns 0. Here we could perhaps trigger the timer and clear some data if the situation hasn't improved in the meanwhile.

Comment 12

[Jari Jalkanen]

Attachment: Bug 1755206 - Start with empty localstorage when quota does not allow migration. r=#dom-storage

Comment 13

[Jens Stutte [:jstutte]]

Jari, are you able to work on the planned changes of that patch?

Comment 14

[Jens Stutte [:jstutte]]

(In reply to Jari Jalkanen from comment #11)

Timewise, the occurrance of this issue coincided with the landing of the patch which extended the support of localstorage values from valid UTF to all DOM strings. In the code, one of the places where the error NS_ERROR_FILE_NO_DEVICE_SPACE may originate is right in the middle of the related migration. It is already known, that this migration takes up a lot of space.

Is this space taken by the migration only temporarily needed? Then I'd assume it should not count against the quota at all?

Comment 15

[Jari Jalkanen]

We check if there is any archived data to be migrated and try to update usage to the value after migration. If this fails, local storage initialization stops and returns with error NS_ERROR_FILE_NO_DEVICE_SPACE.

With this algorithm, in principle the user may now clear some quota and reload, which should allow the migration to proceed. However, in practice this is cumbersome and not obvious at all and I'm not sure if we should really assume that many users would follow this path.

Clearing the whole origin is also probably not a good solution because localstorage takes up so little quota so that it's probably not the root cause, and we can't really tell if deleting the other files is okay.

Maybe the solution of least harm would be to just skip the migration if it fails and let the local storage be initialized as if there was nothing to migrate.

Comment 16

[Phabricator Automation]

Comment on attachment 9301201 [details]
Bug 1755206 - Start with empty localstorage when quota does not allow migration. r=#dom-storage

Revision D160831 was moved to bug 1837614. Setting attachment 9301201 [details] to obsolete.

Comment 17

[Jari Jalkanen]

Attachment: Bug 1755206 - Allow setting context for remote debugging in content process. r=#dom-storage

Comment 18

[Jari Jalkanen]

Attachment: Bug 1755206 - Add remote debugging info for SetItem quota errors. r=#dom-storage

Depends on D180432

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4de33024b83e
Allow setting context for remote debugging in content process. r=dom-storage-reviewers,janv

Comment 20

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/4de33024b83e

Comment 21

[Jan Varga [:janv]]

(In reply to Randell Jesup [:jesup] (needinfo me) from comment #6)

Given the ideas from asuth in comment 4, how easy would it be to introduce some type of mitigation for this? Having a very popular site like wordle break is bad, especially if general users don't have any obvious way to resolve it. If their storage is simply accumulating, it sounds like most users of the site will eventually hit this and be stuck.

Even the most simplistic (clear everything) sounds like it would be better than doing nothing. (I presume this causes no issues with the functionality of the site, correct? bz?)

We are currently adding remote debugging for this to see how often this actually happens these days.

Comment 22

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e59de52b8ebe
Add remote debugging info for SetItem quota errors. r=dom-storage-reviewers,janv

Comment 23

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/e59de52b8ebe