1423744 - Extension fails to Sync every time after storage reaches 204800 Bytes once (even after clearing its local sync data and re-uploading)

Status: NEW

(WebExtensions :: Storage, defect, P3)

Description

[Daniel]

Attachment: Sync Test.zip

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0
Build ID: 20171128222554

Steps to reproduce:

I encountered a problem with Sync that I could reproduce on Firefox 57.0.1 (64-bit), Developer Edition 58.0b9 (64-bit), and Nightly 59.0a1 (2017-12-06) (64-bit).

If you try uploading a lot of data for an extension (above a limit of 204800 Bytes) that extension fails to Sync every time, even if afterwards the local sync storage is cleared and much less data (under the limit) is uploaded.

Steps (use attached WebExtension):
- Install as temporary extension and open the debug window.
- Go to extension settings page. Here you can save/load a string of a certain length to/from Sync. The default is 10000 chars. At the bottom there's an estimate size in bytes (2 * length) (this is probably wrong if compression is applied before syncing).
- Press "Load from Sync" and it should print "loaded! has length 0".
- Press "Save to Sync" and it should clear sync storage for the extension (this is irrelevant right now) and then write the string to it. To avoid running into "Maximum bytes per object exceeded ([number of bytes] > 16384 Bytes.)", the string is split into 1000-char strings before uploading.
- In the Firefox UI, press the Sync refresh button to force upload to Sync.
- Go to "about:sync-log" and there should be no errors.
- Increase the length of the string to something big like 900000. This is not allowed as it's over the Sync limit of 100KB.
- Press "Save to Sync", force Sync in the Firefox UI (it may take a few seconds), go to "about:sync-log" again, and check the error.
- From now on, even if you return to 10000 chars and retry uploading (remember that sync storage is even cleared before the upload), there will be an error in "about:sync-log".

If the extension is removed there are no more sync errors. If it is added again after that it's still fine. However, if we we try uploading any quantity of data from the extension to Sync again, the error returns. It's as if the maximum you ever tried uploading is the value your storage size is judged on. This is currently blocking me from testing Sync on my own extension, since I reached the limit once (I can change the ID for testing, though).

Furthermore, in the process I created a few dummy extensions (and changed their IDs) to test this and apparently their data is in Sync now and there's no way (?) to remove them.


Actual results:

The error is:

ERROR	Syncing [e-mail]: request failed: Error: HTTP 507; Error: HTTP 507 Insufficient Storage: Resource access is forbidden for this user (Collection maximum size exceeded (225351 > 204800 Bytes).) (resource://services-common/kinto-http-client.js:2354:21)

JS Stack trace:

[e-mail]:2377:21
[e-mail]:2352:14
async*[e-mail]:2432:14
async*[e-mail]:745:26
async*[e-mail]:661:33
async*[7]</_batchRequests/<@kinto-http-client.js:659:48
[15]</pMap/<@kinto-http-client.js:2678:37
Async*[e-mail]:2676:9
async*[e-mail]:659:15
async*[e-mail]:708:29
async*[e-mail]:2840:18
[e-mail]:2157:12
async*[6]</pushChanges/<@kinto-offline-client.js:1850:28
[e-mail]:875:183
[6]</_asyncToGenerator/</<@kinto-offline-client.js:875:437
[6]</_asyncToGenerator/<@kinto-offline-client.js:875:99
[e-mail]:1837:12
[6]</sync/<@kinto-offline-client.js:2029:15
[e-mail]:875:183
[6]</_asyncToGenerator/</</step/<@kinto-offline-client.js:875:361
promise callback*[e-mail]:875:314
promise callback*[e-mail]:875:314
[6]</_asyncToGenerator/</<@kinto-offline-client.js:875:437
[6]</_asyncToGenerator/<@kinto-offline-client.js:875:99
[e-mail]:2001:12
_syncCollection/<@ExtensionStorageSync.jsm:828:14
[e-mail]:839:20
async*[e-mail]:820:12
[e-mail]:758:27
async*syncAll/promises</<@ExtensionStorageSync.jsm:732:16
promise callback*syncAll/promises<@ExtensionStorageSync.jsm:731:14
[e-mail]:730:22
async*[e-mail]:40:12
async*[e-mail]:173:27
async*[e-mail]:726:12


Expected results:

Extension should fail to Sync when limit is exceeded (though an asynchronous error callback would be nice!) but after clearing and trying again with an acceptable storage size it should upload normally.