Open Bug 1482662 Opened 2 years ago Updated 4 months ago

[meta] The War on Storage Initialization Failures


(Core :: Storage: Quota Manager, defect, P1)




Tracking Status
firefox63 --- affected


(Reporter: janv, Assigned: janv)


(Depends on 14 open bugs, Blocks 4 open bugs)


(Keywords: meta)

No description provided.
Depends on: 1423917
Depends on: 1395102
Depends on: CVE-2018-12401
Depends on: 1348705
Depends on: 1436128
Depends on: 1436188
Depends on: 1425146
Depends on: 1345585
Depends on: 1395133
Depends on: 1482812
Priority: -- → P2
Depends on: 1481716
Keywords: meta
Summary: The War on Storage Initialization Failures → The War on Storage Initialization Failures [meta]
Duplicate of this bug: 1426119
Depends on: 944918
Blocks: 1487479
Depends on: 1487779
See Also: → 1471375
Depends on: 1493262
Status of this project this week:

[1] I plan to finish patches in bug 1423917 and send them to review this week and they should cover:
- A minor QuotaManager upgrade (mainly for cleaning obsolete origins)
  - Obsolete origins:
    - about+xxx+number (was found in bug 1422456)
    - Whitelist ".*", "desktop.ini", "Desktop.ini", and "Thumbs.db" (was reported in bug 1493262, bug 1423917, bug 944918)
    - Recognize chrome protocol
    - Ignore invalid origins during ungrades

[2] Since it's are hard to identify all the issues that affecting QM initializing, we should see the result after finishing bug 1436188 and its followup (bug 1487779).

I haven't carefully checked about bug 1436128, bug 1395133, bug 1348705, and bug 1267349, but I don't think they affects many users. Will take care of them after completing [1] and [2].
Add the bug for having a nicer error message when indexedDB finds an unexpected file in the indexedDB directory.
Depends on: 1502077
Status of this project this week:

1. Most patches in the bug 1423917 have sent to review. The rest of the patches are ready too, but they might be affected by changes in bug 1491637 and bug 1436188, so I haven't sent them yet.

2. Bug 1348705 seems to be fixed in another issue. Will try to verify that by a simple test.


High priority issues or issues that having patches:
Bug 1436188 has patches and is waiting for feedback. Bug 1487779 is the follow-up bug for it.
Bug 1491637 has patches and is waiting for review and feedback.

Bug 944918, 1423917, 1493262 should be fixed in bug 1423917. However, since the range of them are a little vague to me, so that I will monitor whether they are fixed by checking the data in Bug 1436188.

Bug 1436128 should be taken care in the local storage next generation.

Bug 1267349 and 1502077 is related to having a proper error message so their priorities are low to me.
Bug 1395133 is to add a test so that its priority is low to me.
Depends on: 1504535
Depends on: 1519822
See Also: → 1517531
Depends on: 1322569
Summary: The War on Storage Initialization Failures [meta] → [meta] The War on Storage Initialization Failures
No longer depends on: 944918
See Also: → 1541370
No longer blocks: Session_managers
Depends on: 1557289

LSNG is going to be disabled in 70.0.1 :(
It looks like there are too many quota manager storage initialization failures in the wild and since LSNG depends on quota manager and given widespread use of LS API, sites become unusable for many users. We need to revive telemetry for storage initialization failures on Nightly and possibly uplift that to beta too. We need to reduce storage initialization failures the same way as we reduced quota manager shutdown hangs. Then we can finally ship LSNG.

Assignee: nobody → jvarga
Priority: P2 → P1
Depends on: 1592204

I found a telemetry for Nightly 69:!cumulative=0&end_date=2019-07-07&include_spill=0&keys=internal!external!__none__!__none__&max_channel_version=nightly%252F69&measure=QM_INIT_TELEMETRY_ERROR&min_channel_version=nightly%252F63&os=Windows_NT&processType=parent&product=Firefox&sanitize=1&sort_by_value=0&sort_keys=submissions&start_date=2019-05-21&table=0&trim=1&use_submission_date=0

It seems there were some other problems still (not just unknown files/directories). The situation on beta and release can be (slightly) different.

Anyway, I propose to add a new simple telemetry that reflects temporary storage initialization success rate. I guess that can be uplifted to all channels given simple patch and simplicity of the statistics.

Depends on: 1592934
Depends on: 1594075
Depends on: 1594080
Blocks: 1541370
See Also: 1541370
Depends on: 1593365
No longer depends on: 1395133
Blocks: 1395133
Depends on: 1595913
Depends on: 1598555
Blocks: 1599979
Depends on: 1600352

Some of the bugs tracked here need to be revised with respect to the decision to ignore unknown files and directories in current QM version and also with respect to QM v4, I'll start cleaning it up following days.

See Also: → IndexedDB-SM
Depends on: 1619325
Depends on: 1619592
Depends on: 1619891
Depends on: 1619893
Depends on: 1619895
Depends on: 1619896
Depends on: 1616003
Depends on: 1606318
Depends on: 1617842
Depends on: 1615552

The bug dependency was messy, so I cleaned it up a bit. In general, this is the main meta bug for addressing storage initialization failures. Bug 1593365 is a sub project that is happening in parallel to fixing issues in the current storage version.

There are also new bugs that cover individual keys of the QM_FIRST_INITIALIZATION_ATTEMPT telemetry probe. For example storage upgrade fixes should be tracked under bug 1619891.

I will try to clean up the bug dependency even more.

Depends on: 1536796
No longer depends on: 1536796
No longer depends on: 1594075
No longer depends on: 1617842, 1616003
No longer depends on: 1615552
No longer depends on: 1619325
No longer depends on: 1592204
No longer depends on: 1619592
Severity: normal → S3
Depends on: 1635351
Blocks: IndexedDB-SM
Depends on: 1645312
No longer depends on: 1645312
You need to log in before you can comment on or make changes to this bug.