Open
Bug 1482662
Opened 6 years ago
Updated 8 months ago
[meta] The War on Storage Initialization Failures
Categories
(Core :: Storage: Quota Manager, defect, P1)
Core
Storage: Quota Manager
Tracking
()
NEW
| Tracking | Status | |
|---|---|---|
| firefox63 | --- | affected |
People
(Reporter: janv, Unassigned)
References
(Depends on 13 open bugs, Blocks 3 open bugs)
Details
(Keywords: meta)
Attachments
(1 file, 3 obsolete files)
|
9.16 KB,
text/plain
|
Details |
No description provided.
|
Reporter | |
Updated•6 years ago
|
Depends on: CVE-2018-12401
|
||
Updated•6 years ago
|
Priority: -- → P2
|
|
||
Updated•6 years ago
|
Keywords: meta
Summary: The War on Storage Initialization Failures → The War on Storage Initialization Failures [meta]
|
||
Comment 2•6 years ago
|
||
Status of this project this week:
[1] I plan to finish patches in bug 1423917 and send them to review this week and they should cover:
- A minor QuotaManager upgrade (mainly for cleaning obsolete origins)
- Obsolete origins:
- about+xxx+number (was found in bug 1422456)
- Whitelist ".*", "desktop.ini", "Desktop.ini", and "Thumbs.db" (was reported in bug 1493262, bug 1423917, bug 944918)
- Recognize chrome protocol
- Ignore invalid origins during ungrades
[2] Since it's are hard to identify all the issues that affecting QM initializing, we should see the result after finishing bug 1436188 and its followup (bug 1487779).
I haven't carefully checked about bug 1436128, bug 1395133, bug 1348705, and bug 1267349, but I don't think they affects many users. Will take care of them after completing [1] and [2].
|
|
||
Comment 3•6 years ago
|
||
Add the bug for having a nicer error message when indexedDB finds an unexpected file in the indexedDB directory.
Depends on: 1502077
|
|
||
Comment 4•6 years ago
|
||
Status of this project this week:
1. Most patches in the bug 1423917 have sent to review. The rest of the patches are ready too, but they might be affected by changes in bug 1491637 and bug 1436188, so I haven't sent them yet.
2. Bug 1348705 seems to be fixed in another issue. Will try to verify that by a simple test.
So:
High priority issues or issues that having patches:
Bug 1436188 has patches and is waiting for feedback. Bug 1487779 is the follow-up bug for it.
Bug 1491637 has patches and is waiting for review and feedback.
Bug 944918, 1423917, 1493262 should be fixed in bug 1423917. However, since the range of them are a little vague to me, so that I will monitor whether they are fixed by checking the data in Bug 1436188.
Bug 1436128 should be taken care in the local storage next generation.
Others:
Bug 1267349 and 1502077 is related to having a proper error message so their priorities are low to me.
Bug 1395133 is to add a test so that its priority is low to me.
Blocks: Session_managers
|
||
Updated•6 years ago
|
Summary: The War on Storage Initialization Failures [meta] → [meta] The War on Storage Initialization Failures
No longer blocks: Session_managers
|
|
Reporter | |
Comment 5•5 years ago
•
|
||
LSNG is going to be disabled in 70.0.1 :(
It looks like there are too many quota manager storage initialization failures in the wild and since LSNG depends on quota manager and given widespread use of LS API, sites become unusable for many users. We need to revive telemetry for storage initialization failures on Nightly and possibly uplift that to beta too. We need to reduce storage initialization failures the same way as we reduced quota manager shutdown hangs. Then we can finally ship LSNG.
|
|
Reporter | |
Updated•5 years ago
|
Assignee: nobody → jvarga
Status: NEW → ASSIGNED
Priority: P2 → P1
|
|
Reporter | |
Comment 6•5 years ago
|
||
It seems there were some other problems still (not just unknown files/directories). The situation on beta and release can be (slightly) different.
Anyway, I propose to add a new simple telemetry that reflects temporary storage initialization success rate. I guess that can be uplifted to all channels given simple patch and simplicity of the statistics.
|
|
Reporter | |
Comment 7•5 years ago
|
||
Some of the bugs tracked here need to be revised with respect to the decision to ignore unknown files and directories in current QM version and also with respect to QM v4, I'll start cleaning it up following days.
See Also: → IndexedDB-SM
|
|
Reporter | |
Comment 8•5 years ago
|
||
The bug dependency was messy, so I cleaned it up a bit. In general, this is the main meta bug for addressing storage initialization failures. Bug 1593365 is a sub project that is happening in parallel to fixing issues in the current storage version.
There are also new bugs that cover individual keys of the QM_FIRST_INITIALIZATION_ATTEMPT telemetry probe. For example storage upgrade fixes should be tracked under bug 1619891.
I will try to clean up the bug dependency even more.
|
|
Reporter | |
Updated•5 years ago
|
|
|
Reporter | |
Updated•5 years ago
|
Severity: normal → S3
Blocks: IndexedDB-SM
|
||
Comment 9•4 years ago
|
||
|
|
||
Updated•4 years ago
|
Attachment #9196363 -
Attachment mime type: text/x-python → text/plain
|
|
||
Comment 10•4 years ago
•
|
||
For build id 20210109092457, the current failures according to the script are:
Initialization::TemporaryStorage
2447 dom/cache/FileUtils.cpp:762 NS_ERROR_FILE_NOT_FOUND # not propagated
dom/cache/QuotaClient.cpp:374 NS_ERROR_FILE_NOT_FOUND
not propagated
1132 dom/quota/ActorsParent.cpp:2711 NS_ERROR_FILE_NOT_FOUND # Bug 1679541 (will add call sites)
dom/quota/ActorsParent.cpp:5016 NS_ERROR_FILE_NOT_FOUND
638 dom/quota/ActorsParent.cpp:5342 NS_ERROR_FAILURE # Bug 1686041 (call to IsDirectory)
dom/quota/ActorsParent.cpp:5424 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5251 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5255 NS_ERROR_FAILURE
312 dom/quota/ActorsParent.cpp:4552 # Bug 1686697
dom/quota/ActorsParent.cpp:4590 NS_ERROR_FAILURE
301 dom/localstorage/ActorsParent.cpp:1189 # not propagated
not propagated
239 dom/quota/ActorsParent.cpp:4572 # Bug 1685098 (will add QM_TRY calls after this non-propagated failure)
dom/quota/ActorsParent.cpp:4590 NS_ERROR_FAILURE
224 dom/indexeddb/ActorsParent.cpp:5929 NS_ERROR_FILE_ACCESS_DENIED # Bug 1686060
dom/indexeddb/ActorsParent.cpp:6098 NS_ERROR_FILE_ACCESS_DENIED
dom/indexeddb/ActorsParent.cpp:13110 NS_ERROR_FILE_ACCESS_DENIED
dom/indexeddb/ActorsParent.cpp:13137 NS_ERROR_UNEXPECTED
dom/indexeddb/ActorsParent.cpp:13036 NS_ERROR_UNEXPECTED
dom/quota/ActorsParent.cpp:5384 NS_ERROR_UNEXPECTED
dom/quota/ActorsParent.cpp:5424 NS_ERROR_UNEXPECTED
dom/quota/ActorsParent.cpp:5251 NS_ERROR_UNEXPECTED
dom/quota/ActorsParent.cpp:5255 NS_ERROR_UNEXPECTED
150 dom/quota/ActorsParent.cpp:1136 NS_ERROR_FAILURE # Bug 1686041 (call to IsDirectory, this is actually in dom/localstorage/ActorsParent.cpp)
dom/localstorage/ActorsParent.cpp:8774 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5384 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5424 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5251 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5255 NS_ERROR_FAILURE
129 dom/quota/ActorsParent.cpp:2697 NS_ERROR_FILE_IS_LOCKED # Bug 1685098 (will add intermediate call sites)
dom/quota/ActorsParent.cpp:10556 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:5217 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:5255 NS_ERROR_FILE_IS_LOCKED
120 dom/indexeddb/ActorsParent.cpp:13122 # not propagated
not propagated
115 dom/quota/ActorsParent.cpp:5259 NS_ERROR_ABORT # shutdown
102 dom/localstorage/ActorsParent.cpp:1174 NS_ERROR_FILE_NOT_FOUND # not propagated
not propagated
74 dom/quota/QuotaCommon.h:1172 NS_ERROR_FAILURE # Bug 1686041 (call to GetDirectoryEntries)
dom/indexeddb/ActorsParent.cpp:13476 NS_ERROR_FAILURE
dom/indexeddb/ActorsParent.cpp:13077 NS_ERROR_FAILURE
dom/indexeddb/ActorsParent.cpp:13036 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5384 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5424 NS_ERROR_FAILURE # not propagated immediately in Nightly!
dom/quota/ActorsParent.cpp:1136 NS_ERROR_FAILURE
dom/localstorage/ActorsParent.cpp:8774 NS_ERROR_FAILURE # Bug 1686041 (call to IsDirectory, this is actually in dom/localstorage/ActorsParent.cpp)
dom/quota/ActorsParent.cpp:5384 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5424 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5251 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5255 NS_ERROR_FAILURE
67 dom/indexeddb/ActorsParent.cpp:5929 NS_ERROR_FILE_ACCESS_DENIED # Bug 1686060
dom/indexeddb/ActorsParent.cpp:6098 NS_ERROR_FILE_ACCESS_DENIED
dom/indexeddb/ActorsParent.cpp:13110 NS_ERROR_FILE_ACCESS_DENIED
dom/indexeddb/ActorsParent.cpp:13137 NS_ERROR_UNEXPECTED
dom/indexeddb/ActorsParent.cpp:13036 NS_ERROR_UNEXPECTED
dom/quota/ActorsParent.cpp:5384 NS_ERROR_UNEXPECTED
dom/quota/ActorsParent.cpp:5424 NS_ERROR_UNEXPECTED
dom/quota/ActorsParent.cpp:4583 NS_ERROR_UNEXPECTED
dom/quota/ActorsParent.cpp:4590 NS_ERROR_UNEXPECTED
47 dom/cache/QuotaClient.cpp:45 # shutdown
dom/cache/QuotaClient.cpp:50 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:107 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:416 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:47 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:50 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:450 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5384 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5424 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5428 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5251 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5255 NS_ERROR_ABORT
24 dom/quota/QuotaCommon.h:1172 NS_ERROR_FAILURE # Bug 1686041 (call to GetDirectoryEntries)
dom/indexeddb/ActorsParent.cpp:13476 NS_ERROR_FAILURE
dom/indexeddb/ActorsParent.cpp:13077 NS_ERROR_FAILURE
dom/indexeddb/ActorsParent.cpp:13036 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5384 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5424 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5251 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5255 NS_ERROR_FAILURE
20 dom/quota/ActorsParent.cpp:5428 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5251 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5255 NS_ERROR_ABORT
15 dom/indexeddb/ActorsParent.cpp:5977 NS_ERROR_FILE_ACCESS_DENIED
dom/indexeddb/ActorsParent.cpp:13130 NS_ERROR_FILE_ACCESS_DENIED
dom/indexeddb/ActorsParent.cpp:13134 NS_ERROR_UNEXPECTED
12 dom/localstorage/ActorsParent.cpp:1174 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
not propagated
10 dom/quota/ActorsParent.cpp:4533 NS_ERROR_ILLEGAL_VALUE
dom/quota/ActorsParent.cpp:4590 NS_ERROR_ILLEGAL_VALUE
7 dom/quota/ActorsParent.cpp:2697 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:10556 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:4569 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:4590 NS_ERROR_FILE_IS_LOCKED
6 dom/cache/QuotaClient.cpp:45
dom/cache/QuotaClient.cpp:50 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:450 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5384 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5424 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5428 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5251 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5255 NS_ERROR_ABORT
5 dom/cache/QuotaClient.cpp:45
dom/cache/QuotaClient.cpp:50 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:107 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:416 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:47 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:50 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:450 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5384 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5424 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5428 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:4583 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:4590 NS_ERROR_ABORT
4 dom/quota/ActorsParent.cpp:5428 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:4583 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:4590 NS_ERROR_ABORT
2 dom/cache/QuotaClient.cpp:45
dom/cache/QuotaClient.cpp:50 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:450 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5384 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5424 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5251 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5255 NS_ERROR_ABORT
2 dom/cache/QuotaClient.cpp:45
dom/cache/QuotaClient.cpp:50 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:107 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:416 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:47 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:50 NS_ERROR_ABORT
dom/cache/QuotaClient.cpp:450 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5384 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5424 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5251 NS_ERROR_ABORT
dom/quota/ActorsParent.cpp:5255 NS_ERROR_ABORT
2 dom/quota/QuotaCommon.h:1172 NS_ERROR_FAILURE # Bug 1686041 (call to GetDirectoryEntries)
dom/indexeddb/ActorsParent.cpp:13476 NS_ERROR_FAILURE
dom/indexeddb/ActorsParent.cpp:13077 NS_ERROR_FAILURE
dom/indexeddb/ActorsParent.cpp:13036 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5384 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5424 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:1136 NS_ERROR_FAILURE
dom/localstorage/ActorsParent.cpp:8774 NS_ERROR_FAILURE
1 dom/quota/ActorsParent.cpp:2711 NS_ERROR_FILE_IS_LOCKED # Bug 1679541 (will add call sites)
dom/quota/ActorsParent.cpp:5016 NS_ERROR_FILE_IS_LOCKED
1 dom/indexeddb/ActorsParent.cpp:5929 NS_ERROR_FILE_ACCESS_DENIED # Bug 1686060
dom/indexeddb/ActorsParent.cpp:6098 NS_ERROR_FILE_ACCESS_DENIED
dom/indexeddb/ActorsParent.cpp:13110 NS_ERROR_FILE_ACCESS_DENIED
Initialization::Storage
77 dom/quota/ActorsParent.cpp:2711 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:10382 NS_ERROR_FILE_NOT_FOUND
67 dom/indexeddb/ActorsParent.cpp:12934
not propagated
I'm going to file individual bugs for the top items here, and include the bug numbers in the list.
|
|
||
Comment 11•4 years ago
|
||
@janv, I guess all failures involving NS_ERROR_ABORT can be ignored, as they are due to a user request for shutdown during storage initialization?
Flags: needinfo?(jvarga)
|
|
Reporter | |
Comment 12•4 years ago
|
||
(In reply to Simon Giesecke [:sg] [he/him] from comment #11)
@janv, I guess all failures involving
NS_ERROR_ABORTcan be ignored, as they are due to a user request for shutdown during storage initialization?
Yes, I think I raised that during reviews that we need to do something about that.
Flags: needinfo?(jvarga)
|
|
||
Comment 13•4 years ago
|
||
(In reply to Jan Varga [:janv] from comment #12)
(In reply to Simon Giesecke [:sg] [he/him] from comment #11)
@janv, I guess all failures involving
NS_ERROR_ABORTcan be ignored, as they are due to a user request for shutdown during storage initialization?Yes, I think I raised that during reviews that we need to do something about that.
Sorry, I don't remember exactly. What do you think should be done about the aborts?
|
|
Reporter | |
Comment 14•4 years ago
•
|
||
The first thing that came into my mind is to add a quota module specific abort error and then detect that in LogError.
All our NS_ABORT_ERROR cases which are used just for cancelling an operation (either by a user request or by shutdown condition) can be then converted to the new error.
|
|
Reporter | |
Comment 15•4 years ago
|
||
xpcom/base/ErrorList.py
+modules["QUOTA_MANAGER"] = Mod(44)
+# 44: NS_ERROR_MODULE_QUOTA_MANAGER
+# =======================================================================
+with modules["QUOTA_MANAGER"]:
- errors["NS_ERROR_QM_ABORT"] = FAILURE(1)
+# =======================================================================
I planned to add such error codes anyway.
|
|
||
Comment 16•4 years ago
|
||
Updated script with autodetection of split points.
|
|
||
Updated•4 years ago
|
Attachment #9196558 -
Attachment is obsolete: true
|
|
||
Updated•4 years ago
|
Attachment #9196558 -
Attachment is obsolete: false
|
|
||
Updated•4 years ago
|
Attachment #9196363 -
Attachment is obsolete: true
|
|
||
Comment 17•4 years ago
•
|
||
On the latest Nightly builds, the top 4 propagated issues are all blocked by Bug 1686041 and show a NS_ERROR_FAILURE from some nsLocalFile method.
The errors, leaving out the non-propagated and NS_ERROR_ABORT ones are:
Initialization::TemporaryStorage
1546 dom/quota/QuotaCommon.h:1181 NS_ERROR_FAILURE # Blocked by Bug 1686041
dom/quota/ActorsParent.cpp:5068 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:4530 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:6532 NS_ERROR_FAILURE
1136 dom/cache/QuotaClient.cpp:63 NS_ERROR_FAILURE # Blocked by Bug 1686041
dom/cache/QuotaClient.cpp:47 NS_ERROR_FAILURE
dom/cache/QuotaClient.cpp:50 NS_ERROR_FAILURE
dom/cache/QuotaClient.cpp:106 NS_ERROR_FAILURE
dom/cache/QuotaClient.cpp:428 NS_ERROR_FAILURE
dom/cache/QuotaClient.cpp:47 NS_ERROR_FAILURE
dom/cache/QuotaClient.cpp:50 NS_ERROR_FAILURE
dom/cache/QuotaClient.cpp:462 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5193 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5233 NS_ERROR_FAILURE
1136 dom/quota/ActorsParent.cpp:5060 NS_ERROR_FAILURE # Blocked by Bug 1686041
dom/quota/ActorsParent.cpp:5064 NS_ERROR_FAILURE
1133 dom/quota/ActorsParent.cpp:4530 NS_ERROR_FAILURE # Blocked by Bug 1686041
dom/quota/ActorsParent.cpp:6532 NS_ERROR_FAILURE
161 dom/quota/ActorsParent.cpp:5060 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST # Tail of the two issues below
dom/quota/ActorsParent.cpp:5064 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
81 dom/indexeddb/ActorsParent.cpp:12766 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST # Bug 1687256
dom/indexeddb/ActorsParent.cpp:12776 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
dom/indexeddb/ActorsParent.cpp:13191 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
dom/indexeddb/ActorsParent.cpp:13036 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
dom/quota/ActorsParent.cpp:5193 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
dom/quota/ActorsParent.cpp:5233 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
80 dom/cache/QuotaClient.cpp:63 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST # Bug 1687254
dom/cache/QuotaClient.cpp:47 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
dom/cache/QuotaClient.cpp:50 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
dom/cache/QuotaClient.cpp:106 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
dom/cache/QuotaClient.cpp:428 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
dom/cache/QuotaClient.cpp:47 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
dom/cache/QuotaClient.cpp:50 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
dom/cache/QuotaClient.cpp:462 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
dom/quota/ActorsParent.cpp:5193 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
dom/quota/ActorsParent.cpp:5233 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
80 dom/quota/ActorsParent.cpp:4530 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST # Maybe blocked by https://phabricator.services.mozilla.com/D102181?
dom/quota/ActorsParent.cpp:6532 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
8 dom/cache/FileUtils.cpp:479 NS_ERROR_FILE_ACCESS_DENIED
6 dom/cache/FileUtils.cpp:479 NS_ERROR_FILE_DIR_NOT_EMPTY
5 dom/quota/ActorsParent.cpp:4530 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:6532 NS_ERROR_FILE_IS_LOCKED
4 dom/quota/ActorsParent.cpp:2585 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:11170 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:10023 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:11158 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:4835 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:4908 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:5024 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:5064 NS_ERROR_FILE_IS_LOCKED
3 dom/quota/QuotaCommon.h:1181 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5068 NS_ERROR_FAILURE
2 dom/quota/ActorsParent.cpp:4530 NS_ERROR_FAILURE
1 dom/cache/QuotaClient.cpp:63 NS_ERROR_FILE_TARGET_DOES_NOT_EXIST
1 dom/quota/ActorsParent.cpp:464 NS_ERROR_FILE_NO_DEVICE_SPACE
dom/quota/ActorsParent.cpp:4544 NS_ERROR_FILE_NO_DEVICE_SPACE
dom/quota/ActorsParent.cpp:6532 NS_ERROR_FILE_NO_DEVICE_SPACE
1 dom/localstorage/ActorsParent.cpp:998 NS_ERROR_FILE_IS_LOCKED
dom/localstorage/ActorsParent.cpp:976 NS_ERROR_FILE_IS_LOCKED
dom/localstorage/ActorsParent.cpp:8365 NS_ERROR_FILE_IS_LOCKED
dom/localstorage/ActorsParent.cpp:8371 NS_ERROR_FILE_IS_LOCKED
dom/localstorage/ActorsParent.cpp:8379 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:5193 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:5233 NS_ERROR_FILE_IS_LOCKED
1 dom/quota/ActorsParent.cpp:5060 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:5064 NS_ERROR_FILE_IS_LOCKED
1 dom/cache/QuotaClient.cpp:63 NS_ERROR_FAILURE
dom/cache/QuotaClient.cpp:47 NS_ERROR_FAILURE
dom/cache/QuotaClient.cpp:50 NS_ERROR_FAILURE
1 dom/cache/QuotaClient.cpp:63 NS_ERROR_FAILURE
1 dom/quota/QuotaCommon.h:1181 NS_ERROR_FAILURE
Initialization::Storage
none!
|
|
||
Comment 18•4 years ago
|
||
Updated the script to sort by distinct client counts, and filter out NS_ERROR_ABORT pseudo-failures.
Attachment #9196558 -
Attachment is obsolete: true
|
|
||
Comment 19•4 years ago
•
|
||
Results from this weekend's builds (leaving out the non-propagated and NS_ERROR_ABORT ones):
Initialization::TemporaryStorage
Clients Sessions Location Result
3 8 dom/cache/FileUtils.cpp:479 NS_ERROR_FILE_DIR_NOT_EMPTY # caller is not instrumented, this is not propagated
1 152 dom/localstorage/ActorsParent.cpp:8330 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:5187 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:5227 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:5054 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:5058 NS_ERROR_FILE_IS_LOCKED
1 151 dom/quota/ActorsParent.cpp:4529 NS_ERROR_FILE_IS_LOCKED # tail from the stack above
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FILE_IS_LOCKED
1 5 dom/cache/FileUtils.cpp:479 NS_ERROR_FILE_ACCESS_DENIED # caller is not instrumented, this is not propagated
1 73 dom/quota/QuotaCommon.cpp:180 WIN32(0x570) # 0x570 is ERROR_FILE_CORRUPT, maybe this is running on VMware?
dom/cache/QuotaClient.cpp:64 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:111 WIN32(0x570)
dom/cache/QuotaClient.cpp:423 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:457 WIN32(0x570)
dom/quota/ActorsParent.cpp:5187 WIN32(0x570)
dom/quota/ActorsParent.cpp:5227 WIN32(0x570)
dom/quota/ActorsParent.cpp:5054 WIN32(0x570)
dom/quota/ActorsParent.cpp:5058 WIN32(0x570)
1 73 dom/quota/ActorsParent.cpp:4529 WIN32(0x570) # tail from the stack above
dom/quota/ActorsParent.cpp:6526 WIN32(0x570)
1 1 dom/cache/FileUtils.cpp:501 NS_ERROR_FILE_ACCESS_DENIED # caller is not instrumented, this is not propagated
Initialization::Storage
Clients Sessions Location Result
1 3999 dom/quota/ActorsParent.cpp:576
dom/quota/ActorsParent.cpp:6185 NS_ERROR_FILE_CORRUPTED
1 1 dom/quota/ActorsParent.cpp:576
The previous results were very distorted as they were sorted by session numbers, and it seems that some errors are associated with a huge number of sessions per client. Now the sorting is by unique clients that are affected. As can be seen, only a small number of clients was affected. But we will need to monitor the results over a work week as well.
All errors that are shown above, which are actually propagated, affect only a single client. Before taking further action, we should wait for the results of some more days.
|
||
Comment 20•4 years ago
|
||
FWIW, it seems that while we know NS_ERROR_FILE_CORRUPTED in our code, it is not mapped from any WIN32 error message in nsLocalFileWin.cpp.
|
|
||
Comment 21•4 years ago
|
||
We now have first results from beta/aurora 86.0b1. There are the propagating, non-abort stacks so far that affect more than 1 client:
Initialization::TemporaryStorage
Clients Sessions Location Result
2 313 dom/localstorage/ActorsParent.cpp:956 WIN32(0x570) # A
dom/localstorage/ActorsParent.cpp:8320 WIN32(0x570)
dom/quota/ActorsParent.cpp:5187 WIN32(0x570)
dom/quota/ActorsParent.cpp:5227 WIN32(0x570)
dom/quota/ActorsParent.cpp:5231 WIN32(0x570)
dom/quota/ActorsParent.cpp:5234 WIN32(0x570)
dom/quota/ActorsParent.cpp:5054 WIN32(0x570)
dom/quota/ActorsParent.cpp:5058 WIN32(0x570)
dom/quota/ActorsParent.cpp:5062 WIN32(0x570)
dom/quota/ActorsParent.cpp:5065 WIN32(0x570)
dom/quota/ActorsParent.cpp:4529 WIN32(0x570)
dom/quota/ActorsParent.cpp:4532 WIN32(0x570)
dom/quota/ActorsParent.cpp:6526 WIN32(0x570)
2 1502 dom/quota/ActorsParent.cpp:462 NS_ERROR_FILE_CORRUPTED # B
dom/quota/ActorsParent.cpp:4543 NS_ERROR_FILE_CORRUPTED
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FILE_CORRUPTED
Initialization::Storage
Clients Sessions Location Result
4 1563 dom/quota/ActorsParent.cpp:462 NS_ERROR_FILE_CORRUPTED # C
dom/quota/ActorsParent.cpp:6303 NS_ERROR_FILE_CORRUPTED
3 3 dom/quota/ActorsParent.cpp:546 NS_ERROR_FILE_CORRUPTED # D
dom/quota/ActorsParent.cpp:6200 NS_ERROR_FILE_CORRUPTED
3 3534 dom/quota/ActorsParent.cpp:546 NS_ERROR_FILE_CORRUPTED # E
dom/quota/ActorsParent.cpp:5985 NS_ERROR_FILE_CORRUPTED
dom/quota/ActorsParent.cpp:6208 NS_ERROR_FILE_CORRUPTED
A is a case where we get the Windows System Error ERROR_FILE_CORRUPT when checking a path (of a usage file in this case) for being a directory.
B-E are all cases where executing a SQL statement results in NS_ERROR_FILE_CORRUPTED. It's conceivable that these are cases where a file operation failed with ERROR_FILE_CORRUPTED as well, but inside SQLite code. Cases B and C are inside InvalidateCache, which is only called if the cache was deemed usable before, so it's very unlikely this is caused by internal actions. Cases D and E are inside InitializeLocalStorageArchive on the first CREATE TABLE statement on a freshly created database, which also seems to indicate this is an externally caused condition.
In all of these cases, the question is if this is just a transient condition, which might go away by retrying.
|
|
Reporter | |
Comment 22•4 years ago
|
||
Everything related to the local storage archive (for example InitializeLocalStorageArchive) which returns NS_ERROR_FILE_CORRUPTED should probably trigger removal of the archive and creating an empty one. The archive is created from the database used by the old localStorage implementation. We don't do any vacuuming of the archive after its copied from the old database because that could slow down entire initialization.
|
|
||
Comment 23•4 years ago
|
||
Might we see here also the consequences of bug 1588498 in some cases? Something like: QM Shutdown hang -> Unclean SQLite shutdown -> broken database file?
|
|
Reporter | |
Comment 24•4 years ago
•
|
||
(In reply to Jens Stutte [:jstutte] from comment #23)
Might we see here also the consequences of bug 1588498 in some cases? Something like: QM Shutdown hang -> Unclean SQLite shutdown -> broken database file?
Yes, but that should be rare since SQLite should be able to recover from a crash in most of the cases. Here's a description of how to corrupt an SQLite database file: https://www.sqlite.org/howtocorrupt.html
It also depends on the setting of the "synchronous" flag:
https://www.sqlite.org/pragma.html#pragma_synchronous
For example, the storage.sqlite database which is very important for QM, has this set to "EXTRA". The database is small and we don't do a lot of writes.
IDB: synchronous = NORMAL (until dom.indexedDB.fullSynchronous is set to true which will set the pragma to FULL)
DOMCACHE: synchronous = default setting (I think)
LSNG: synchronous = FULL
|
|
Reporter | |
Comment 25•4 years ago
|
||
We should also start thinking if we somehow can get QM_TRY data from Beta (not only early Beta) or even Release.
There are two main issues, increased binary size and possibly huge amount of data.
However the telemetry reporting is currently scoped to storage initialization only and maybe it's possible to enable it for a fraction of clients.
|
|
||
Comment 26•4 years ago
|
||
(In reply to Jan Varga [:janv] from comment #25)
We should also start thinking if we somehow can get QM_TRY data from Beta (not only early Beta) or even Release.
There are two main issues, increased binary size and possibly huge amount of data.
However the telemetry reporting is currently scoped to storage initialization only and maybe it's possible to enable it for a fraction of clients.
If we have a pref for it, we can ask for an experiment.
|
|
||
Comment 27•4 years ago
|
||
(In reply to Jan Varga [:janv] from comment #25)
We should also start thinking if we somehow can get QM_TRY data from Beta (not only early Beta) or even Release.
Late beta maybe, but why? Is there a different population for late beta than early beta?
However, I don't think we should do this for Release. We should only consider this if we really need to, i.e. when we notice that Release population suffers from issues that we couldn't see in Beta. But we don't have any indication for that right now.
There are two main issues, increased binary size and possibly huge amount of data.
There's also a performance impact (independent of whether telemetry reporting is active or not).
However the telemetry reporting is currently scoped to storage initialization only and maybe it's possible to enable it for a fraction of clients.
I don't think that's the critical point. When it's in the build, it can be activated unconditionally.
|
|
||
Comment 28•4 years ago
|
||
(In reply to Jan Varga [:janv] from comment #22)
Everything related to the local storage archive (for example
InitializeLocalStorageArchive) which returnsNS_ERROR_FILE_CORRUPTEDshould probably trigger removal of the archive and creating an empty one. The archive is created from the database used by the old localStorage implementation. We don't do any vacuuming of the archive after its copied from the old database because that could slow down entire initialization.
Ok, I didn't look at that closely enough yet. I was assuming that a newly created storage archive is empty, but if it's not that can better explain the NS_ERROR_FILE_CORRUPTED errors. I will reanalyze the beta failure today or tomorrow and then file a bug for this issue.
|
|
Reporter | |
Comment 29•4 years ago
|
||
(In reply to Simon Giesecke [:sg] [he/him] from comment #27)
However, I don't think we should do this for Release. We should only consider this if we really need to, i.e. when we notice that Release population suffers from issues that we couldn't see in Beta. But we don't have any indication for that right now.
I think otherwise. It happened multiple times in the past that quite big issues manifested only on Release. For example LSNG on Release which had to be disabled by a dot release. There was extensive testing before that, even manual testing by QA. The other one was the issue with the old localStorage implementation, bug 1683299.
Experimental data from Release could help with eliminating unexpected surprises after enabling LSNG on Release again.
We could enable QM_TRY only in Release 87 and then disable it. I think we should explore options here more, especially if the performance impact is so big/noticeable.
|
|
||
Comment 30•4 years ago
•
|
||
These are the top beta failures, affecting more than 3 clients:
Initialization::TemporaryStorage
Clients Sessions Location Result
28 35726 dom/quota/ActorsParent.cpp:462 NS_ERROR_FILE_CORRUPTED # Bug 1689636
dom/quota/ActorsParent.cpp:4543 NS_ERROR_FILE_CORRUPTED
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FILE_CORRUPTED
15 24 dom/indexeddb/ActorsParent.cpp:13441 NS_ERROR_FILE_NOT_FOUND # Bug 1689655
dom/indexeddb/ActorsParent.cpp:13491 NS_ERROR_FILE_NOT_FOUND
dom/indexeddb/ActorsParent.cpp:13092 NS_ERROR_FILE_NOT_FOUND
dom/indexeddb/ActorsParent.cpp:13051 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5187 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5227 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5231 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5234 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5054 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5058 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5062 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5065 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:4529 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:4532 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FILE_NOT_FOUND
12 2226 dom/quota/QuotaCommon.h:1174 WIN32(0x570)
dom/cache/FileUtilsImpl.h:83 WIN32(0x570)
dom/cache/QuotaClient.cpp:109 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:111 WIN32(0x570)
dom/cache/QuotaClient.cpp:423 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:457 WIN32(0x570)
dom/quota/ActorsParent.cpp:5187 WIN32(0x570)
dom/quota/ActorsParent.cpp:5227 WIN32(0x570)
dom/quota/ActorsParent.cpp:5231 WIN32(0x570)
dom/quota/ActorsParent.cpp:5234 WIN32(0x570)
dom/quota/ActorsParent.cpp:5054 WIN32(0x570)
dom/quota/ActorsParent.cpp:5058 WIN32(0x570)
dom/quota/ActorsParent.cpp:5062 WIN32(0x570)
dom/quota/ActorsParent.cpp:5065 WIN32(0x570)
dom/quota/ActorsParent.cpp:4529 WIN32(0x570)
dom/quota/ActorsParent.cpp:4532 WIN32(0x570)
dom/quota/ActorsParent.cpp:6526 WIN32(0x570)
8 76 dom/cache/FileUtilsImpl.h:42 NS_ERROR_FILE_NOT_FOUND # Bug 1689656
dom/cache/FileUtilsImpl.h:83 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:109 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:49 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:52 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:111 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:423 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:49 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:52 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:457 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5187 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5227 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5231 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5234 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5054 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5058 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5062 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5065 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:4529 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:4532 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FILE_NOT_FOUND
8 34 dom/quota/ActorsParent.cpp:2557 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:11175 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:10028 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:11163 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:4834 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:4907 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5018 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5058 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5062 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5065 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:4529 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:4532 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FAILURE
8 12 dom/quota/ActorsParent.cpp:462 NS_ERROR_FILE_CORRUPTED # Bug 1689636
dom/quota/ActorsParent.cpp:4543 NS_ERROR_FILE_CORRUPTED
7 2621 dom/localstorage/ActorsParent.cpp:956 WIN32(0x570)
dom/localstorage/ActorsParent.cpp:8320 WIN32(0x570)
dom/quota/ActorsParent.cpp:5187 WIN32(0x570)
dom/quota/ActorsParent.cpp:5227 WIN32(0x570)
dom/quota/ActorsParent.cpp:5231 WIN32(0x570)
dom/quota/ActorsParent.cpp:5234 WIN32(0x570)
dom/quota/ActorsParent.cpp:5054 WIN32(0x570)
dom/quota/ActorsParent.cpp:5058 WIN32(0x570)
dom/quota/ActorsParent.cpp:5062 WIN32(0x570)
dom/quota/ActorsParent.cpp:5065 WIN32(0x570)
dom/quota/ActorsParent.cpp:4529 WIN32(0x570)
dom/quota/ActorsParent.cpp:4532 WIN32(0x570)
dom/quota/ActorsParent.cpp:6526 WIN32(0x570)
7 2329 dom/indexeddb/ActorsParent.cpp:13441 WIN32(0x570)
dom/indexeddb/ActorsParent.cpp:13491 WIN32(0x570)
dom/indexeddb/ActorsParent.cpp:13092 WIN32(0x570)
dom/indexeddb/ActorsParent.cpp:13051 WIN32(0x570)
dom/quota/ActorsParent.cpp:5187 WIN32(0x570)
dom/quota/ActorsParent.cpp:5227 WIN32(0x570)
dom/quota/ActorsParent.cpp:5231 WIN32(0x570)
dom/quota/ActorsParent.cpp:5234 WIN32(0x570)
dom/quota/ActorsParent.cpp:5054 WIN32(0x570)
dom/quota/ActorsParent.cpp:5058 WIN32(0x570)
dom/quota/ActorsParent.cpp:5062 WIN32(0x570)
dom/quota/ActorsParent.cpp:5065 WIN32(0x570)
dom/quota/ActorsParent.cpp:4529 WIN32(0x570)
dom/quota/ActorsParent.cpp:4532 WIN32(0x570)
dom/quota/ActorsParent.cpp:6526 WIN32(0x570)
6 1028 dom/cache/FileUtilsImpl.h:42 WIN32(0x570)
dom/cache/FileUtilsImpl.h:83 WIN32(0x570)
dom/cache/QuotaClient.cpp:109 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:111 WIN32(0x570)
dom/cache/QuotaClient.cpp:423 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:457 WIN32(0x570)
dom/quota/ActorsParent.cpp:5187 WIN32(0x570)
dom/quota/ActorsParent.cpp:5227 WIN32(0x570)
dom/quota/ActorsParent.cpp:5231 WIN32(0x570)
dom/quota/ActorsParent.cpp:5234 WIN32(0x570)
dom/quota/ActorsParent.cpp:5054 WIN32(0x570)
dom/quota/ActorsParent.cpp:5058 WIN32(0x570)
dom/quota/ActorsParent.cpp:5062 WIN32(0x570)
dom/quota/ActorsParent.cpp:5065 WIN32(0x570)
dom/quota/ActorsParent.cpp:4529 WIN32(0x570)
dom/quota/ActorsParent.cpp:4532 WIN32(0x570)
dom/quota/ActorsParent.cpp:6526 WIN32(0x570)
6 31 dom/quota/QuotaCommon.cpp:194 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:4477 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:4494 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:6526 NS_ERROR_STORAGE_IOERR
6 14 dom/cache/QuotaClient.cpp:419 NS_ERROR_FILE_NOT_FOUND # Bug 1689657
dom/cache/QuotaClient.cpp:49 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:52 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:457 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5187 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5227 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5231 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5234 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5054 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5058 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5062 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5065 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:4529 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:4532 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FILE_NOT_FOUND
6 1868 dom/quota/QuotaCommon.h:1174 WIN32(0x570)
dom/indexeddb/ActorsParent.cpp:13491 WIN32(0x570)
dom/indexeddb/ActorsParent.cpp:13092 WIN32(0x570)
dom/indexeddb/ActorsParent.cpp:13051 WIN32(0x570)
dom/quota/ActorsParent.cpp:5187 WIN32(0x570)
dom/quota/ActorsParent.cpp:5227 WIN32(0x570)
dom/quota/ActorsParent.cpp:5231 WIN32(0x570)
dom/quota/ActorsParent.cpp:5234 WIN32(0x570)
dom/quota/ActorsParent.cpp:5054 WIN32(0x570)
dom/quota/ActorsParent.cpp:5058 WIN32(0x570)
dom/quota/ActorsParent.cpp:5062 WIN32(0x570)
dom/quota/ActorsParent.cpp:5065 WIN32(0x570)
dom/quota/ActorsParent.cpp:4529 WIN32(0x570)
dom/quota/ActorsParent.cpp:4532 WIN32(0x570)
dom/quota/ActorsParent.cpp:6526 WIN32(0x570)
5 688 dom/quota/QuotaCommon.h:1209 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:457 WIN32(0x570)
dom/quota/ActorsParent.cpp:5187 WIN32(0x570)
dom/quota/ActorsParent.cpp:5227 WIN32(0x570)
dom/quota/ActorsParent.cpp:5231 WIN32(0x570)
dom/quota/ActorsParent.cpp:5234 WIN32(0x570)
dom/quota/ActorsParent.cpp:5054 WIN32(0x570)
dom/quota/ActorsParent.cpp:5058 WIN32(0x570)
dom/quota/ActorsParent.cpp:5062 WIN32(0x570)
dom/quota/ActorsParent.cpp:5065 WIN32(0x570)
dom/quota/ActorsParent.cpp:4529 WIN32(0x570)
dom/quota/ActorsParent.cpp:4532 WIN32(0x570)
dom/quota/ActorsParent.cpp:6526 WIN32(0x570)
5 1806 dom/quota/QuotaCommon.cpp:180 WIN32(0x570)
dom/cache/QuotaClient.cpp:64 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:111 WIN32(0x570)
dom/cache/QuotaClient.cpp:423 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:457 WIN32(0x570)
dom/quota/ActorsParent.cpp:5187 WIN32(0x570)
dom/quota/ActorsParent.cpp:5227 WIN32(0x570)
dom/quota/ActorsParent.cpp:5231 WIN32(0x570)
dom/quota/ActorsParent.cpp:5234 WIN32(0x570)
dom/quota/ActorsParent.cpp:5054 WIN32(0x570)
dom/quota/ActorsParent.cpp:5058 WIN32(0x570)
dom/quota/ActorsParent.cpp:5062 WIN32(0x570)
dom/quota/ActorsParent.cpp:5065 WIN32(0x570)
dom/quota/ActorsParent.cpp:4529 WIN32(0x570)
dom/quota/ActorsParent.cpp:4532 WIN32(0x570)
dom/quota/ActorsParent.cpp:6526 WIN32(0x570)
5 7 dom/quota/ActorsParent.cpp:4850 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:4909 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5018 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5058 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5062 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5065 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:4529 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:4532 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FAILURE
Initialization::Storage
Clients Sessions Location Result
37 21340 dom/quota/ActorsParent.cpp:462 NS_ERROR_FILE_CORRUPTED # Bug 1689636
dom/quota/ActorsParent.cpp:6303 NS_ERROR_FILE_CORRUPTED
24 28339 dom/quota/ActorsParent.cpp:576 # Bug 1689680
dom/quota/ActorsParent.cpp:6185 NS_ERROR_FILE_CORRUPTED
19 50884 dom/quota/ActorsParent.cpp:546 NS_ERROR_FILE_CORRUPTED # Bug 1689680
dom/quota/ActorsParent.cpp:5985 NS_ERROR_FILE_CORRUPTED
dom/quota/ActorsParent.cpp:6208 NS_ERROR_FILE_CORRUPTED
18 19 dom/quota/ActorsParent.cpp:546 NS_ERROR_FILE_CORRUPTED # Bug 1689680
dom/quota/ActorsParent.cpp:6200 NS_ERROR_FILE_CORRUPTED
14 2665 dom/quota/ActorsParent.cpp:504 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:5761 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:5931 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:6176 NS_ERROR_STORAGE_IOERR
12 4122 dom/quota/ActorsParent.cpp:10848 NS_ERROR_FILE_ACCESS_DENIED
dom/quota/ActorsParent.cpp:10028 NS_ERROR_FILE_ACCESS_DENIED
dom/quota/ActorsParent.cpp:10580 NS_ERROR_FILE_ACCESS_DENIED
dom/quota/ActorsParent.cpp:5416 NS_ERROR_FILE_ACCESS_DENIED
dom/quota/ActorsParent.cpp:5440 NS_ERROR_FILE_ACCESS_DENIED
dom/quota/ActorsParent.cpp:6147 NS_ERROR_FILE_ACCESS_DENIED
8 2339 dom/quota/ActorsParent.cpp:6070 NS_ERROR_STORAGE_BUSY
7 7 dom/quota/ActorsParent.cpp:576 # Bug 1689680
7 13 dom/quota/ActorsParent.cpp:546 NS_ERROR_FILE_CORRUPTED # Bug 1689680
dom/quota/ActorsParent.cpp:5985 NS_ERROR_FILE_CORRUPTED
6 6 dom/quota/ActorsParent.cpp:5817 NS_ERROR_FILE_IS_LOCKED
6 6 dom/quota/ActorsParent.cpp:5931 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:6176 NS_ERROR_FILE_IS_LOCKED
6 1033 dom/quota/ActorsParent.cpp:6070 NS_ERROR_STORAGE_IOERR
5 8 dom/quota/ActorsParent.cpp:546 NS_ERROR_FILE_CORRUPTED # Bug 1689680
4 1077 dom/quota/ActorsParent.cpp:5792 WIN32(0x17)
dom/quota/ActorsParent.cpp:5931 WIN32(0x17)
dom/quota/ActorsParent.cpp:6176 WIN32(0x17)
4 4 dom/quota/ActorsParent.cpp:546 NS_ERROR_FAILURE # Bug 1689680
dom/quota/ActorsParent.cpp:6200 NS_ERROR_FAILURE
|
|
||
Comment 31•4 years ago
•
|
||
As an update, these are the failures of beta over the weekend, showing propagating failures affecting more than 3 clients. I couldn't query the numbers the full week for some reason (the query then took excessively long). I can try to combine the results, but maybe that's not necessary:
Initialization::TemporaryStorage
Clients Sessions Location Result
55 67859 dom/quota/ActorsParent.cpp:462 NS_ERROR_FILE_CORRUPTED # Bug 1689636
dom/quota/ActorsParent.cpp:4543 NS_ERROR_FILE_CORRUPTED
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FILE_CORRUPTED
20 47 dom/quota/ActorsParent.cpp:462 NS_ERROR_FILE_CORRUPTED # Bug 1689636
dom/quota/ActorsParent.cpp:4543 NS_ERROR_FILE_CORRUPTED
17 2115 dom/cache/FileUtilsImpl.h:42 WIN32(0x570) # Bug 1690234
dom/cache/FileUtilsImpl.h:83 WIN32(0x570)
dom/cache/QuotaClient.cpp:109 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:111 WIN32(0x570)
dom/cache/QuotaClient.cpp:423 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:457 WIN32(0x570)
dom/quota/ActorsParent.cpp:5187 WIN32(0x570)
dom/quota/ActorsParent.cpp:5227 WIN32(0x570)
dom/quota/ActorsParent.cpp:5231 WIN32(0x570)
dom/quota/ActorsParent.cpp:5234 WIN32(0x570)
dom/quota/ActorsParent.cpp:5054 WIN32(0x570)
dom/quota/ActorsParent.cpp:5058 WIN32(0x570)
dom/quota/ActorsParent.cpp:5062 WIN32(0x570)
dom/quota/ActorsParent.cpp:5065 WIN32(0x570)
dom/quota/ActorsParent.cpp:4529 WIN32(0x570)
dom/quota/ActorsParent.cpp:4532 WIN32(0x570)
dom/quota/ActorsParent.cpp:6526 WIN32(0x570)
14 20 dom/quota/ActorsParent.cpp:462 NS_ERROR_FILE_CORRUPTED # Bug 1689636
14 2952 dom/quota/QuotaCommon.h:1174 WIN32(0x570) # Bug 1690234
dom/cache/FileUtilsImpl.h:83 WIN32(0x570)
dom/cache/QuotaClient.cpp:109 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:111 WIN32(0x570)
dom/cache/QuotaClient.cpp:423 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:457 WIN32(0x570)
dom/quota/ActorsParent.cpp:5187 WIN32(0x570)
dom/quota/ActorsParent.cpp:5227 WIN32(0x570)
dom/quota/ActorsParent.cpp:5231 WIN32(0x570)
dom/quota/ActorsParent.cpp:5234 WIN32(0x570)
dom/quota/ActorsParent.cpp:5054 WIN32(0x570)
dom/quota/ActorsParent.cpp:5058 WIN32(0x570)
dom/quota/ActorsParent.cpp:5062 WIN32(0x570)
dom/quota/ActorsParent.cpp:5065 WIN32(0x570)
dom/quota/ActorsParent.cpp:4529 WIN32(0x570)
dom/quota/ActorsParent.cpp:4532 WIN32(0x570)
dom/quota/ActorsParent.cpp:6526 WIN32(0x570)
13 22 dom/indexeddb/ActorsParent.cpp:13441 NS_ERROR_FILE_NOT_FOUND # Bug 1689655
dom/indexeddb/ActorsParent.cpp:13491 NS_ERROR_FILE_NOT_FOUND
dom/indexeddb/ActorsParent.cpp:13092 NS_ERROR_FILE_NOT_FOUND
dom/indexeddb/ActorsParent.cpp:13051 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5187 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5227 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5231 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5234 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5054 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5058 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5062 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5065 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:4529 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:4532 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FILE_NOT_FOUND
13 40 dom/cache/FileUtilsImpl.h:42 NS_ERROR_FILE_NOT_FOUND # Bug 1689656
dom/cache/FileUtilsImpl.h:83 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:109 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:49 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:52 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:111 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:423 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:49 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:52 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:457 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5187 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5227 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5231 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5234 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5054 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5058 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5062 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5065 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:4529 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:4532 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FILE_NOT_FOUND
10 6350 dom/indexeddb/ActorsParent.cpp:13441 WIN32(0x570) # Bug 1690234
dom/indexeddb/ActorsParent.cpp:13491 WIN32(0x570)
dom/indexeddb/ActorsParent.cpp:13092 WIN32(0x570)
dom/indexeddb/ActorsParent.cpp:13051 WIN32(0x570)
dom/quota/ActorsParent.cpp:5187 WIN32(0x570)
dom/quota/ActorsParent.cpp:5227 WIN32(0x570)
dom/quota/ActorsParent.cpp:5231 WIN32(0x570)
dom/quota/ActorsParent.cpp:5234 WIN32(0x570)
dom/quota/ActorsParent.cpp:5054 WIN32(0x570)
dom/quota/ActorsParent.cpp:5058 WIN32(0x570)
dom/quota/ActorsParent.cpp:5062 WIN32(0x570)
dom/quota/ActorsParent.cpp:5065 WIN32(0x570)
dom/quota/ActorsParent.cpp:4529 WIN32(0x570)
dom/quota/ActorsParent.cpp:4532 WIN32(0x570)
dom/quota/ActorsParent.cpp:6526 WIN32(0x570)
10 2905 dom/quota/QuotaCommon.h:1174 WIN32(0x570) # Bug 1671932 (This is not happening during a directory scan but when initializing a specific origin)
dom/indexeddb/ActorsParent.cpp:13491 WIN32(0x570)
dom/indexeddb/ActorsParent.cpp:13092 WIN32(0x570)
dom/indexeddb/ActorsParent.cpp:13051 WIN32(0x570)
dom/quota/ActorsParent.cpp:5187 WIN32(0x570)
dom/quota/ActorsParent.cpp:5227 WIN32(0x570)
dom/quota/ActorsParent.cpp:5231 WIN32(0x570)
dom/quota/ActorsParent.cpp:5234 WIN32(0x570)
dom/quota/ActorsParent.cpp:5054 WIN32(0x570)
dom/quota/ActorsParent.cpp:5058 WIN32(0x570)
dom/quota/ActorsParent.cpp:5062 WIN32(0x570)
dom/quota/ActorsParent.cpp:5065 WIN32(0x570)
dom/quota/ActorsParent.cpp:4529 WIN32(0x570)
dom/quota/ActorsParent.cpp:4532 WIN32(0x570)
dom/quota/ActorsParent.cpp:6526 WIN32(0x570)
9 2575 dom/localstorage/ActorsParent.cpp:956 WIN32(0x570) # Bug 1671932 (This is not happening during a directory scan but when initializing a specific origin)
dom/localstorage/ActorsParent.cpp:8320 WIN32(0x570)
dom/quota/ActorsParent.cpp:5187 WIN32(0x570)
dom/quota/ActorsParent.cpp:5227 WIN32(0x570)
dom/quota/ActorsParent.cpp:5231 WIN32(0x570)
dom/quota/ActorsParent.cpp:5234 WIN32(0x570)
dom/quota/ActorsParent.cpp:5054 WIN32(0x570)
dom/quota/ActorsParent.cpp:5058 WIN32(0x570)
dom/quota/ActorsParent.cpp:5062 WIN32(0x570)
dom/quota/ActorsParent.cpp:5065 WIN32(0x570)
dom/quota/ActorsParent.cpp:4529 WIN32(0x570)
dom/quota/ActorsParent.cpp:4532 WIN32(0x570)
dom/quota/ActorsParent.cpp:6526 WIN32(0x570)
8 13 dom/cache/QuotaClient.cpp:419 NS_ERROR_FILE_NOT_FOUND # Bug 1689657
dom/cache/QuotaClient.cpp:49 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:52 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:457 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5187 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5227 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5231 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5234 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5054 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5058 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5062 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5065 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:4529 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:4532 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FILE_NOT_FOUND
8 47 dom/quota/ActorsParent.cpp:2557 NS_ERROR_FAILURE # TODO Creating directory metadata
dom/quota/ActorsParent.cpp:11175 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:10028 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:11163 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:4834 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:4907 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5018 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5058 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5062 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5065 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:4529 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:4532 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FAILURE
7 139 dom/quota/QuotaCommon.cpp:194 NS_ERROR_STORAGE_IOERR # TODO Reading QM Cache
dom/quota/ActorsParent.cpp:4477 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:4494 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:6526 NS_ERROR_STORAGE_IOERR
7 545 dom/quota/ActorsParent.cpp:462 NS_ERROR_FILE_ACCESS_DENIED # Bug 1689636 (strange error at this point)
dom/quota/ActorsParent.cpp:4543 NS_ERROR_FILE_ACCESS_DENIED
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FILE_ACCESS_DENIED
6 7 dom/localstorage/ActorsParent.cpp:493 NS_ERROR_STORAGE_IOERR # Bug 1690291
dom/localstorage/ActorsParent.cpp:8359 NS_ERROR_STORAGE_IOERR
dom/localstorage/ActorsParent.cpp:8372 NS_ERROR_STORAGE_IOERR
dom/localstorage/ActorsParent.cpp:8380 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:5187 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:5227 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:5231 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:5234 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:5054 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:5058 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:5062 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:5065 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:4529 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:4532 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:6526 NS_ERROR_STORAGE_IOERR
6 147 dom/quota/ActorsParent.cpp:462 NS_ERROR_FILE_NO_DEVICE_SPACE # Bug 1689636
dom/quota/ActorsParent.cpp:4543 NS_ERROR_FILE_NO_DEVICE_SPACE
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FILE_NO_DEVICE_SPACE
5 21 dom/quota/ActorsParent.cpp:4850 NS_ERROR_FAILURE # TODO Reading directory metadata
dom/quota/ActorsParent.cpp:4909 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5018 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5058 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5062 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:5065 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:4529 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:4532 NS_ERROR_FAILURE
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FAILURE
5 6 dom/quota/ActorsParent.cpp:462 NS_ERROR_STORAGE_IOERR # Bug 1689636
dom/quota/ActorsParent.cpp:4543 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:6526 NS_ERROR_STORAGE_IOERR
5 1751 dom/quota/QuotaCommon.cpp:180 WIN32(0x570) # Bug 1690234
dom/cache/QuotaClient.cpp:64 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:111 WIN32(0x570)
dom/cache/QuotaClient.cpp:423 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:457 WIN32(0x570)
dom/quota/ActorsParent.cpp:5187 WIN32(0x570)
dom/quota/ActorsParent.cpp:5227 WIN32(0x570)
dom/quota/ActorsParent.cpp:5231 WIN32(0x570)
dom/quota/ActorsParent.cpp:5234 WIN32(0x570)
dom/quota/ActorsParent.cpp:5054 WIN32(0x570)
dom/quota/ActorsParent.cpp:5058 WIN32(0x570)
dom/quota/ActorsParent.cpp:5062 WIN32(0x570)
dom/quota/ActorsParent.cpp:5065 WIN32(0x570)
dom/quota/ActorsParent.cpp:4529 WIN32(0x570)
dom/quota/ActorsParent.cpp:4532 WIN32(0x570)
dom/quota/ActorsParent.cpp:6526 WIN32(0x570)
5 7 dom/quota/QuotaCommon.h:1174 WIN32(0x570) # Bug 1690234
dom/cache/FileUtils.cpp:476 WIN32(0x570)
4 4 dom/cache/FileUtilsImpl.h:42 WIN32(0x570) # Bug 1690234
dom/cache/FileUtilsImpl.h:83 WIN32(0x570)
dom/cache/QuotaClient.cpp:109 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:111 WIN32(0x570)
dom/cache/QuotaClient.cpp:423 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:457 WIN32(0x570)
dom/quota/ActorsParent.cpp:5187 WIN32(0x570)
4 6 dom/quota/QuotaCommon.h:1174 WIN32(0x570) # Bug 1690234
dom/cache/FileUtilsImpl.h:83 WIN32(0x570)
dom/cache/QuotaClient.cpp:109 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:111 WIN32(0x570)
4 9 dom/cache/DBSchema.cpp:517 NS_ERROR_FILE_READ_ONLY # TODO Strange error at this point, shouldn't we know this when opening the database? NOTE Stack is continued below
dom/cache/DBAction.cpp:209 NS_ERROR_FILE_READ_ONLY
4 9 dom/cache/QuotaClient.cpp:141 NS_ERROR_FILE_READ_ONLY
dom/cache/QuotaClient.cpp:391 NS_ERROR_FILE_READ_ONLY
dom/cache/QuotaClient.cpp:401 NS_ERROR_FILE_READ_ONLY
dom/quota/ActorsParent.cpp:5187 NS_ERROR_FILE_READ_ONLY
dom/quota/ActorsParent.cpp:5227 NS_ERROR_FILE_READ_ONLY
dom/quota/ActorsParent.cpp:5231 NS_ERROR_FILE_READ_ONLY
dom/quota/ActorsParent.cpp:5234 NS_ERROR_FILE_READ_ONLY
dom/quota/ActorsParent.cpp:5054 NS_ERROR_FILE_READ_ONLY
dom/quota/ActorsParent.cpp:5058 NS_ERROR_FILE_READ_ONLY
dom/quota/ActorsParent.cpp:5062 NS_ERROR_FILE_READ_ONLY
dom/quota/ActorsParent.cpp:5065 NS_ERROR_FILE_READ_ONLY
dom/quota/ActorsParent.cpp:4529 NS_ERROR_FILE_READ_ONLY
dom/quota/ActorsParent.cpp:4532 NS_ERROR_FILE_READ_ONLY
dom/quota/ActorsParent.cpp:6526 NS_ERROR_FILE_READ_ONLY
Initialization::Storage
Clients Sessions Location Result
39 18152 dom/quota/ActorsParent.cpp:462 NS_ERROR_FILE_CORRUPTED # Bug 1689636
dom/quota/ActorsParent.cpp:6303 NS_ERROR_FILE_CORRUPTED
36 49133 dom/quota/ActorsParent.cpp:576 # Bug 1689680
dom/quota/ActorsParent.cpp:6185 NS_ERROR_FILE_CORRUPTED
24 5573 dom/quota/ActorsParent.cpp:10848 NS_ERROR_FILE_ACCESS_DENIED # TODO Upgrade 0.0 to 1.0, renaming origin directory
dom/quota/ActorsParent.cpp:10028 NS_ERROR_FILE_ACCESS_DENIED
dom/quota/ActorsParent.cpp:10580 NS_ERROR_FILE_ACCESS_DENIED
dom/quota/ActorsParent.cpp:5416 NS_ERROR_FILE_ACCESS_DENIED
dom/quota/ActorsParent.cpp:5440 NS_ERROR_FILE_ACCESS_DENIED
dom/quota/ActorsParent.cpp:6147 NS_ERROR_FILE_ACCESS_DENIED
21 52789 dom/quota/ActorsParent.cpp:546 NS_ERROR_FILE_CORRUPTED # Bug 1689680
dom/quota/ActorsParent.cpp:5985 NS_ERROR_FILE_CORRUPTED
dom/quota/ActorsParent.cpp:6208 NS_ERROR_FILE_CORRUPTED
11 12 dom/quota/ActorsParent.cpp:576 # Bug 1689680
9 9 dom/quota/ActorsParent.cpp:5817 NS_ERROR_FILE_IS_LOCKED # TODO CreateLocalStorageArchiveConnectionFromWebAppsStore, renaming temp file to archive, probably (implicit) close failed before
dom/quota/ActorsParent.cpp:5931 NS_ERROR_FILE_IS_LOCKED
dom/quota/ActorsParent.cpp:6176 NS_ERROR_FILE_IS_LOCKED
9 9 dom/quota/ActorsParent.cpp:546 NS_ERROR_FILE_CORRUPTED # Bug 1689680
8 226 dom/quota/ActorsParent.cpp:6070 NS_ERROR_STORAGE_IOERR # Bug 1690291
7 9 dom/quota/ActorsParent.cpp:546 NS_ERROR_FILE_CORRUPTED # Bug 1689680
dom/quota/ActorsParent.cpp:6200 NS_ERROR_FILE_CORRUPTED
7 418 dom/quota/ActorsParent.cpp:504 NS_ERROR_STORAGE_IOERR # Bug 1690291
dom/quota/ActorsParent.cpp:5761 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:5931 NS_ERROR_STORAGE_IOERR
dom/quota/ActorsParent.cpp:6176 NS_ERROR_STORAGE_IOERR
6 9 dom/quota/ActorsParent.cpp:546 NS_ERROR_FILE_CORRUPTED # Bug 1689680
dom/quota/ActorsParent.cpp:5985 NS_ERROR_FILE_CORRUPTED
6 6 dom/quota/ActorsParent.cpp:462 NS_ERROR_FILE_CORRUPTED # Bug 1689636
5 6 dom/quota/ActorsParent.cpp:5901 NS_ERROR_STORAGE_IOERR # Bug 1690291
dom/quota/ActorsParent.cpp:6176 NS_ERROR_STORAGE_IOERR
4 11 dom/quota/ActorsParent.cpp:5901 NS_ERROR_STORAGE_BUSY
dom/quota/ActorsParent.cpp:6176 NS_ERROR_STORAGE_BUSY
|
|
||
Comment 32•4 years ago
|
||
Errors from the weekend Nightlies so far, after some fixes landed:
Initialization::TemporaryStorage
Clients Sessions Location Result
1 1 dom/cache/QuotaClient.cpp:419 NS_ERROR_FILE_NOT_FOUND # Bug 1687321
dom/cache/QuotaClient.cpp:49 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:52 NS_ERROR_FILE_NOT_FOUND
dom/cache/QuotaClient.cpp:457 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5296 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5336 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5163 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:5167 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:4667 NS_ERROR_FILE_NOT_FOUND
dom/quota/ActorsParent.cpp:6554 NS_ERROR_FILE_NOT_FOUND
1 1 dom/cache/FileUtilsImpl.h:42 WIN32(0x570) # Bug 1687321
dom/cache/FileUtilsImpl.h:83 WIN32(0x570)
dom/cache/QuotaClient.cpp:109 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:111 WIN32(0x570)
dom/cache/QuotaClient.cpp:423 WIN32(0x570)
dom/cache/QuotaClient.cpp:49 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:457 WIN32(0x570)
dom/quota/ActorsParent.cpp:5296 WIN32(0x570)
dom/quota/ActorsParent.cpp:5336 WIN32(0x570)
dom/quota/ActorsParent.cpp:5163 WIN32(0x570)
dom/quota/ActorsParent.cpp:5167 WIN32(0x570)
Both of these issues have already fixes that have not landed yet.
|
|
||
Comment 33•4 years ago
•
|
||
Errors from the weekend Nightlies:
Initialization::TemporaryStorage
Clients Sessions Location Result
2 2 dom/quota/ActorsParent.cpp:552 NS_ERROR_FILE_NO_DEVICE_SPACE (TS.A)
dom/quota/ActorsParent.cpp:555 NS_ERROR_FILE_NO_DEVICE_SPACE
dom/quota/ActorsParent.cpp:4200 NS_ERROR_FILE_NO_DEVICE_SPACE
dom/quota/ActorsParent.cpp:6156 NS_ERROR_FILE_NO_DEVICE_SPACE
1 4 dom/quota/QuotaCommon.cpp:201 NS_ERROR_FILE_CORRUPTED (TS.B)
dom/localstorage/ActorsParent.cpp:2881 NS_ERROR_FILE_CORRUPTED
dom/localstorage/ActorsParent.cpp:2882 NS_ERROR_FILE_CORRUPTED
dom/localstorage/ActorsParent.cpp:8139 NS_ERROR_FILE_CORRUPTED
dom/localstorage/ActorsParent.cpp:8147 NS_ERROR_FILE_CORRUPTED
dom/localstorage/ActorsParent.cpp:8155 NS_ERROR_FILE_CORRUPTED
dom/quota/ActorsParent.cpp:4810 NS_ERROR_FILE_CORRUPTED
dom/quota/ActorsParent.cpp:4882 NS_ERROR_FILE_CORRUPTED
dom/quota/ActorsParent.cpp:4678 NS_ERROR_FILE_CORRUPTED
dom/quota/ActorsParent.cpp:4701 NS_ERROR_FILE_CORRUPTED
dom/quota/ActorsParent.cpp:4186 NS_ERROR_FILE_CORRUPTED
dom/quota/ActorsParent.cpp:6156 NS_ERROR_FILE_CORRUPTED
1 3 dom/localstorage/ActorsParent.cpp:973 WIN32(0x570) (TS.C)
dom/localstorage/ActorsParent.cpp:8095 WIN32(0x570)
dom/quota/ActorsParent.cpp:4810 WIN32(0x570)
dom/quota/ActorsParent.cpp:4882 WIN32(0x570)
dom/quota/ActorsParent.cpp:4678 WIN32(0x570)
dom/quota/ActorsParent.cpp:4701 WIN32(0x570)
dom/quota/ActorsParent.cpp:4186 WIN32(0x570)
dom/quota/ActorsParent.cpp:6156 WIN32(0x570)
Initialization::Storage
Clients Sessions Location Result
3 4 dom/quota/ActorsParent.cpp:5932 NS_ERROR_STORAGE_BUSY (S.A)
Some details:
- TS.A
- Module: Quota Manager
- Description: In
InvalidateCache, deleting the cache failed withNS_ERROR_FILE_NO_DEVICE_SPACE, subsequently also setting the invalid flag failed, probably because the physical device has no space left for writing the journal. - Permanence: Temporary, but might require user intervention to resolve
- Mitigation options: Evict some origins, Notify storage pressure
- TS.B
- Module: Local Storage
- Description: After loading the usage file failed, we encounter an
NS_ERROR_FILE_CORRUPTEDerror inGetUsage. - Permanence: Probably permanent
- Mitigation options: Clear affected origin
- TS.C
- Module: Local Storage
- Description: Checking existence of the usage file fails due to a
ERROR_FILE_CORRUPTerror. - Permanence: Unknown
- Mitigation options: Treat usage file as non-existent (but probably updating the usage file later will fail as well)
- S.A
- Module: Quota Manager
- Description: Opening the storage database file fails with
NS_ERROR_STORAGE_BUSY. - Permanence: Probably temporary
- Mitigation options: Retry
|
|
Reporter | |
Comment 34•4 years ago
|
||
(In reply to Simon Giesecke [:sg] [he/him] from comment #33)
Some details:
- TS.A
- Module: Quota Manager
- Description: In
InvalidateCache, deleting the cache failed withNS_ERROR_FILE_NO_DEVICE_SPACE, subsequently also setting the invalid flag failed, probably because the physical device has no space left for writing the journal.- Permanence: Temporary, but might require user intervention to resolve
- Mitigation options: Evict some origins, Notify storage pressure
Yes, it seems we can move https://searchfox.org/mozilla-central/rev/f47a4b67643b3048ef9a2e2ac0c34edf6d1ebff3/dom/quota/ActorsParent.cpp#4199-4201 to https://searchfox.org/mozilla-central/rev/f47a4b67643b3048ef9a2e2ac0c34edf6d1ebff3/dom/quota/ActorsParent.cpp#6166
But I will have to double check during review.
- TS.B
- Module: Local Storage
- Description: After loading the usage file failed, we encounter an
NS_ERROR_FILE_CORRUPTEDerror inGetUsage.- Permanence: Probably permanent
- Mitigation options: Clear affected origin
Yes, luckily, the temporary storage is only being initialized (not in use) so we can just add orElse here: https://searchfox.org/mozilla-central/rev/f47a4b67643b3048ef9a2e2ac0c34edf6d1ebff3/dom/quota/ActorsParent.cpp#4672
The orElse will remove entire origin directory.
However, I wonder if we should also use the new CallWithDelayedRetriesIfAccessDenied method there.
I'll take a look at the rest in a minute.
|
|
Reporter | |
Comment 35•4 years ago
|
||
(In reply to Simon Giesecke [:sg] [he/him] from comment #33)
- TS.C
- Module: Local Storage
- Description: Checking existence of the usage file fails due to a
ERROR_FILE_CORRUPTerror.- Permanence: Unknown
- Mitigation options: Treat usage file as non-existent (but probably updating the usage file later will fail as well)
Ok, this is again FS corruption and we don't have a good solution for that yet...
- S.A
- Module: Quota Manager
- Description: Opening the storage database file fails with
NS_ERROR_STORAGE_BUSY.- Permanence: Probably temporary
- Mitigation options: Retry
Yes, we actually do that in IDB (but because some of our threads may be causing it), for example https://searchfox.org/mozilla-central/rev/f47a4b67643b3048ef9a2e2ac0c34edf6d1ebff3/dom/indexedDB/ActorsParent.cpp#6886
It seems that CallWithDelayedRetriesIfAccessDenied needs to be generalized even more.
|
|
||
Comment 36•4 years ago
|
||
Updated script with more recent build id
Attachment #9198601 -
Attachment is obsolete: true
|
|
Reporter | |
Comment 37•4 years ago
•
|
||
Errors from the weekend Nightlies:
build ids: 20210227094458/20210227214359/20210228103709/20210228215216
changeset: https://hg.mozilla.org/mozilla-central/rev/1add9a1bbfacd4ff1bee961fa8886b8cd0566d23
query: https://sql.telemetry.mozilla.org/queries/78426/source
Initialization::TemporaryStorage
Clients Sessions Location Result
4 6 dom/cache/FileUtils.cpp:487 NS_ERROR_FILE_DIR_NOT_EMPTY
3 29 dom/cache/FileUtils.cpp:487 NS_ERROR_FILE_ACCESS_DENIED
1 30 dom/indexeddb/ActorsParent.cpp:5794 NS_ERROR_FILE_ACCESS_DENIED
dom/indexeddb/ActorsParent.cpp:12823 NS_ERROR_FILE_ACCESS_DENIED
dom/indexeddb/ActorsParent.cpp:12827 NS_ERROR_UNEXPECTED
1 1 dom/quota/ActorsParent.cpp:544 NS_ERROR_FILE_ACCESS_DENIED
1 2 dom/cache/FileUtils.cpp:519 NS_ERROR_FILE_ACCESS_DENIED
1 1 dom/quota/QuotaCommon.h:1212 WIN32(0x570)
dom/cache/QuotaClient.cpp:52 WIN32(0x570)
dom/cache/QuotaClient.cpp:478 WIN32(0x570)
dom/quota/ActorsParent.cpp:4825 WIN32(0x570)
dom/quota/ActorsParent.cpp:4897 WIN32(0x570)
dom/quota/ActorsParent.cpp:4693 WIN32(0x570)
dom/quota/ActorsParent.cpp:4716 WIN32(0x570)
dom/quota/ActorsParent.cpp:4187 WIN32(0x570)
dom/quota/ActorsParent.cpp:6172 WIN32(0x570)
I'll double check if I got the "not_propagate" list right.
It seems, all these errors (except the last one) are related to the interference with anti-viruses, so we should probably add the retrying there.
|
|
||
Comment 38•2 years ago
|
||
This is monitored but not actively worked on if not in some of the blocking bugs.
Assignee: jvarga → nobody
Status: ASSIGNED → NEW
You need to log in
before you can comment on or make changes to this bug.
Description
•