1677736 - Have a about page for dom storage for troubleshooting

Status: ASSIGNED

(Core :: Storage: Quota Manager, task, P3)

Description

[Tom Tung [:tt, :ttung]]

While someone reported an issue related to storage, we often ask them to visit https://firefox-storage-test.glitch.me/ and paste the result on the bug.

It's useful in the sense that we can understand if the storage initialization succeeded or not, and we can even have a ballpark figure about where it failed by combining the information at about:telemetry.

However, by only getting information on that page and about:telemetry:

• We cannot differentiate if there is an issue on DOMCache for the user or the user installs an extension to block third-party resources.
• Sometimes, we ask the user to run a debug build to get more information.

Another thing is that I have seen some extension developers ask their users to remove their storage directory or even reset their profile to fix their broken storage. That's sad because we were worried about whether removing storage for broken origins would accidentally remove some important data for users.

So, we might want to roughly have a page named about:storage that provides abilities:

• To show the status of storage like troubleshooting in about:support.
  • For instance, whether storage/temporary storage has been initialized yet.
• To print logging from QM_TRY things like about:networking#logging.
• To show why storage initialization fails or even a list of broken origins.
• To remove the broken origin if there is one by a button.

This is just a rough idea and I haven't thought about this and its implementation deeply yet but I will update a detailed plan (and ask for feedback) once I finish that.

Comment 1

[Tom Tung [:tt, :ttung]]

The issues that I want to resolve or mitigate are:

1. We couldn't entirely rely on the result on https://firefox-storage-test.glitch.me/ since it might be affected by extensions. Also, what I often wanted to check initially was whether the temporary storage is initialized yet. If not, the issue might be related to initialization failures. If so, then the issue is probably related to storage for the origin itself.
2. Downloading debug build and run it might be a non-trivial task for some users.
3. While encountering storage breakage, some users tend to reset their profile.

So, the goals that I want to achieve here are:
A. Provide an initial troubleshooting page for QM initialization failures.
B. Provide a way to dump QM_TRY's log into a file.
C. Provide a way for users to remove a minimum amount of storage for recovering from storage breakage if that's possible.

For the (A), we generally want to check:

• Has the storage been initialized yet?
  • If not, where it failed?
• Has the temporary storagebeen initialized yet?
  • If not, where it failed?

These can be done by collecting results for storageInitialized, and temporaryStorageInitialized and logging from QM_TRYs. If we don't want to make it depends on QM_TRYs logging because that might take time to be shipped to Release, we can create a class that maintains phases just like shutdown blockers so that we would know which phase we were in while failing to initialize.

After the temporary storage is initialized, we might want to check info for origins:

• Persistence type (Persistent or best-effort)
• Group
• Origin
• LeafName (in case we have an issue around string sanitization)

For (B), we probably steal logic from about:networking#logging and customize for QM_TRY stuff. We can add some functions for reporting tickets in Bugzilla. For instance, hashing some sensitive data, focusing logging from a quota client and quota manager by filtering out logs from other clients, ... etc.

For (C), I am thinking we can achieve that by adding another argument in initTemporaryStorage() so check it while traversing directories.

Page might appromixately look like:
Initial version:

• TroubleShooting:
  • Storage 's and temporaryStorage's status:
    • Storage: ${IsStorageInitialized}/${PharseDuringLastInit}
    • TemporaryStorage: ${IsTemporaryStorageInitialized}/${PharseDuringLastInit}
  • Origins' status:
    • ${PersistenceType}
    • ${GroupName}
    • ${OriginName}
    • ${DirectoryName}
  • Diagnosis:
    • Logging (QM_TRYs)
      • Start/Stop
      • Initialize Tempeorary Storage and record logging
  • InitStorage if possible (by removing blockers)

Note:

• To avoid performance issues, the origins' status would only be present if the temporary storage has been initialized and their info is got from the cached value (OriginInfos).

Jan and Andrew, what do you think about this?

Comment 2

[Andrew Sutherland [:asuth] (he/him)]

This seems like a great plan. I'd suggest deferring (B) initially, at least in terms of the back-end of the logging and presenting the logs to the page. It absolutely makes sense to me that QM_TRY serve as the origin of the logs. Specifically, bug 1506892 is adding an in-memory (compressed) circular logging mechanism that I think provides better ergonomics for then getting the data and displaying it in an about page, and it should be landing in the short term. I think that's a good candidate for the back-end and worth discussing once it gets through review.

Comment 3

[Tom Tung [:tt, :ttung]]

(In reply to Andrew Sutherland [:asuth] (he/him) from comment #2)

This seems like a great plan. I'd suggest deferring (B) initially, at least in terms of the back-end of the logging and presenting the logs to the page. It absolutely makes sense to me that QM_TRY serve as the origin of the logs. Specifically, bug 1506892 is adding an in-memory (compressed) circular logging mechanism that I think provides better ergonomics for then getting the data and displaying it in an about page, and it should be landing in the short term. I think that's a good candidate for the back-end and worth discussing once it gets through review.

Thanks for the feedback! I will definitely read the patch before starting to work on (B) since I believe the in-memory compressed circular logging mechanism is highly likely to be re-used here.

Comment 4

[Harveer Singh]

Attachment: Bug 1677736: Introducing a new about page 'about:storage'. r=#dom-storage-reviewers,janv,jstutte

Comment 5

[Jan Varga [:janv]]

Yeah, the plan looks good at first glance, but if someone decides to take this bug and continue further, the plan should be revised and reviewed based on current situation (for example storage initialization failures seem to be under control now).

Comment 6

[Jan Varga [:janv]]

Actually, Harveer already took this bug, we'll get back to this once we have more time for it.

Comment 7

[Jan Varga [:janv]]

Actually, Harveer already took this bug, we'll get to this once we have more time