Open Bug 1593365 Opened 3 years ago Updated 10 months ago

QuotaManager storage v4

Categories

(Core :: Storage: Quota Manager, enhancement, P3)

enhancement

Tracking

()

People

(Reporter: janv, Unassigned)

References

(Blocks 18 open bugs)

Details

Attachments

(1 file)

After more thinking about storage initialization failures and unknown stuff in /storage, I've come to a conclusion that we should get rid of directory traversals for good. We should keep a list of origins and their mapping to directories on disk in a database. Consistency between the database and stuff on disk would be ensured by marker files. Client directories and stuff in some client directories would be managed the same way.
This would solve many problems (including very long file names) and it would open the door for new functionality and optimizations (like getting rid of .metadata-v2 and origin parser, replace flat repository directory structure with multi level repository directory structure).
We wouldn't have to worry about unknown stuff and performance wouldn't be badly affected by too many unknown files that need to be checked during storage initialization.

The new functionality would include for example better support for private browsing and buckets.

(In reply to Jan Varga [:janv] from comment #1)

The new functionality would include for example better support for private browsing and buckets.

and new SessionStorage (for Fission) which needs to support multiple origin directories on disk for the same origin

Priority: -- → P2
Summary: QM: Get rid of directory traversals → [meta] QuotaManager storage v2
Blocks: 1445464
Blocks: 781982
Blocks: 1594740
Blocks: 1056339
Blocks: 1143003
Blocks: 1482662
Blocks: 1157979
Blocks: 1342415
Depends on: 1395133
Blocks: 1395133
No longer depends on: 1395133
Blocks: 1493006
Blocks: 1512750
Blocks: 1522464
Blocks: 1595002
Blocks: 1595010

I originally thought that everything would follow this schema:
<profile>/storage2/
UUID1/
UUID2/
UUID3/

However, asuth suggested that it might be good to have a special case for private browsing:
<profile>/storage2/
UUID1/
UUID2/
UUID3/

<profile>/storage2/pb/
UUID4/
UUID5/
UUID6/

This way, it would be easy to delete private browsing data without loading mappings from storage.sqlite. This can be especially useful when we implement a separate daemon for deleting private browsing data that would be launched after Firefox shutdown.

Actually the final layout would be a bit different, we need multilevel structure like:
<profile>/storage2/00/
UUID1
UUID2
UUID3

<profile>/storage2/01/
UUID4
UUID5
UUID6

<profile>/storage2/02/
UUID7
UUID8
UUID9

Summary: [meta] QuotaManager storage v2 → [meta] QuotaManager storage v3
See Also: → 1414737

Bumping this to v4 since we can't use v3 because we already used v3 in the past (and then downgraded to 2.1).

Summary: [meta] QuotaManager storage v3 → [meta] QuotaManager storage v4
Blocks: 1592404
Depends on: 1608025

It seems there will only be multiple patches.

Keywords: meta
Summary: [meta] QuotaManager storage v4 → QuotaManager storage v4
Depends on: 1608759
Depends on: 1606318
Depends on: 1594075
Assignee: nobody → jvarga
Status: NEW → ASSIGNED
Blocks: 1343576
Depends on: 1615552
Depends on: 1616003
Depends on: 1617842
No longer depends on: 1608025
No longer depends on: 1616003
No longer depends on: 1608759
No longer depends on: 1606318
No longer depends on: 1617842
No longer depends on: 1615552
No longer depends on: 1594075

The dependency was messy, so I cleaned it up a bit. In general, the main meta bug for addressing storage initialization failures is bug 1482662.
This bug is about QuotaManager storage v4 which is a sub project of bug 1482662.

Severity: normal → S3

We are now focusing on asynchronous storage initialization instead, bug 1671932.
QMv4 might be addressed in H2 of 2021.

Assignee: jvarga → nobody
Status: ASSIGNED → NEW
Priority: P2 → P3
See Also: → 1706006
You need to log in before you can comment on or make changes to this bug.