Stop creating .metadata files

RESOLVED FIXED in Firefox 68

Status

()

task
P2
normal
RESOLVED FIXED
2 months ago
12 days ago

People

(Reporter: janv, Assigned: tt)

Tracking

Trunk
mozilla69
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox68 fixed, firefox69 fixed)

Details

(Whiteboard: [qf:p1:pageload])

Attachments

(1 attachment)

Reporter

Description

2 months ago

We need to reduce number of synchronously created files in bug 1540777.
I'm not sure if we can get rid of calling EnsureOriginIsInitialized which currently creates .metadata and .metadata-v2

We can save some I/O, roughly 50% if we stop creating .metadata files.

Reporter

Updated

2 months ago
Blocks: 1540777
Reporter

Comment 1

2 months ago

Tom, I think you could take this.

Flags: needinfo?(shes050117)
Assignee

Comment 2

2 months ago

Setting this bug as P2 since bug 1540777 is P2.

Assignee: nobody → shes050117
Status: NEW → ASSIGNED
Flags: needinfo?(shes050117)
Priority: -- → P2
Assignee

Comment 3

2 months ago

The places for creating ".metadata":

https://searchfox.org/mozilla-central/search?q=symbol:_ZN7mozilla3dom5quota12_GLOBAL__N_123CreateDirectoryMetadataEP7nsIFilelRK12nsTSubstringIcES8_S8_%2C_ZN7mozilla3dom5quota12_GLOBAL__N_123CreateDirectoryMetadataEP7nsIFilexRK12nsTSubstringIcES8_S8_&redirect=false

Only the top two are not related the upgrade:
dom::quota::(anonymous namespace)::CreateDirectoryMetadataFiles
dom::quota::QuotaManager::GetDirectoryMetadata2

The callers of CreateDirectoryMetadataFiles are:

dom::quota::QuotaManager::EnsureOriginIsInitializedInternal
dom::quota::(anonymous namespace)::PersistOp::DoDirectoryWork

https://searchfox.org/mozilla-central/search?q=symbol:_ZN7mozilla3dom5quota12_GLOBAL__N_128CreateDirectoryMetadataFilesEP7nsIFilebRK12nsTSubstringIcES8_S8_Pl&redirect=false

The callers of GetDirectoryMetadata2 are:
dom::quota::QuotaManager::GetDirectoryMetadata2WithRestore (The callers of it are XXXOp)
dom::indexedDB::(anonymous namespace)::Maintenance::DirectoryWork

https://searchfox.org/mozilla-central/search?q=symbol:_ZN7mozilla3dom5quota12QuotaManager21GetDirectoryMetadata2EP7nsIFilePlPbR12nsTSubstringIcES9_S9_&redirect=false

I'm assuming we don't want to have even an additional minor upgrade at this moment which is riskier. So, I will let existing ".metadata" files exist and just stop creating them in this bug.

If the patch is granted, I will open a follow-up bug to establish another minor upgrade to remove the existing ".metadata" and maybe remove the code for creating that on the old upgrades.

Assignee

Comment 4

2 months ago

The main goal of this issue is to reduce IO during the storage initialization
and stop creating .metadata file since we already have .metadata-v2 file to
track the information for the origin directory. Therefore, this patch only stop
create, write, and read the .metadata file. And, leaving the work for removing
it to the next minor upgrade.

Attachment #9060698 - Attachment description: Bug 1546752 - Stop creating, writing, and reading the .metadata file in QuotaManager expect the upgrades; → Bug 1546752 - Stop creating, writing, and reading the .metadata file in QuotaManager except the upgrades;
Whiteboard: [qf]

(In reply to Tom Tung [:tt, :ttung] from comment #2)

Setting this bug as P2 since bug 1540777 is P2.

FYI, bug 1540777 is now P1

Whiteboard: [qf] → [qf:p1:pageload]
Reporter

Comment 6

Last month

Tom, can you update this patch ? Thanks.

Assignee

Comment 7

Last month

(In reply to Jan Varga [:janv] from comment #6)

Tom, can you update this patch ? Thanks.

Jan, I have updated it. Thanks for the notice!

Reporter

Comment 8

Last month

Ok, I'll take a look.

Comment 9

Last month
Pushed by shes050117@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/29bdbbe89264
Stop creating, writing, and reading the .metadata file in QuotaManager except the upgrades; r=janv

Comment 10

Last month
bugherder
Status: ASSIGNED → RESOLVED
Closed: Last month
Resolution: --- → FIXED
Target Milestone: --- → mozilla69

Is this something we need to fix in 68?

Flags: needinfo?(jvarga)
Reporter

Comment 12

12 days ago

Comment on attachment 9060698 [details]
Bug 1546752 - Stop creating, writing, and reading the .metadata file in QuotaManager except the upgrades;

Beta/Release Uplift Approval Request

  • User impact if declined: Creating the additional file can affect performance on really slow (old) machines.
  • Is this code covered by automated tests?: Yes
  • Has the fix been verified in Nightly?: Yes
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): Metadata files version 1 haven't been used for long time. We can safely stop creating them. The patch landed on Nightly 3 weeks ago. No regressions were reported.
  • String changes made/needed: None
Flags: needinfo?(jvarga)
Attachment #9060698 - Flags: approval-mozilla-beta?

Comment on attachment 9060698 [details]
Bug 1546752 - Stop creating, writing, and reading the .metadata file in QuotaManager except the upgrades;

remove some io in quotamanager, approved for 68.0b10

Attachment #9060698 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
You need to log in before you can comment on or make changes to this bug.