Cache more things from omnijars with the StartupCache
Categories
(Core :: XPCOM, enhancement)
Tracking
()
| Tracking | Status | |
|---|---|---|
| firefox80 | --- | fixed |
People
(Reporter: dthayer, Assigned: dthayer)
References
(Blocks 2 open bugs)
Details
(Whiteboard: [fxperf:p2])
Attachments
(6 files)
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review |
During a startup where the StartupCache has been populated, we still read about 2MB (Edit: 4MB) jumping across the entire space of our two omnijars. If we cached more in the StartupCache by allowing caching of things loaded in background threads and child processes, we could reduce this to nearly 0.
|
||
Comment 1•1 year ago
|
||
Please note that we reorder omnijars during the build and preload parts of them on startup, but we only do so on PGO builds, so you won't see it on local builds.
|
Assignee | |
Comment 2•1 year ago
|
||
Yes, I've been doing all my testing on PGO builds for that reason. (Though we generally read far more during startup than is actually included in that section.)
|
|
Assignee | |
Comment 3•1 year ago
|
||
Also until bug 1627040 lands, we actually read the entire combined 75MB of both omnijars during startup.
|
|
||
Comment 4•1 year ago
|
||
(In reply to Doug Thayer [:dthayer] from comment #2)
(Though we generally read far more during startup than is actually included in that section.)
If that's the case, we should fix that.
|
|
Assignee | |
Comment 6•1 year ago
|
||
|
|
Assignee | |
Comment 7•1 year ago
|
||
Depends on D77632
|
|
Assignee | |
Comment 8•1 year ago
|
||
Depends on D77633
|
|
Assignee | |
Comment 9•1 year ago
|
||
Depends on D77634
|
||
Updated•1 year ago
|
|
|
||
Updated•1 year ago
|
|
|
||
Updated•1 year ago
|
|
|
Assignee | |
Comment 10•1 year ago
|
||
Opening our Omnijars can be expensive, and it should be deferrable until after
startup is completed, provided we have a startup cache. In a previous patch in this
stack, we implemented caching of the zip central directory for omnijars, but we
still have to open the file in order to hand the object off to various omnijar
consumers. In a later patch, we will wrap nsZipArchive access in a class which
will allow us to transparently cache nsZipArchive results. These two get us
most of the way to not needing to read from the underlying omnijar files during
startup, but there are still nontrivial pieces, like nsZipFind for instance,
which we don't want to just duplicate inside of a wrapper class, so we would
like to sort out a way in which we can use an nsZipArchive class, but not
actually back it up with the real underlying file until we really need data
from it which we can't find in a cache.
Depends on D77633
|
|
||
Updated•1 year ago
|
|
|
||
Updated•1 year ago
|
|
|
Assignee | |
Comment 11•1 year ago
|
||
Depends on D77635
|
||
Comment 12•11 months ago
|
||
Pushed by dothayer@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/763a5a7b43a0 Build Omnijar file list from startup cache r=froydnj https://hg.mozilla.org/integration/autoland/rev/0f0d1bd2a8ac Init StartupCache before Omnijar r=froydnj https://hg.mozilla.org/integration/autoland/rev/6cd31f17a931 Allow lazily initializing nsZipArchives r=froydnj https://hg.mozilla.org/integration/autoland/rev/65c0e6790a33 OMT and OMP StartupCache access r=froydnj https://hg.mozilla.org/integration/autoland/rev/b07887474f51 Route Omnijar requests through StartupCache r=froydnj https://hg.mozilla.org/integration/autoland/rev/cf23b456ba12 Include WebGLParent.h in WebGLChild.h r=handyman,jgilbert
|
||
Comment 13•11 months ago
|
||
Backed out 6 changesets for causing failures regarding startupcache.
Backout link: https://hg.mozilla.org/integration/autoland/rev/e1c28fef32cdd9b6717e4faa186139d558adeb95
Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=308229242&repo=autoland&lineNumber=47252
|
|
||
Comment 14•11 months ago
|
||
Pushed by dothayer@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/b6029c7c0016 Build Omnijar file list from startup cache r=froydnj https://hg.mozilla.org/integration/autoland/rev/f5106898239f Init StartupCache before Omnijar r=froydnj https://hg.mozilla.org/integration/autoland/rev/6c4eedaa0d04 Allow lazily initializing nsZipArchives r=froydnj https://hg.mozilla.org/integration/autoland/rev/ece9a4223349 OMT and OMP StartupCache access r=froydnj https://hg.mozilla.org/integration/autoland/rev/7ae4df10749c Route Omnijar requests through StartupCache r=froydnj https://hg.mozilla.org/integration/autoland/rev/699212a443c3 Include WebGLParent.h in WebGLChild.h r=handyman,jgilbert
|
|
||
Comment 15•11 months ago
|
||
Pushed by archaeopteryx@coole-files.de: https://hg.mozilla.org/integration/autoland/rev/9705b2759d45 Remove unused 'start' variable from startupcache/StartupCache.cpp. a=Bb-fix CLOSED TREE
|
||
Comment 16•11 months ago
•
|
||
Backed out 7 changesets (Bug 1627075) for causing @nsZipArchive crashes CLOSED TREE
Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&resultStatus=testfailed%2Cbusted%2Cexception&revision=699212a443c35f37bc998f21cad29cb9fc2e8c95
Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=308295093&repo=autoland&lineNumber=666
Backout: https://hg.mozilla.org/integration/autoland/rev/4c73ffd9cf3141bb98f321b059465e32a9d52834
|
|
||
Comment 17•11 months ago
|
||
Pushed by dothayer@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/ebdcd96a9d20 Build Omnijar file list from startup cache r=froydnj https://hg.mozilla.org/integration/autoland/rev/920398d1c3d3 Init StartupCache before Omnijar r=froydnj https://hg.mozilla.org/integration/autoland/rev/2cb021a493d8 Allow lazily initializing nsZipArchives r=froydnj https://hg.mozilla.org/integration/autoland/rev/288a67aed661 OMT and OMP StartupCache access r=froydnj https://hg.mozilla.org/integration/autoland/rev/a345e05df151 Route Omnijar requests through StartupCache r=froydnj https://hg.mozilla.org/integration/autoland/rev/fc144caf5d06 Include WebGLParent.h in WebGLChild.h r=handyman,jgilbert
|
|
||
Comment 18•11 months ago
|
||
Backed out 6 changesets for causing bustages in StartupCache.cpp
Backout link: https://hg.mozilla.org/integration/autoland/rev/232921af2e09cccce37ca4378c9d18068551bb5c
Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=308793702&repo=autoland&lineNumber=29203
|
|
||
Comment 19•11 months ago
|
||
Pushed by dothayer@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/333d035afe92 Build Omnijar file list from startup cache r=froydnj https://hg.mozilla.org/integration/autoland/rev/13ecd68b3c0d Init StartupCache before Omnijar r=froydnj https://hg.mozilla.org/integration/autoland/rev/dfd71f4ecb81 Allow lazily initializing nsZipArchives r=froydnj https://hg.mozilla.org/integration/autoland/rev/eb5265addd5e OMT and OMP StartupCache access r=froydnj https://hg.mozilla.org/integration/autoland/rev/e29b15980da2 Route Omnijar requests through StartupCache r=froydnj https://hg.mozilla.org/integration/autoland/rev/21605186687e Include WebGLParent.h in WebGLChild.h r=handyman,jgilbert
|
||
Comment 20•11 months ago
|
||
Backed out for bustages on startupcache/StartupCache.cpp
Backout link: https://hg.mozilla.org/integration/autoland/rev/560ca8fba0a1591b00730db8e6252379bf9ab38d
Log link: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=308871117&repo=autoland&lineNumber=23150
|
|
Assignee | |
Comment 21•11 months ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=47e403d91b24d252906ba3d0a8dab9a78462f0e4
Nth time's the charm...
|
|
||
Comment 22•11 months ago
|
||
Pushed by dothayer@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/4208ed5fc3b0 Build Omnijar file list from startup cache r=froydnj https://hg.mozilla.org/integration/autoland/rev/1d838c1c4669 Init StartupCache before Omnijar r=froydnj https://hg.mozilla.org/integration/autoland/rev/d431b24bebf6 Allow lazily initializing nsZipArchives r=froydnj https://hg.mozilla.org/integration/autoland/rev/d2e40366118c OMT and OMP StartupCache access r=froydnj https://hg.mozilla.org/integration/autoland/rev/aca7cd850cc9 Route Omnijar requests through StartupCache r=froydnj https://hg.mozilla.org/integration/autoland/rev/4d3569a5cd9f Include WebGLParent.h in WebGLChild.h r=handyman,jgilbert
|
||
Comment 23•11 months ago
|
||
| bugherder | ||
https://hg.mozilla.org/mozilla-central/rev/4208ed5fc3b0
https://hg.mozilla.org/mozilla-central/rev/1d838c1c4669
https://hg.mozilla.org/mozilla-central/rev/d431b24bebf6
https://hg.mozilla.org/mozilla-central/rev/d2e40366118c
https://hg.mozilla.org/mozilla-central/rev/aca7cd850cc9
https://hg.mozilla.org/mozilla-central/rev/4d3569a5cd9f
Description
•