Open Bug 986323 Opened 6 years ago Updated 9 months ago

Reduce start-up memory usage [meta]

Categories

(Firefox :: General, defect)

21 Branch
x86_64
All
defect
Not set

Tracking

()

ASSIGNED

People

(Reporter: njn, Assigned: njn)

References

(Depends on 4 open bugs)

Details

(Keywords: meta, Whiteboard: [tracking][MemShrink:P2])

Attachments

(3 files)

If you look at AWSY, you'd conclude that Firefox 13 was the best release in terms of memory consumption -- all the measurements have been ticking up gradually since then.

That would be a bogus conclusion -- we're much better in terms of avoiding really bad memory situations, which is when memory consumption tends to hurt us. E.g. FF13 pre-dates huge improvements like khuey's add-on leak fix, and tn's image decoding overhau, plus many smaller leak fixes.

Nonetheless, going from ~75 MiB RSS to ~135 MiB RSS at start-up isn't great. And the RSS measurements after opening all the tabs have gone from ~300 to ~415 MiB. It would be nice if we could claw some of that back.

This is a tracking bug.
I did a comparison between a Firefox from today, and one from about a year ago.
For both I created a new profile, then started up, waited 30 seconds, and then
triggered memory dumps (via a signal, not by loading about:memory).

Here's the high-level diff.

> 29,112,536 B (100.0%) -- explicit
> ├──10,997,520 B (37.78%) ++ js-non-window
> ├───8,867,976 B (30.46%) ++ heap-overhead
> ├───3,431,808 B (11.79%) ++ window-objects
> ├───2,739,912 B (09.41%) ++ xpconnect
> ├───2,468,992 B (08.48%) ── startup-cache/data
> ├─────535,552 B (01.84%) ++ workers/workers()
> ├─────425,088 B (01.46%) ── heap-unclassified
> ├────-323,704 B (-1.11%) ++ storage
>
> 31,170,560 B ── resident

So, about a 30 MiB increase.

Looking at the sub-trees, a lot of the reporter paths have changed quite a bit,
esp. the JS ones, which makes comparisons hard.  I'll point out notable things
I was able to determine.

----

The js-non-window stuff is hard to analyze closely because the reporter details
have changed a lot. But we have a big increase in the number of system
compartments, going from 145 to 234, which accounts for a big chunk of it.

Plus we have this:

> │  ├───1,508,928 B (05.18%) -- runtime
> │  │   ├──1,225,152 B (04.21%) ── script-data
> ...
> │  │   ├────216,128 B (00.74%) -- script-sources

Which is more evidence that we're running more JS code.

I will attach a diff of the compartment lists. There are lots of differences,
but some areas where multiple new compartments are present: commonjs/sdk/
(whatever that is), devtools, FxAccounts, sessionstore, services-common,
pdf.js, shumway.

Surely a lot of these don't need to be loaded at startup?

----

> ├───8,867,976 B (30.46%) ++ heap-overhead

Ignore this one; it's something that wasn't measured in the older profile. With
that in mind, the difference in 'explicit' drops to ~22 MiB, less than the 30
MiB 'resident' increase. Part of the gap is probably due to libxul being
bigger, I suspect.

----

> ├───3,431,808 B (11.79%) ++ window-objects
> ...
> │   │  │  ├──1,080,040 B (03.71%) --
> window(chrome://browser/content/browser.xul)
> │   │  │  │  ├────463,216 B (01.59%) ── style-sheets
> │   │  │  │  ├────312,848 B (01.07%) ++ js-compartment([System Principal],
> about:blank)
> │   │  │  │  ├────242,544 B (00.83%) ++ layout
> │   │  │  │  ├─────60,952 B (00.21%) ++ dom

There's now lots more style stuff in browser.xul!

----

> │   ├────924,816 B (01.14%) ++ top(about:newtab, id=17)
> ...
> │  │  │  ├─────113,120 B (00.14%) -- compartment([System Principal], about:newtab)

about:newtab wasn't present in the old profile.

----

> │   │  │  ├────626,952 B (00.77%) -- window(data:application/vnd.mozilla.xul+xml;charset=utf-8,<window%20id='win'/>)
> ...
> │  │  │  ├─────181,200 B (00.22%) -- compartment([System Principal], inProcessTabChildGlobal?ownedBy=data:application/vnd.mozilla.xul+xml;charset=utf-8,<window%20id='win'/>)

This vnd.mozilla.xul thing wasn't present in the old profile.

----

> ├───2,739,912 B (09.41%) -- xpconnect
> │   ├──2,531,328 B (08.69%) ── proto-iface-cache [+]

The proto-iface-cache is new. Bug 948445 is about shrinking it.

----

> ├───2,468,992 B (08.48%) ── startup-cache/data

This went from 4.85 to 7.20 MiB.

----

> ├─────535,552 B (01.84%) -- workers/workers()
> │     ├──2,672,264 B (09.18%) ++ worker(resource:///modules/sessionstore/SessionWorker.js, 0xNNN)
> │     └──-2,136,712 B (-7.34%) ++ worker(resource://gre/modules/osfile/osfile_async_worker.js, 0xNNN)

osfile_async_worker.js shrank quite a bit, which is good. SessionWorker.js is
new. (PageThumbsWorker.js didn't show up in this profile, but it's also new.)
Depends on: 948445
BTW, if you want to compare the two sets of memory reports yourself, you'll need to apply the patch from bug 986300. Without it, about:memory will assert and show a mostly blank page after doing "Load and diff...".

Also (inside baseball alert) I hand-modified some of the entries in the old Firefox's reports to anonymise some addresses that weren't prefixed with '0x' and so about:memory's anonymisation didn't work.
Attached file diff-compartments.txt
Here's the change in system compartments.

(If you're curious: I generated this by using about:compartments in the old Firefox, the 'js-main-runtime-compartments' sub-tree in the new Firefox, and then doing some munging by hand before and after running |diff|.)
Depends on: 986300
Whiteboard: p=0
https://github.com/mozilla/pdf.js/pull/4501 is a PR for pdf.js to lazily load pdf.js/network.js.
Depends on: 986683
Depends on: 987146
> > ├───2,468,992 B (08.48%) ── startup-cache/data
> 
> This went from 4.85 to 7.20 MiB.

The startup cache holds the xulcache, xblcache, and jsloader cache. These all hold code, and so as we get more code at startup, it's natural that it gets bigger. But wow, 7.2 MiB is a lot.
Summary: Investigate slow increase in AWSY numbers → Investigate slow increase in AWSY numbers [meta]
Depends on: 988096
Depends on: 988624
Whiteboard: p=0 → [tracking]
Whiteboard: [tracking] → [tracking][MemShrink:P2]
No longer blocks: fxdesktopbacklog
Depends on: 996438
Depends on: 996430
Depends on: 996448
Depends on: 996989
Depends on: 997590
Since most of the discussion and blocking bugs are about start-up memory usage, I'm going to re-purpose this bug to be about that.
Depends on: 997906, 1001094
Summary: Investigate slow increase in AWSY numbers [meta] → Reduce start-up memory usage [meta]
Depends on: 975639
OS: Linux → All
You need to log in before you can comment on or make changes to this bug.