410678 - Linux Footprint

Status: RESOLVED WONTFIX

(Core Graveyard :: GFX, defect)

Description

[Robert Sayre]

Attachment: massif data

Attached is a massif data and a graph of one run throught the talos pageset.

The primary memory hogs seem to be Linux font code, and some of our hashtables. I'll need to rerun this with some more functions listed as allocators, but I suspect we'll find it's the cycle collector creating these hashtables.

The good news is that the Frame/Node arenas and the gfx image cache seem to shink, and the cycle collector does kick in and collapse most of the "other" category.

Comment 1

[Robert Sayre]

Attachment: massif graph for one talos run

Comment 2

[Stuart Parmenter]

We know we can reduce some of the cairo allocations (at least the non-Xlib-surface ones which we haven't looked at (we haven't looked at the cairo font code either))

The cycle collector does create very large hashtables.  There is a bug on file to try to avoid realloc churn on them but we discussed their size and couldn't come up with a significantly smaller solution.

Comment 3

[Robert Sayre]

Talking with stuart, next step is to run this over multiple runs, and see if the font stuff remains unbounded.

As for the cycle collector, we can experiment with alternative hashtable implementations to see if we can shrink it.

Comment 4

[Brendan Eich [:brendan]]

(In reply to comment #3)
> As for the cycle collector, we can experiment with alternative hashtable
> implementations to see if we can shrink it.

google::dense_hash_foo could cut out the stored keyHash.

Radix tree might be even better (faster too). Depends on the distribution of addresses used as keys. That could be studied without actually implementing a radix tree, just by counting "pages".

/be

Comment 5

[Robert Sayre]

Attachment: directions + hack to run Tp under valgrind / massif

Attaching patch to produce this data in case other people want to try.

Comment 6

[Brendan Eich [:brendan]]

(In reply to comment #4)
> (In reply to comment #3)
> > As for the cycle collector, we can experiment with alternative hashtable
> > implementations to see if we can shrink it.
> 
> google::dense_hash_foo could cut out the stored keyHash.

Sayrer pointed out that I meant sparse, not dense, of course.

/be

Comment 7

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

On my machine, last I checked (a few months ago), pango loads about half a meg of stuff into RAM per font loaded.  (Bold and Italic variants count as separate fonts.)  A substantial part of that is the gpos and gsub tables loaded into an in-memory representation via harfbuzz.  (One would think this could be done by just mmap-ing the font file.)

(In reply to comment #2)
> The cycle collector does create very large hashtables.  There is a bug on file
> to try to avoid realloc churn on them

bug 407454

Comment 8

[Robert Sayre]

Attachment: 3 runs of the pageset

looks like massif got a little confused on this one, but still useful

Comment 9

[Robert Sayre]

Attachment: massif data from 3 consecutive talos runs

Comment 10

[Robert Sayre]

We're definitely leaking moz_drawing_surfaces (and therefore GTK stuff). This can be shown in any leak tool, and grows with each page visited.

Comment 11

[Reed Loden [:reed]]

(In reply to comment #10)
> We're definitely leaking moz_drawing_surfaces (and therefore GTK stuff). This
> can be shown in any leak tool, and grows with each page visited.

Mind filing a separate bug on that and requesting blocking, please?

Comment 12

[Robert Sayre]

Filed bug 411030.

Comment 13

[Gordon P. Hemsley [:GPHemsley]]

This bug has been buried in the graveyard and has not been updated in over 5 years. It is probably safe to assume that it will never be fixed, so resolving as WONTFIX.

[Mass-change filter: graveyard-wontfix-2014-09-24]