unclassified heap block (DMD) results from a MemBench run

NEW
Unassigned

Status

()

Core
General
5 years ago
5 years ago

People

(Reporter: njn, Unassigned)

Tracking

(Depends on: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [MemShrink:P2])

Attachments

(5 attachments)

(Reporter)

Description

5 years ago
I ran MemBench (http://gregor-wagner.com/tmp/mem) under DMD on Linux64.  It opens and closes 150 tabs.  I triggered DMD after they all closed.  The main sources of unreported memory:

- GLContext stuff.

- Font stuff.

- Cache and other stuff.

I'll attach files with details shortly.
(Reporter)

Comment 1

5 years ago
Created attachment 719802 [details]
GLContext stuff

All up there was ~14 MiB of GLContext stuff;  this file shows most of it.
(Reporter)

Comment 2

5 years ago
Created attachment 719804 [details]
Font/text stuff

There were some Chinese(?) pages in the 150 tabs, which I think explains the hyphenation stuff.  I don't think there were any Thai pages (is that what libthai.so relates to?) though I'm not certain.
(Reporter)

Comment 3

5 years ago
Created attachment 719805 [details]
Cache and nsFeedSniffer stuff
(Reporter)

Comment 4

5 years ago
Created attachment 719809 [details]
memory report dump (JSON)

Just for completeness, here's the memory report JSON dump for a similar run.  View it in about:memory via the buttons at the bottom.
Created attachment 719811 [details] [diff] [review]
use no global context on glx

For the GL part, try this please on Linux.

Not landable as-is as that would break the (disabled by default) GLX layers.

If that fixes it that would mean that this leak was induced by our usage of share groups, which might prevent the GL from releasing GL resources. Not clear to me then whether that would be our fault or the GL's but that's a simple fix.

If that does not fix it that almost certainly a plain leak in the GL.

jgilbert: the WebGL context already deletes all the GL objects associated with it on destruction; does the GL context likewise delete remaining objects (e.g. the framebuffer 0)?
Yes, we delete all the GLContext accessories. *However*, we deal somewhat strangely with objects held by OGL Layers on platforms with shared GL contexts. That's my guess.

Comment 7

5 years ago
(In reply to Nicholas Nethercote [:njn] from comment #2)
> Created attachment 719804 [details]
> Font/text stuff
> 
> There were some Chinese(?) pages in the 150 tabs, which I think explains the
> hyphenation stuff.  I don't think there were any Thai pages (is that what
> libthai.so relates to?) though I'm not certain.

Er, not sure what DMD is but stacks 1,2,5,6 are in OTS, the sanitizer
for downloadable fonts, stacks 4,7 is in Japanese line breaking code
(nsJISx4051LineBreaker) calling through to Pango, and stack 3 is the
hyphenation code. God knows why Pango is calling libthai in this instance.  I doubt Chinese pages have much to do with hyphenation.
(Reporter)

Comment 8

5 years ago
> Er, not sure what DMD is

It's a tool for identifying memory allocations that aren't reported and so fall into the "heap-unclassified" bucket in about:memory.  See https://wiki.mozilla.org/Performance/MemShrink/DMD for more details.

Updated

5 years ago
Summary: DMD results from a MemBench run → unclassified heap block (DMD) results from a MemBench run
(In reply to John Daggett (:jtd) from comment #7)
> (In reply to Nicholas Nethercote [:njn] from comment #2)
> > Created attachment 719804 [details]
> > Font/text stuff
> > 
> > There were some Chinese(?) pages in the 150 tabs, which I think explains the
> > hyphenation stuff.  I don't think there were any Thai pages (is that what
> > libthai.so relates to?) though I'm not certain.
> 
> Er, not sure what DMD is but stacks 1,2,5,6 are in OTS, the sanitizer
> for downloadable fonts, 

Which implies that the tabs involved downloadable fonts, and those font entries are still present. It's possible that if you wait for a couple of minutes, these would expire from cache (they're managed by an expiration timer after the last reference to them is released).

Maybe we could add a memory reporter that would cover these.

> stacks 4,7 is in Japanese line breaking code
> (nsJISx4051LineBreaker) calling through to Pango, and stack 3 is the
> hyphenation code. God knows why Pango is calling libthai in this instance. 
> I doubt Chinese pages have much to do with hyphenation.

The hyphenation won't be for Chinese (or similar), it'll be a page that has hyphens:auto and is tagged with a language, very likely en-US, for which we have a hyphenation dictionary. We load the hyphenation resources on first use, but then we keep them in memory so that we won't kill performance by having to do the (expensive) dictionary-loading operation for every line-break.

I'm not sure if there'd be a good way to report this memory without hacking at libhyphen internals, which is not something I'd be keen to do.

(The hyphenation manager should probably listen for the memory-pressure notification and unload dictionaries when memory is tight; I don't think we have code in place to do that at present. But we definitely want to keep them loaded in the normal case.)
FTR, I filed bug 846690 to discard hyphenation resources on memory-pressure. This won't remove them from DMD in the normal case, but it should mean that if you fire a memory-pressure notification, the memory will be freed (unless the hyphenation code actually leaks).
(Reporter)

Comment 11

5 years ago
> Which implies that the tabs involved downloadable fonts, and those font
> entries are still present.
> 
> Maybe we could add a memory reporter that would cover these.

Can you file a bug?


> We load the hyphenation resources on first
> use, but then we keep them in memory so that we won't kill performance by
> having to do the (expensive) dictionary-loading operation for every
> line-break.
> 
> I'm not sure if there'd be a good way to report this memory without hacking
> at libhyphen internals, which is not something I'd be keen to do.

Fair enough.  Tracking memory in libraries is a pain, and the biggest remaining shortcoming in about:memory's coverage (Cairo is another example).


> FTR, I filed bug 846690 to discard hyphenation resources on memory-pressure.

Thanks!

Updated

5 years ago
Depends on: 847787

Comment 12

5 years ago
(In reply to Nicholas Nethercote [:njn] from comment #11)
> > Which implies that the tabs involved downloadable fonts, and those font
> > entries are still present.
> > 
> > Maybe we could add a memory reporter that would cover these.
> 
> Can you file a bug?

Done.  Looks like gfxDownloadedFcFontEntry needs to add memory reporter methods.
(Reporter)

Comment 13

5 years ago
> For the GL part, try this please on Linux.
> 
> Not landable as-is as that would break the (disabled by default) GLX layers.

I tried it and the browser crashed part-way through the MemBench run, presumably on a page that used WebGL.  Any other things I can try?
If my patch made it crash where it didn't crash before, then that's very surprising and I'd need to debug this myself to understand what's going on. Sorry, I'm out of ideas at the moment. A tip if you want to get this prioritized by the gfx team: try to see if by chance this reproduces also on Mac. If it does, that would up a lot the priority. We're swamped at the moment.
(Reporter)

Comment 15

5 years ago
> try to see if by chance this reproduces also on Mac.

I tried that and DMD crashed while getting a stack trace, in a way I haven't seen before :(
(Reporter)

Updated

5 years ago
Whiteboard: [MemShrink] → [MemShrink:P2]
You need to log in before you can comment on or make changes to this bug.