915749 - Cookie clicker game causes high heap-unclassified; on Linux it's due to libmurrine

Status: RESOLVED INCOMPLETE

(Core :: General, defect)

Description

[David Zbarsky (:dzbarsky)]

My guess is that it's http://orteil.dashnet.org/cookieclicker/

Comment 1

[David Zbarsky (:dzbarsky)]

Attachment: memory-report.json.gz

Comment 2

[Andrew McCreight [:mccr8]]

In IRC, dzbarsky said he was getting this:
   1,839.64 MB (67.23%) ── heap-unclassified
He's going to make a local DMD build and see what it turns up.

Comment 3

[Nicholas Nethercote [inactive]]

> My guess is that it's http://orteil.dashnet.org/cookieclicker/

I think I can reproduce, but the game has to run for a long time, because heap-unclassified increases slowly.  Fortunately the game basically runs itself.

Comment 4

[Nicholas Nethercote [inactive]]

dzbarsky, have you had a chance to do a DMD run?

Comment 5

[David Zbarsky (:dzbarsky)]

I tried running it, but firefox crashed when I ran DMDReportAndDump.  This was on OSX 10.8, maybe it works better on linux?

Comment 6

[Nicholas Nethercote [inactive]]

Attachment: DMD output from Linux

I ran the game for about 20 hours.  heap-unclassified crept up from 16MB/~16% to 70MB/~35% in that time.  Attached are the top 10 reports from the DMD output.

Unfortunately, the main source of heap-unclassified is libmurrine, which appears to be part of GTK.  (http://rpmfind.net/linux/rpm2html/search.php?query=libmurrine say "The Murrine engine is a cairo-based GTK2 theming tool.")  It's going to be hard to measure that memory usage.  Also, there's a good chance that this library isn't used on OS X and so the heap-unclassified that dzbarsky saw is from a different source :(

There is also some unclassified memory being allocated under CreateHTMLElement().  We should be able to report that pretty easily.  bz, do you know where that ends up being stored?

Comment 7

[Boris Zbarsky [:bzbarsky]]

The GTK2 theming stuff is definitely not used on OSX.

The CreateHTMLElement bits in that output are just showing script doing document.createElement().  After that it's just a node... the script can store it a JS variable, put it in the DOM, etc.  But I thought we reported all nodes now?

Comment 8

[Andrew McCreight [:mccr8]]

We track nodes that are in actively displayed documents, and those in documents that have nodes that are reachable from JS.  I'm not sure how else a page could keep nodes alive, but that's a question a CC log should be able to answer.

Comment 9

[Nicholas Nethercote [inactive]]

> that's a question a CC log should be able to answer.

Here's a question I should know the answer to:  what's the best way to get a CC log?  A google search suggested either (a) a magic javascript: invocation, or (b) about:cc, or (c) about:ccdump.

Comment 10

[Andrew McCreight [:mccr8]]

Those all work. ;)  (though the last two require addons...) There's also environment variables.  But for this case you just want to do something like this: https://wiki.mozilla.org/Performance:Leak_Tools#Cycle_collector_heap_dump

Comment 11

[Nicholas Nethercote [inactive]]

Attachment: GC log

Comment 12

[Nicholas Nethercote [inactive]]

Attachment: CC log

Comment 13

[Nicholas Nethercote [inactive]]

mccr8, can you do your CC log magic?  Thanks.

Comment 14

[Andrew McCreight [:mccr8]]

Oops, I should have suggested that you get an AllTraces log, which is the third one in there. If you could get me one of those, I can look at it again. The CC log you attached isn't very interesting, so presumably whatever is causing the heap-unclassified is being optimized out of the CC graph.

From the GC log, these are the most common things:
  159008 shape
   51655 base_shape
   21522 HTMLSpanElement <no private>
   18980 Array <no private>
   16564 type_object
   15893 Object <no private>
   11727 Function

Presumably it is the span elements that are causing the heap-unclassified?  Pretty much all of the span elements are colored white, which is weird, because that means they've been allocated since the last GC?

Comment 15

[Nicholas Nethercote [inactive]]

Attachment: allTraces CC log

> Oops, I should have suggested that you get an AllTraces log, which is the
> third one in there. If you could get me one of those, I can look at it
> again.

Here you go.  heap-unclassified was 120 MB (57%) after running for close to 24 hours when I took this.  Let me know if you need the GC log from that invocation as well.

Comment 16

[Andrew McCreight [:mccr8]]

I started looking at the log.  There's a ton of about:memory stuff.  Is that expected, Nick?

Other than that, I haven't come up with anything particularly useful.  I'll try digging around some more.

Comment 17

[Nicholas Nethercote [inactive]]

> I started looking at the log.  There's a ton of about:memory stuff.  Is that
> expected, Nick?

Somewhat.  I hit "measure" in about:memory a few times and the strings can build up between GCs when that happens.  (This is why I want to improve the string handling.)

Comment 18

[Andrew McCreight [:mccr8]]

Well, what I'm seeing isn't strings per se, but tens of thousands of DOM nodes:
   20101 nsGenericDOMDataNode
    7525 JS Object
    3872 FragmentOrElement (xhtml) span class='treeline' about:memory
    3872 FragmentOrElement (xhtml) span class='mrValue' about:memory
    3872 FragmentOrElement (xhtml) span class='mrSep' about:memory
    3872 FragmentOrElement (xhtml) span class='mrName' about:memory
    3859 FragmentOrElement (xhtml) span class='mrPerc' about:memory
    2556 XPCWrappedNative

Comment 19

[Nicholas Nethercote [inactive]]

> Well, what I'm seeing isn't strings per se, but tens of thousands of DOM
> nodes:

That's believable too.  Every time you re-measure, all the existing DOM nodes (of which there are many) are discarded.

Comment 20

[Andrew McCreight [:mccr8]]

Maybe I'll eventually be able to figure something out here...

Comment 21

[Andrew McCreight [:mccr8]]

(In reply to Andrew McCreight [:mccr8] from comment #20)
> Maybe I'll eventually be able to figure something out here...

I didn't.