If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

instrument boehm GC to dump "in use" information

RESOLVED INCOMPLETE

Status

()

Core
XPCOM
P3
normal
RESOLVED INCOMPLETE
17 years ago
8 years ago

People

(Reporter: Chris Waterson, Unassigned)

Tracking

({footprint})

Trunk
x86
Linux
footprint
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(9 attachments)

(Reporter)

Description

17 years ago
We may be able to get more traction on the bloat problem if we had a tool that
could "snapshot" the current memory state, and identify all the objects that
were currently in use. (Ideally, with information about who allocated each
object.) It was suggested that the Boehm collector could probably do this.
(Reporter)

Updated

17 years ago
Keywords: footprint
Created attachment 13522 [details] [diff] [review]
trivial extension to nsTraceMalloc
This patch provides a function, NS_TraceMallocDumpAllocation(const char
*pathname), that can be called from anywhere in the code.  I'll add a patch to
provide a JS native function TraceMallocDumpAllocations in a sec.  You have to
--enable-trace-malloc in your Linux build, as usual, and you should run mozilla
with --trace-malloc /dev/null or a real file, to capture allocations and their
compressed stack backtraces.

The output file is likely to be very large; beware.  Obviously, we could do much
better by minimizing callsite tree path redundancy, as the trace-malloc logfile
format strives to do in a binary fashion.

/be
Created attachment 13524 [details] [diff] [review]
Add TraceMallocDumpAllocations JS function
Created attachment 13527 [details] [diff] [review]
consolidated patch with a few cleanups
Created attachment 13538 [details] [diff] [review]
Clean up header line spacing to match columns; add semicolon separator between callsites for better perl/sed/awk munging
Created attachment 13539 [details] [diff] [review]
Check NS_TraceMallocDumpAllocations' return value and report JS error if necessary
Starting ./mozilla --trace-malloc /dev/null to first window loading mozilla.org,
then loading file:/tmp/dump.html (see next attachment) results in a 250MB file
with > 126,000 lines, one for each live allocation when I clicked on the Dump
button in the form after entering /tmp/big.dump in the text input.

Still, it's a crude attempt at what this bug wants, and it can be post-processed
by good ol' Unix utilities.  Go nuts.

/be
(Reporter)

Comment 8

17 years ago
Created attachment 13668 [details]
"web page" that will dump live allocations to /tmp/dump.log
(Reporter)

Comment 9

17 years ago
Created attachment 13669 [details]
blame.pl, which will process dump.log and attribute allocations by call stack
(Reporter)

Comment 10

17 years ago
Created attachment 13670 [details]
a style sheet to make it pretty
(Reporter)

Comment 11

17 years ago
Put a log file of "opening a new window" at

  http://ftp.mozilla.org/bloat-reports/2000-08-28-live-objects.html

Take a look (may take a while for the FTP server to sync up). It's pretty cool.
Some first order comments...

  - About 9Mb of live allocations in the heap
  - NS_InitXPCOM allocates 300Kb.
  - ~170KB is from PR_NewLock().
  - ~500KB comes from 300 XUL templates. How on earth did we get
    that many XUL templates?
  - 1.2Mb is from JS_Malloc().
  - At least 90Kb of parser junk lying around. I thought parsers
    got killed as soon as they were done being used?
  - 1.2Mb of string data lying around
  - My global history is taking up about 80Kb. AFAICT, I only have
    a dozen or so sites. We should see how this scales...
  - XBL accounts for ~400Kb.

N.B. that I've done nothing to deal with recursive routines, so I think that
this'll be double-counting a routine that's entered >1 time on a call stack.

Brendan: how soon until you'd feel comfortable checking these patches in? When
you do so, I'll check my scripts and stuff into mozilla/tools somewhere.
(Reporter)

Comment 12

17 years ago
Oops, make that

  http://ftp.mozilla.org/pub/bloat-reports/2000-08-28-live-objects.html

Still not there. Should be there in an hour or so.
Created attachment 13674 [details] [diff] [review]
sorry, forgot to attach my (html3.2-ish) dump.html form page
(Reporter)

Comment 14

17 years ago
Checked in blame.pl and blame.css into mozilla/tools/trace-malloc, a new
directory I've created to house brendan's family of trace-malloc based tools.
Brendan, check in your TraceMallocDumpAllocations() patch. I crave it.
I checked in yesterday afternoon.

/be
http://ftp.mozilla.org/pub/bloat-reports/2000-08-28-live-object.html is the 
right URL, it seems.  Good stuff, mysteries of the deep.

/be

Updated

17 years ago
Status: NEW → ASSIGNED

Updated

17 years ago
QA Contact: leger → kandrot

Comment 17

17 years ago
over to thesteve
Assignee: wade → thesteve
Status: ASSIGNED → NEW

Updated

12 years ago
QA Contact: kandrot → nobody
Assignee: thesteve → nobody
QA Contact: nobody → xpcom

Comment 18

8 years ago
boehmgc is no longer in our tree
Status: NEW → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.