Last Comment Bug 676724 - (DMD) DMD: A Valgrind tool to find dark matter
(DMD)
: DMD: A Valgrind tool to find dark matter
Status: RESOLVED FIXED
:
Product: Core
Classification: Components
Component: General (show other bugs)
: Trunk
: All Linux
: -- normal with 8 votes (vote)
: ---
Assigned To: Nicholas Nethercote [:njn]
:
Mentors:
Depends on: 704400 NewDMD
Blocks: DarkMatter 680358 682219 692748 697332 126212 262234 636220 MemShrinkTools 677466 677653 678126 678376 678811 678977 682215 682216 686795 693016 696690 697041 697335 698326 699721 704723 707842 708762 722595 723799 726764 729008 729403
  Show dependency treegraph
 
Reported: 2011-08-04 16:40 PDT by Nicholas Nethercote [:njn]
Modified: 2012-11-19 16:51 PST (History)
26 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
DMD, v1 (51.55 KB, patch)
2011-08-14 21:58 PDT, Nicholas Nethercote [:njn]
no flags Details | Diff | Review
Firefox annotations, v1 (10.53 KB, patch)
2011-08-14 22:04 PDT, Nicholas Nethercote [:njn]
no flags Details | Diff | Review
DMD, v2 (57.43 KB, patch)
2011-08-19 03:08 PDT, Nicholas Nethercote [:njn]
no flags Details | Diff | Review
Firefox annotations, v2 (44.51 KB, patch)
2011-08-19 03:08 PDT, Nicholas Nethercote [:njn]
no flags Details | Diff | Review
DMD, v3 (61.12 KB, patch)
2011-10-24 17:40 PDT, Nicholas Nethercote [:njn]
no flags Details | Diff | Review
Firefox annotations, v3 (19.40 KB, patch)
2011-10-31 22:47 PDT, Nicholas Nethercote [:njn]
no flags Details | Diff | Review
DMD, v4 (62.10 KB, patch)
2011-11-03 22:58 PDT, Nicholas Nethercote [:njn]
no flags Details | Diff | Review
Firefox annotations, v4 (17.24 KB, patch)
2011-11-03 22:59 PDT, Nicholas Nethercote [:njn]
no flags Details | Diff | Review
DMD, v5 (63.89 KB, patch)
2011-12-08 21:13 PST, Nicholas Nethercote [:njn]
no flags Details | Diff | Review

Description Nicholas Nethercote [:njn] 2011-08-04 16:40:15 PDT
[The following text is from bug 563700 comment 23]

I was thinking some more about how to get *serious* about hunting down all the heap-unclassified bytes.

You'd add instrumentation code to maintain a data structure that records 1 bit of information for every malloc'd byte.  A 0 means "not reported", a 1 means "reported".  You'd also add instrumentation code to record, for every heap block, the stack trace at its allocation point.

Then, you'd modify every heap memory reporter so that when it is queried, it sets the "reported" bits of all the heap bytes it counts.

Once about:memory was loaded, you'd iterate over all the heap blocks.  Any heap block fully reported would be ignored.  Any heap block partially or not-at-all reported would be recorded.  You'd aggregate repeated stack traces for recorded heap blocks and present the stack traces in order so that the ones responsible for the most unreported bytes are shown first.

Also, while the memory reporters are doing their thing, you'd complain about any byte on the heap whose "reported" bit was set more than once -- such bytes are double-counted.  You could print out its stack trace and the name of the reporter that was second to count it.

Tracking one-bit-per-byte metadata and per-heap-block stack traces is something that Valgrind excels at, BTW.  A Valgrind tool that implemented this analysis would have to use its own heap allocator rather than jemalloc (that's just how Valgrind works), so it would have to be careful about the rounding up of request sizes, but it could definitely be made to work.  Memory reporters would use client requests to tell the tool which bytes have been accounted for.  A client request would also be used to tell the tool when all the memory reporters have been consulted.
Comment 1 Nicholas Nethercote [:njn] 2011-08-04 16:43:34 PDT
In addition to the Valgrind tool, each memory reporter will have to be modified to include annotations that tell the tool which heap bytes have been accounted for.  These annotations wouldn't be landed.
Comment 2 Justin Lebar (not reading bugmail) 2011-08-05 06:57:20 PDT
(In reply to Nicholas Nethercote [:njn] from comment #1)
> In addition to the Valgrind tool, each memory reporter will have to be
> modified to include annotations that tell the tool which heap bytes have
> been accounted for.  These annotations wouldn't be landed.

Why not?  If they're not landed, then:

 * I have to patch FF to run the tool, and
 * that patch probably will bitrot.

I presume this is something we're going to want to run every once in a while; it's not fix once and forget.
Comment 3 Nicholas Nethercote [:njn] 2011-08-05 14:44:00 PDT
Well, let's see how intrusive they are.
Comment 4 Nicholas Nethercote [:njn] 2011-08-14 21:58:42 PDT
Created attachment 553103 [details] [diff] [review]
DMD, v1

This is an in-progress version of DMD.  Features:

- Reports unreported heap blocks.

- Reports double-reported heap blocks.

- Reports how many bytes each annotated reporter reported.  This is useful for cross-checking with about:memory.

- Tracks both requested and slop bytes; emulates jemalloc's round-up behaviour.

- Has a couple of regression tests.

This patch applies to an SVN trunk version of Valgrind (I have r11976).  If you want to use it, you'll need to follow the usual Valgrind tricks for Firefox, see https://developer.mozilla.org/en/Debugging_Mozilla_with_Valgrind.  Trunk builds of Valgrind are easy, follow the instructions in Valgrind's top-level README file.

You'll need the --tool=exp-dmd flag, and you probably want to set --num-callers to something low like 6, otherwise lots of records that could be sensibly merged won't be.  (Even with 6 you'll still get some like that, but if you ask for fewer than 6 sometimes the stack traces will be too shallow to be useful.)
Comment 5 Nicholas Nethercote [:njn] 2011-08-14 22:04:50 PDT
Created attachment 553104 [details] [diff] [review]
Firefox annotations, v1

Some DMD annotations for Firefox.  Many more need to be done.
Comment 6 Nicholas Nethercote [:njn] 2011-08-19 03:08:16 PDT
Created attachment 554346 [details] [diff] [review]
DMD, v2
Comment 7 Nicholas Nethercote [:njn] 2011-08-19 03:08:55 PDT
Created attachment 554347 [details] [diff] [review]
Firefox annotations, v2
Comment 8 Justin Lebar (not reading bugmail) 2011-08-22 10:15:50 PDT
Did you mean to change this from [MemShrink:P1] to [MemShrink]?  If so, why?
Comment 9 Nicholas Nethercote [:njn] 2011-08-22 17:48:22 PDT
I did, I want to re-triage it.  The bug obviously shouldn't be closed, but I don't think it needs to be a P1 any more now that it's in a state where it's spitting out useful numbers.
Comment 10 Kyle Huey [:khuey] (khuey@mozilla.com) (Away until 6/13) 2011-08-23 04:31:03 PDT
(In reply to Nicholas Nethercote [:njn] from comment #9)
> I did, I want to re-triage it.  The bug obviously shouldn't be closed, but I
> don't think it needs to be a P1 any more now that it's in a state where it's
> spitting out useful numbers.

Might we call it fixed?
Comment 11 Nicholas Nethercote [:njn] 2011-08-23 17:15:47 PDT
(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #10)
> 
> Might we call it fixed?

I'll still be posting new versions of it, as well as new versions of the patch annotating Firefox.  Marking it closed when I'll still be working on it doesn't feel right.
Comment 12 Nicholas Nethercote [:njn] 2011-10-24 17:40:45 PDT
Created attachment 569249 [details] [diff] [review]
DMD, v3

This version also tracks allocations done with mmap, which led me to bug 696690.
Comment 13 Nicholas Nethercote [:njn] 2011-10-31 22:47:22 PDT
Created attachment 570932 [details] [diff] [review]
Firefox annotations, v3
Comment 14 Nicholas Nethercote [:njn] 2011-11-03 22:58:10 PDT
Created attachment 571888 [details] [diff] [review]
DMD, v4
Comment 15 Nicholas Nethercote [:njn] 2011-11-03 22:59:00 PDT
Created attachment 571889 [details] [diff] [review]
Firefox annotations, v4
Comment 16 Nicholas Nethercote [:njn] 2011-12-08 21:11:53 PST
Comment on attachment 571888 [details] [diff] [review]
DMD, v4

Bug 704400 folded DMD into the tree, so this patch is no longer needed.
Comment 17 Nicholas Nethercote [:njn] 2011-12-08 21:12:36 PST
Comment on attachment 571889 [details] [diff] [review]
Firefox annotations, v4

Ugh, wrong patch.
Comment 18 Nicholas Nethercote [:njn] 2011-12-08 21:13:10 PST
Created attachment 580310 [details] [diff] [review]
DMD, v5
Comment 19 Nicholas Nethercote [:njn] 2012-11-19 16:40:58 PST
DMDV (i.e. the Valgrind version of DMD) is now in the tree.  There's no need to keep this bug open.

Note You need to log in before you can comment on or make changes to this bug.