1059612 - Remove diff-breaking info from memory reports in a consistent fashion

Status: RESOLVED FIXED

(Toolkit :: about:memory, defect)

Description

[Nicholas Nethercote [inactive]]

There are a number of things that can appear in memory report process names and
paths that break diffs. These are in two categories.

* Non-deterministic IDs:
  + Addresses
  + PIDs
  + null principal UUIDs
  - Window IDs, e.g. |top("http://foo.com/", id=8)|

* Counts:
  - String copy counts, e.g. |string(length=131878, copies=1, "...")|  
  - Script source reference counts, e.g. |source(scripts=1501, <non-notable files>)|
  - (Potential) Object and shape instance counts e.g. |class(Array, nobjs=43)|

The ones marked with '+' are currently handled (i.e. stripped) in about:memory
via regexps.

It would be nice if we had a consistent way of handling these, instead of
adding them in an ad hoc fashion when we realize they are necessary (e.g. bug
1054318, bug 1005442, bug 1056954).

I don't have a particular idea in mind of how to do this, though.

Comment 1

[Nicholas Nethercote [inactive]]

Attachment: Remove diff-breaking info from memory reports in a consistent fashion

With this any number that needs to be sanitized in a diff can be surrounded by
some special chars, and about:memory will deal with it appropriately. E.g.
"^^[12345]^^" becomes "12345" normally, and "NNN" in a diff.

Pros:

- It works.

- It moves the "does this number need special treatment" logic from
  about:memory (or other memory report consumer) to the memory reporter.

Cons:

- The syntax is ugly -- I had to choose something that was very unlikely to
  match anything in a path. And since paths can contain arbitrary strings,
  which can contain URLs and JS/CSS/HTML code, I had to choose something weird.
  '^' is notable for being one of the few symbols that doesn't come up in
  practice in paths. (Backtick is another... maybe ``123`` would be better
  syntax.)

- AWSY will need updating.

- Similarly, if you're using an older Firefox, and you load memory reports
  generated by a newer Firefox, you'll get lots of batwings in the output.

I'm not totally convinced by this patch, but I don't see how else to do it.
Suggestions are welcome.

Comment 2

[Nicholas Nethercote [inactive]]

erahm and I discusses this on IRC, and concluded that:
- it's pretty ugly;
- more ad hoc filtering in about:memory is probably the least worst thing to do for now; and
- at some point there should be a big overhaul of memory reporting that can address this problem properly.

Comment 3

[Nicholas Nethercote [inactive]]

Attachment: Filter diff-breaking info from memory reports in a more rigorous fashion

Let's recycle this bug.

This patch does the following:

- Adds more comments about the stripping.

- Make the address stripping less promiscuous.

- Adds top window ID stripping.

I decided that counts shouldn't be stripped. Even if we hit a case where it's
relevant (i.e. the memory size is the same but the counts are different) that
seems worth distinguishing.

Comment 4

[Eric Rahm [:erahm]]

Comment on attachment 8491252 [details] [diff] [review]
Filter diff-breaking info from memory reports in a more rigorous fashion

Review of attachment 8491252 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm

Comment 5

[Nicholas Nethercote [inactive]]

Try looks good: https://tbpl.mozilla.org/?tree=Try&rev=5b2b08df68f0. The Mulet bustage appears to be bug 1069702, and not my fault.

Comment 6

[Nicholas Nethercote [inactive]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/b453cf797446

Comment 7

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/b453cf797446