693898 - High heap-unclassified on Xenu's broken link report (large HTML page with many links)

Status: RESOLVED DUPLICATE of bug 682431

(Core :: General, defect)

Description

[Justin Lebar (not reading bugmail)]

Attachment: xzip'ed testcase

I get ~50% heap-unclassified with the attached page.  The page has many links, so this may be a duplicate of bug 682437.

Comment 1

[Boris Zbarsky [:bzbarsky]]

Or the memory reporter for link URIs, yeah.

Comment 2

[Justin Lebar (not reading bugmail)]

Textruns could account for some of this heap-unclassified, but the file is 33mb uncompressed, so the textruns shouldn't be much more than that, right?  The heap-unclassified is 240mb.

(On a completely unrelated note, I am really impressed that xz compressed this file to 13kb.  In comparison, bz2 compresses to 400kb.)

Comment 3

[Boris Zbarsky [:bzbarsky]]

I don't have a good feel for how many bytes per char textruns are.

But all the URIs in that file are absolute as well, and an nsStandardURL is just not _that_ much overhead over its string....

We need some DMD love here.

Comment 4

[Jonathan Kew [:jfkthame]]

(In reply to Boris Zbarsky (:bz) from comment #3)
> I don't have a good feel for how many bytes per char textruns are.

A bare minimum of 4 bytes per char for the CompressedGlyph array (glyph IDs and advance widths), plus a couple hundred bytes overhead per run. An extra one or two bytes per char if the textrun has to store a copy of the text, because it wasn't marked as "persistent" on creation. And some more for DetailedGlyph and GlyphRun records in cases that involve complex glyph layout, multiple fonts, etc.

So 33mb of text could easily account for 150mb or so.

Comment 5

[Jonathan Kew [:jfkthame]]

I loaded this file in a Linux build with the textrun memory reporting patch from bug 671297; it showed about 94MB used by textruns. Which fits with expectations - that 33mb figure includes a bunch of markup, so it's rather less than half actual text, I expect.

Comment 6

[Nicholas Nethercote [inactive]]

I just tested this with trunk and got 25--29% heap-unclassified.

Then I tested it with a build with the patches from bug 723799, bug 682431 and bug 729008 applied, and I got 7% heap-unclassified.  That's easily the lowest heap-unclassified I've ever seen.  I think bug 682431's patch is mostly responsible.