658814 - about:memory: Negative "heap-used/other" value due to overlap between image and gfx memory reporters

Status: RESOLVED FIXED

(Toolkit :: General, defect)

Description

[Nicholas Nethercote [inactive]]

Jesse saw the following output.  It looks very likely that the images/* reporters and the gfx/surface/image reporter are double-counting some memory.  CC'ing some gfx people.


Main Process

Mapped Memory
3,606,937,600 B (100.0%) -- mapped
├──3,183,395,936 B (88.26%) -- other
├────417,708,960 B (11.58%) -- heap
│    ├──366,023,808 B (10.15%) -- used
│    └───51,685,152 B (01.43%) -- unused
└──────5,832,704 B (00.16%) -- js
       ├──5,308,416 B (00.15%) -- mjit-code
       └────524,288 B (00.01%) -- tjit-code

Used Heap Memory
366,023,808 B (100.0%) -- heap-used
├──138,558,573 B (37.86%) -- images
│  ├──138,237,501 B (37.77%) -- content
│  │  ├──135,305,282 B (36.97%) -- used
│  │  │  ├──119,908,256 B (32.76%) -- uncompressed
│  │  │  └───15,397,026 B (04.21%) -- raw
│  │  └────2,932,219 B (00.80%) -- unused
│  │       ├──2,594,860 B (00.71%) -- uncompressed
│  │       └────337,359 B (00.09%) -- raw
│  └──────321,072 B (00.09%) -- chrome
│         ├──321,072 B (00.09%) -- used
│         │  ├──321,072 B (00.09%) -- uncompressed
│         │  └────────0 B (00.00%) -- raw
│         └────────0 B (00.00%) -- unused
│                  ├──0 B (00.00%) -- raw
│                  └──0 B (00.00%) -- uncompressed
├──122,852,976 B (33.56%) -- gfx
│  └──122,852,976 B (33.56%) -- surface
│     └──122,852,976 B (33.56%) -- image
├───80,783,240 B (22.07%) -- js
│   ├──77,594,624 B (21.20%) -- gc-heap
│   ├───2,320,224 B (00.63%) -- tjit-data
│   │   └──2,320,224 B (00.63%) -- allocators
│   │      ├──1,776,000 B (00.49%) -- reserve
│   │      └────544,224 B (00.15%) -- main
│   └─────868,392 B (00.24%) -- mjit-data
├───49,715,672 B (13.58%) -- storage
│   └──49,715,672 B (13.58%) -- sqlite
│      ├──40,946,592 B (11.19%) -- urlclassifier3.sqlite
│      │  ├──40,841,408 B (11.16%) -- cache-used
│      │  ├─────100,944 B (00.03%) -- stmt-used
│      │  └───────4,240 B (00.00%) -- schema-used
│      ├───5,191,008 B (01.42%) -- places.sqlite
│      │   ├──4,717,368 B (01.29%) -- cache-used
│      │   ├────395,280 B (00.11%) -- stmt-used
│      │   └─────78,360 B (00.02%) -- schema-used
│      ├─────984,128 B (00.27%) -- other
│      ├─────733,512 B (00.20%) -- cookies.sqlite
│      │     ├──718,040 B (00.20%) -- cache-used
│      │     ├───12,528 B (00.00%) -- stmt-used
│      │     └────2,944 B (00.00%) -- schema-used
│      ├─────613,256 B (00.17%) -- extensions.sqlite
│      │     ├──429,848 B (00.12%) -- cache-used
│      │     ├──172,192 B (00.05%) -- stmt-used
│      │     └───11,216 B (00.00%) -- schema-used
│      ├─────341,480 B (00.09%) -- addons.sqlite
│      │     ├──264,760 B (00.07%) -- cache-used
│      │     ├───69,760 B (00.02%) -- stmt-used
│      │     └────6,960 B (00.00%) -- schema-used
│      ├─────335,680 B (00.09%) -- chromeappsstore.sqlite
│      │     ├──265,144 B (00.07%) -- cache-used
│      │     ├───64,184 B (00.02%) -- stmt-used
│      │     └────6,352 B (00.00%) -- schema-used
│      ├─────265,904 B (00.07%) -- downloads.sqlite
│      │     ├──256,312 B (00.07%) -- cache-used
│      │     ├────6,768 B (00.00%) -- stmt-used
│      │     └────2,824 B (00.00%) -- schema-used
│      ├─────206,152 B (00.06%) -- webappsstore.sqlite
│      │     ├──135,616 B (00.04%) -- cache-used
│      │     ├───64,184 B (00.02%) -- stmt-used
│      │     └────6,352 B (00.00%) -- schema-used
│      ├──────33,336 B (00.01%) -- formhistory.sqlite
│      │      ├──19,832 B (00.01%) -- stmt-used
│      │      ├──10,824 B (00.00%) -- cache-used
│      │      └───2,680 B (00.00%) -- schema-used
│      ├──────21,936 B (00.01%) -- content-prefs.sqlite
│      │      ├───9,632 B (00.00%) -- stmt-used
│      │      ├───8,280 B (00.00%) -- cache-used
│      │      └───4,024 B (00.00%) -- schema-used
│      ├──────17,264 B (00.00%) -- signons.sqlite
│      │      ├───8,264 B (00.00%) -- cache-used
│      │      ├───4,528 B (00.00%) -- stmt-used
│      │      └───4,472 B (00.00%) -- schema-used
│      ├──────13,296 B (00.00%) -- permissions.sqlite
│      │      ├───6,776 B (00.00%) -- stmt-used
│      │      ├───4,464 B (00.00%) -- cache-used
│      │      └───2,056 B (00.00%) -- schema-used
│      └──────12,128 B (00.00%) -- search.sqlite
│             ├───5,672 B (00.00%) -- stmt-used
│             ├───4,448 B (00.00%) -- cache-used
│             └───2,008 B (00.00%) -- schema-used
├────8,640,641 B (02.36%) -- layout
│    ├──8,640,641 B (02.36%) -- all
│    └──────────0 B (00.00%) -- bidi
└──-34,527,294 B (-9.43%) -- other

Other Measurements
544,038,912 B -- resident
366,025,952 B -- heap-zone0-committed
413,515,776 B -- heap-zone0-used

Comment 1

[Nicholas Nethercote [inactive]]

Bug 660577 comment 27 has a similar example.  The problem shows up on any page that has lots of images.

Comment 2

[Nicholas Nethercote [inactive]]

Attachment: patch

This is the simple, brute force approach -- just move all the gfx/surface/* reporters out of the "explicit allocations" tree into the "other measurements" tree.

Comment 3

[Jeff Muizelaar [:jrmuizel]]

Comment on attachment 536822 [details] [diff] [review]
patch

This is not a patch :)

Comment 4

[Nicholas Nethercote [inactive]]

Attachment: patch v2

Attempt 2!

Comment 5

[Nicholas Nethercote [inactive]]

Attachment: patch v3

Attempt 3!

Comment 6

[Nicholas Nethercote [inactive]]

I set the tracking-firefox6? flag because this is a low-risk fix to a bug that was introduced by the about:memory revamp, which was first introduced in Firefox 6.

Comment 7

[Nicholas Nethercote [inactive]]

http://hg.mozilla.org/mozilla-central/rev/98ba431916e4

Comment 8

[Asa Dotzler [:asa]]

not going to track. please nominate the patch with a risk assessment.

Comment 9

[Nicholas Nethercote [inactive]]

Comment on attachment 537232 [details] [diff] [review]
patch v3

Requesting approval-mozilla-aurora.  The benefit of this patch is that it avoids some nonsensical negative numbers in about:memory.  The risk is extremely low, it just renames and reclassifies the gfx memory reporters so they are shown in the "Other Measurements" part of about:memory instead of the "Explicit Allocations" part.

Comment 10

[Johnathan Nightingale [:johnath]]

Comment on attachment 537232 [details] [diff] [review]
patch v3

Giddyup

Comment 11

[Nicholas Nethercote [inactive]]

http://hg.mozilla.org/releases/mozilla-aurora/rev/1d0b9b879d73

Comment 12

[Daniel Holbert [:dholbert]]

Setting status-firefox6=fixed per comment 11, so that this will disappear from the "approved but not yet landed" bug list.