Closed
Bug 702301
Opened 13 years ago
Closed 13 years ago
Large heap-unclassified (1GB) after a few hours of browsing
Categories
(Core :: General, defect)
Tracking
()
RESOLVED
INVALID
People
(Reporter: developer, Unassigned)
Details
Attachments
(2 files)
User Agent: Mozilla/5.0 (Ubuntu; X11; Linux x86_64; rv:9.0a2) Gecko/20111102 Firefox/9.0a2 Build ID: 20111102041734 Steps to reproduce: I ran a DMD enabled version for a few hours (browsing the web and doing work) and then left it open over the weekend. When I came back in today, I went to about:memory and it showed a resident amount of 1.2GB with only 180MB attributed to memory reporters. I'm attaching the DMD output. My setup was the same as in Bug 699801 except that I have the updated firebug from that bug. about:memory Main Process Explicit Allocations 178.87 MB (100.0%) -- explicit ├──136.63 MB (76.39%) -- js │ ├───51.42 MB (28.75%) -- compartment([System Principal], 0x19479c20) │ │ ├──37.53 MB (20.98%) -- gc-heap │ │ │ ├──14.42 MB (08.06%) -- objects │ │ │ │ ├───7.23 MB (04.04%) -- non-function │ │ │ │ └───7.19 MB (04.02%) -- function │ │ │ ├───9.76 MB (05.46%) -- arena │ │ │ │ ├──9.07 MB (05.07%) -- unused │ │ │ │ └──0.69 MB (00.39%) -- (2 omitted) │ │ │ ├───8.31 MB (04.64%) -- shapes │ │ │ │ ├──5.53 MB (03.09%) -- tree │ │ │ │ └──2.78 MB (01.55%) -- dict │ │ │ ├───2.76 MB (01.54%) -- strings │ │ │ ├───2.11 MB (01.18%) -- scripts │ │ │ └───0.18 MB (00.10%) -- (2 omitted) │ │ ├───3.56 MB (01.99%) -- script-data │ │ ├───2.91 MB (01.63%) -- shapes-extra │ │ │ ├──1.58 MB (00.88%) -- tree-tables │ │ │ └──1.33 MB (00.74%) -- (3 omitted) │ │ ├───2.45 MB (01.37%) -- object-slots │ │ ├───2.28 MB (01.28%) -- string-chars │ │ ├───1.82 MB (01.02%) -- analysis-temporary │ │ └───0.86 MB (00.48%) -- (3 omitted) │ ├───22.89 MB (12.80%) -- compartment(https://192.168.2.84/) │ │ ├──10.21 MB (05.71%) -- string-chars │ │ ├───9.50 MB (05.31%) -- gc-heap │ │ │ ├──4.37 MB (02.44%) -- arena │ │ │ │ ├──4.19 MB (02.34%) -- unused │ │ │ │ └──0.18 MB (00.10%) -- (2 omitted) │ │ │ ├──2.60 MB (01.45%) -- objects │ │ │ │ ├──1.32 MB (00.74%) -- function │ │ │ │ └──1.28 MB (00.71%) -- non-function │ │ │ ├──1.59 MB (00.89%) -- shapes │ │ │ │ ├──1.13 MB (00.63%) -- tree │ │ │ │ └──0.46 MB (00.26%) -- (1 omitted) │ │ │ └──0.95 MB (00.53%) -- (3 omitted) │ │ ├───1.12 MB (00.62%) -- shapes-extra │ │ │ └──1.12 MB (00.62%) -- (4 omitted) │ │ ├───1.09 MB (00.61%) -- (4 omitted) │ │ └───0.98 MB (00.55%) -- script-data │ ├───21.36 MB (11.94%) -- gc-heap-decommitted │ ├───10.84 MB (06.06%) -- compartment(http://www.google.com/search?q=webdriver+file+upload&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:unofficial&client=firefox-a) │ │ ├───6.66 MB (03.72%) -- gc-heap │ │ │ ├──2.81 MB (01.57%) -- objects │ │ │ │ ├──2.12 MB (01.18%) -- function │ │ │ │ └──0.70 MB (00.39%) -- (1 omitted) │ │ │ ├──1.61 MB (00.90%) -- shapes │ │ │ │ ├──1.37 MB (00.76%) -- tree │ │ │ │ └──0.24 MB (00.13%) -- (1 omitted) │ │ │ ├──1.29 MB (00.72%) -- scripts │ │ │ └──0.95 MB (00.53%) -- (3 omitted) │ │ ├───2.13 MB (01.19%) -- script-data │ │ └───2.05 MB (01.14%) -- (6 omitted) │ ├────8.47 MB (04.73%) -- compartment(http://code.google.com/p/selenium/wiki/GettingStarted) │ │ ├──5.77 MB (03.23%) -- gc-heap │ │ │ ├──2.12 MB (01.18%) -- objects │ │ │ │ ├──1.39 MB (00.78%) -- function │ │ │ │ └──0.72 MB (00.41%) -- (1 omitted) │ │ │ ├──1.38 MB (00.77%) -- shapes │ │ │ │ ├──1.08 MB (00.60%) -- tree │ │ │ │ └──0.30 MB (00.17%) -- (1 omitted) │ │ │ ├──1.26 MB (00.70%) -- arena │ │ │ │ ├──1.14 MB (00.64%) -- unused │ │ │ │ └──0.12 MB (00.07%) -- (2 omitted) │ │ │ └──1.01 MB (00.57%) -- (3 omitted) │ │ ├──1.56 MB (00.87%) -- (6 omitted) │ │ └──1.14 MB (00.64%) -- script-data │ ├────8.00 MB (04.47%) -- stack │ ├────6.51 MB (03.64%) -- compartment(atoms) │ │ ├──4.46 MB (02.50%) -- gc-heap │ │ │ ├──4.10 MB (02.29%) -- strings │ │ │ └──0.36 MB (00.20%) -- (1 omitted) │ │ └──2.05 MB (01.15%) -- string-chars │ ├────2.32 MB (01.30%) -- runtime │ │ ├──2.00 MB (01.12%) -- atoms-table │ │ └──0.32 MB (00.18%) -- (1 omitted) │ ├────1.42 MB (00.79%) -- compartment(http://seleniumhq.org/docs/03_webdriver.html) │ │ └──1.42 MB (00.79%) -- (8 omitted) │ ├────1.39 MB (00.78%) -- gc-heap-chunk-admin │ ├────1.11 MB (00.62%) -- compartment(http://planet.mozilla.org/) │ │ └──1.11 MB (00.62%) -- (8 omitted) │ └────0.92 MB (00.51%) -- (7 omitted) ├───16.98 MB (09.49%) -- storage │ └──16.98 MB (09.49%) -- sqlite │ ├───9.11 MB (05.10%) -- places.sqlite │ │ ├──8.74 MB (04.88%) -- cache-used [3] │ │ └──0.38 MB (00.21%) -- (2 omitted) │ ├───5.07 MB (02.83%) -- webappsstore.sqlite │ │ ├──5.01 MB (02.80%) -- cache-used │ │ └──0.07 MB (00.04%) -- (2 omitted) │ ├───1.85 MB (01.03%) -- (12 omitted) │ └───0.94 MB (00.53%) -- other ├───14.43 MB (08.07%) -- layout │ ├───2.31 MB (01.29%) -- (6 omitted) │ ├───1.62 MB (00.91%) -- shell(http://www.google.com/search?q=webdriver+file+upload&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:unofficial&client=firefox-a) │ │ ├──1.02 MB (00.57%) -- styledata │ │ └──0.60 MB (00.34%) -- (1 omitted) │ ├───1.61 MB (00.90%) -- shell(http://www.google.com/search?hl=en&source=hp&q=debug%20firefox%20webdriver&aq=f&aqi=&aql=&oq=&gs_rfai=) │ │ ├──1.02 MB (00.57%) -- styledata │ │ └──0.59 MB (00.33%) -- (1 omitted) │ ├───1.59 MB (00.89%) -- shell(http://www.google.com/search?hl=en&source=hp&q=webdriver%20profile&aq=f&aqi=&aql=&oq=&gs_rfai=) │ │ ├──1.02 MB (00.57%) -- styledata │ │ └──0.57 MB (00.32%) -- (1 omitted) │ ├───1.50 MB (00.84%) -- shell(chrome://browser/content/browser.xul) │ │ ├──0.98 MB (00.55%) -- arenas │ │ └──0.52 MB (00.29%) -- (2 omitted) │ ├───1.28 MB (00.72%) -- shell(http://code.google.com/p/selenium/wiki/DeveloperTips) │ │ ├──0.93 MB (00.52%) -- styledata │ │ └──0.35 MB (00.20%) -- (1 omitted) │ ├───1.27 MB (00.71%) -- shell(http://code.google.com/p/selenium/wiki/FirefoxDriver) │ │ ├──0.93 MB (00.52%) -- styledata │ │ └──0.34 MB (00.19%) -- (2 omitted) │ ├───1.27 MB (00.71%) -- shell(http://code.google.com/p/selenium/wiki/BuildingWebDriver) │ │ ├──0.93 MB (00.52%) -- styledata │ │ └──0.34 MB (00.19%) -- (1 omitted) │ ├───1.03 MB (00.58%) -- shell(https://192.168.2.84/users/login?r=%2Fdeepsee#{%22ac%22:%22Summary%22,%22pb%22:[%22favorite=500%22]}) │ │ ├──0.95 MB (00.53%) -- styledata │ │ └──0.09 MB (00.05%) -- (1 omitted) │ └───0.94 MB (00.53%) -- shell(https://bug699801.bugzilla.mozilla.org/attachment.cgi?id=573204) │ └──0.94 MB (00.53%) -- (2 omitted) ├────3.58 MB (02.00%) -- spell-check ├────1.97 MB (01.10%) -- dom ├────1.58 MB (00.88%) -- startup-cache ├────1.54 MB (00.86%) -- images │ ├──1.36 MB (00.76%) -- content │ │ ├──1.36 MB (00.76%) -- used │ │ │ ├──1.35 MB (00.75%) -- raw │ │ │ └──0.01 MB (00.01%) -- (2 omitted) │ │ └──0.00 MB (00.00%) -- (1 omitted) │ └──0.18 MB (00.10%) -- (1 omitted) ├────1.33 MB (00.75%) -- xpti-working-set └────0.82 MB (00.46%) -- (6 omitted) Resident Set Size (RSS) Breakdown Proportional Set Size (PSS) Breakdown Virtual Size Breakdown Swap Usage Breakdown Other Measurements 0.19 MB -- gfx-surface-image 23.55 MB -- gfx-surface-xlib 0.00 MB -- heap-allocated [*] 0.00 MB -- heap-unallocated [*] 3 -- js-compartments-system 11 -- js-compartments-user 89.00 MB -- js-gc-heap 15.74 MB -- js-gc-heap-arena-unused 0.00 MB -- js-gc-heap-chunk-clean-unused 0.17 MB -- js-gc-heap-chunk-dirty-unused 21.36 MB -- js-gc-heap-decommitted 41.87% -- js-gc-heap-unused-fraction 2.30 MB -- js-total-analysis-temporary 2.12 MB -- js-total-mjit 25.89 MB -- js-total-objects 12.99 MB -- js-total-scripts 18.17 MB -- js-total-shapes 21.74 MB -- js-total-strings 2.67 MB -- js-total-type-inference 485 -- page-faults-hard 4,374,549 -- page-faults-soft 1,235.40 MB -- resident 2,017.95 MB -- vsize
Blocks: DarkMatter
|
||
Comment 2•13 years ago
|
||
If you still have this instance open, a copy of about:memory?verbose with the RSS and PSS breakdowns expanded would be interesting.
|
|
||
Comment 3•13 years ago
|
||
How is this bug any different from bug 699801?
Sorry, I don't have the instance open anymore. I'll remember that the next time. And it is different from 699801 because that was caused by uploading large files and firebug was leaking the large files. With the updated firebug from 699801, it no longer leaks those files and I also didn't upload any large files in this session. Also, you'll notice in the dmd output from the two bugs are extremely different. bug 699801 has a bunch of large nsBinaryInputStream traces at the top because of the file uploads, while this bug only has 1 of those at the very top.
|
|
||
Comment 5•13 years ago
|
||
I see two potential issues here: 1) DMD doesn't account for this extra resident memory (the difference between explicit and resident). This could be a bug in DMD, or it could be that one of your extensions is increasing RSS in a way that DMD can't notice, for example by mmap'ing large files. But more likely, the |resident| number is meaningless, because you're running inside valgrind, which is using up tons of extra memory for its own purposes, and Firefox can't distinguish between valgrind's memory usage and its own usage. In fact, this is almost certainly what's going on. 2) Detected dark matter: 2a) 30mb of binary input streams. This is probably an extension, as in bug 699801. 2b) 17mb of localstorage (?) strings. This is also probably an extension. The rest of the DMD output looks pretty reasonable to me. I'm not sure that it's a darkmatter bug that storage used by extensions doesn't show up in about:memory. That seems like an extension bug, if it's a bug at all. So I'm removing the |darkmatter| dependency. If the facts change, we can add it back.
|
|
||
Updated•13 years ago
|
No longer blocks: DarkMatter
|
|
||
Comment 6•13 years ago
|
||
Since I'm pretty sure the large heap-unclassified is actually a measurement issue, perhaps you should close this bug as INVALID and file a separate issue on the dark matter in the extensions, or alternatively edit the summary of this bug.
|
||
Comment 7•13 years ago
|
||
> The rest of the DMD output looks pretty reasonable to me.
Except it's showing the css hashtables. We have memory reporters for those, no?|
|
||
Comment 8•13 years ago
|
||
The topmost caller is
> ==24692== by 0x7F49514: nsCSSFrameConstructor::ConstructRootFrame(nsIFrame**) (nsCSSFrameConstructor.cpp:2541)
Knowing nothing about DMD or this code, it seems like DMD just caught the CSS engine in the middle of an operation and that these allocations are temporary.|
|
||
Comment 9•13 years ago
|
||
> The topmost caller is That doesn't matter much here; what matters is the RefreshRuleCascade on the stack. That's the operation that builds (lazily, which is why it only happened when we needed style data for frame construction) the rulehash and rule cascade data. These are decidedly not temporary allocations; they're the allocations we added memory reporters for in bug 676048.
|
|
||
Comment 10•13 years ago
|
||
I filed bug 702984 on the RuleHash output. Perhaps the DMD patch isn't aware of this memory reporter so is just missing this one.
|
||
Comment 11•13 years ago
|
||
I have an ordinary Firefox 9 beta channel build (though I just noticed there's an update I haven't taken; I'll hold off on that in case you want any other diagnostics on this instance), and get similarly large heap-unclassified. about:memory?verbose (after clicking "minimize memory usage") attached. Firefox often stalls for up to a minute for me when the system has any kind of memory pressure at all (but I have 8GB of RAM and the only significant memory usage I have is Firefox!); perhaps this has something to do with it?
Application Basics
Name
Firefox
Version
9.0
User Agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0) Gecko/20100101 Firefox/9.0
Profile Directory
Show in Finder
Enabled Plugins
about:plugins
Build Configuration
about:buildconfig
Crash Reports
about:crashes
Memory Use
about:memory
Extensions
Name
Version
Enabled
ID
A Bit Better RTM
0.0.2.9
true
andrew@paprotsky.com
Diccionario español Mexico
1.1.2
true
es-MX@dictionaries.addons.mozilla.org
DownloadHelper
4.9.7
true
{b9db16a4-6edc-47ec-a1f4-b86292ed211d}
Firebug
1.8.4
true
firebug@software.joehewitt.com
Greasemonkey
0.9.13.beta1
true
{e4a8a97b-f2ed-450b-b12d-ee082ba24781}
Test Pilot
1.2
true
testpilot@labs.mozilla.com
Web Developer
1.1.9
true
{c45c406e-ab73-11d8-be73-000a95be3b12}
Evernote Web Clipper
5.0.0.177572
false
{E0B8C461-F8FB-49b4-8373-FE32E9252800}
Modified Preferences
Name
Value
accessibility.typeaheadfind.flashBar
0
browser.places.smartBookmarksVersion
2
browser.startup.homepage_override.buildID
20111122192043
browser.startup.homepage_override.mstone
rv:9.0
extensions.lastAppVersion
9.0
network.cookie.prefsMigrated
true
places.database.lastMaintenance
1322934522
places.history.expiration.transient_current_max_pages
104858
places.history.expiration.transient_optimal_database_size
167772160
print.macosx.pagesetup-2
PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz4KPCFET0NUWVBFIHBsaXN0IFBVQkxJQyAiLS8vQXBwbGUvL0RURCBQTElTVCAxLjAvL0VO…
print.print_bgcolor
false
print.print_bgimages
false
print.print_command
print.print_downloadfonts
false
print.print_evenpages
true
print.print_in_color
true
print.print_margin_bottom
0.5
print.print_margin_left
0.5
print.print_margin_right
0.5
print.print_margin_top
0.5
print.print_oddpages
true
print.print_orientation
0
print.print_page_delay
50
print.print_paper_data
0
print.print_paper_height
11.00
print.print_paper_size_type
1
print.print_paper_size_unit
0
print.print_paper_width
8.50
print.print_printer
print.print_reversed
false
print.print_scaling
1.00
print.print_shrink_to_fit
true
print.print_to_file
false
print.print_unwriteable_margin_bottom
56
print.print_unwriteable_margin_left
25
print.print_unwriteable_margin_right
25
print.print_unwriteable_margin_top
25
privacy.cpd.cookies
false
privacy.cpd.downloads
false
privacy.cpd.formdata
false
privacy.cpd.history
false
privacy.cpd.sessions
false
privacy.sanitize.migrateFx3Prefs
true
security.disable_button.openCertManager
false
security.warn_viewing_mixed
false
Graphics
Adapter Description
0x24300,0x20400
WebGL Renderer
Intel Inc. -- Intel HD Graphics 3000 OpenGL Engine -- 2.1 APPLE-1.6.38
GPU Accelerated Windows
3/3 OpenGL|
|
||
Comment 12•13 years ago
|
||
|
Reporter | |
Comment 13•13 years ago
|
||
Marking as invalid as per comment #6. Ken, you might want to add your stuff to a different bug.
Status: UNCONFIRMED → RESOLVED
Closed: 13 years ago
Resolution: --- → INVALID
You need to log in
before you can comment on or make changes to this bug.
Description
•