Closed Bug 799535 Opened 13 years ago Closed 12 years ago

Progressive Memory Leak, up to 2.5GB even in safe mode

Categories

(Thunderbird :: General, defect)

Product:
Thunderbird
Component:
General
Version:
16 Branch
Platform:
x86_64
Windows 7
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
RESOLVED INCOMPLETE

People

(Reporter: ricky, Unassigned)

References

Details

(Keywords: perf, Whiteboard: [gone after new profile, cause unknown])

Attachments

(1 file)

DxDiag
44.75 KB, text/plain
Details
Attached file DxDiag —
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20100101 Firefox/16.0 Build ID: 20121002073616 Steps to reproduce: Simply used Thunderbird with all extensions and add-ons disabled. Actual results: Based on system uptime (69 hours) and the fact Thunderbird has been open the whole time, Thunderbird has been allocating memory (currently at 2.5GB) at an approximate rate of 37MB per hour. This also occurred with Thunderbird 15.01 stable. The client has 4 mailboxes, 3 dovecot servers and 1 gmail (with 'All Mail' unsubscribed from). Client system: Windows 7 x86_64, DxDiag attached for additional system info.
(Mozilla IRC nick: 'Orbixx')
(Grabbed from Troubleshooting Information, sorry for the multiple emails) Application Basics Name: Thunderbird Version: 16.0 User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121002 Thunderbird/16.0 Profile Folder: Show Folder (Local drive) Application Build ID: 20121002100030 Enabled Plugins: about:plugins Build Configuration: about:buildconfig Crash Reports: about:crashes Memory Use: about:memory Mail and News Accounts account1: INCOMING: account1, , (imap) mail.exoware.net:143, alwaysSTARTTLS, passwordCleartext OUTGOING: node2.exoware.net:465, alwaysSTARTTLS, passwordCleartext, true account2: INCOMING: account2, , (none) Local Folders, plain, passwordCleartext account3: INCOMING: account3, , (imap) mail.exoware.net:143, alwaysSTARTTLS, passwordCleartext OUTGOING: smtp.exoware.net:465, alwaysSTARTTLS, passwordCleartext, true account4: INCOMING: account4, , (imap) mail.exoware.net:143, alwaysSTARTTLS, passwordCleartext OUTGOING: smtp.exoware.net:465, alwaysSTARTTLS, passwordCleartext, true account5: INCOMING: account5, , (imap) imap.googlemail.com:993, SSL, passwordCleartext OUTGOING: smtp.googlemail.com:465, SSL, passwordCleartext, true Extensions Enigmail, 1.4.4, false, {847b3a00-7ab1-11d4-8f02-006008948af5} Lightning, 1.7, false, {e2fda1a4-762b-4020-b5ad-a41df1933103} Provider for Google Calendar, 0.16, false, {a62ef8ec-5fdc-40c2-873c-223b8a6925cc} Test Pilot for Thunderbird, 1.3.9, false, tbtestpilot@labs.mozilla.com Important Modified Preferences Name: Value accessibility.typeaheadfind.flashBar: 0 browser.cache.disk.capacity: 1048576 browser.cache.disk.smart_size.first_run: false browser.cache.disk.smart_size_cached_value: 727040 extensions.lastAppVersion: 16.0 font.internaluseonly.changed: false font.name.monospace.el: Consolas font.name.monospace.tr: Consolas font.name.monospace.x-baltic: Consolas font.name.monospace.x-central-euro: Consolas font.name.monospace.x-cyrillic: Consolas font.name.monospace.x-unicode: Consolas font.name.monospace.x-western: Consolas font.name.sans-serif.el: Calibri font.name.sans-serif.tr: Calibri font.name.sans-serif.x-baltic: Calibri font.name.sans-serif.x-central-euro: Calibri font.name.sans-serif.x-cyrillic: Calibri font.name.sans-serif.x-unicode: Calibri font.name.sans-serif.x-western: Calibri font.name.serif.el: Cambria font.name.serif.tr: Cambria font.name.serif.x-baltic: Cambria font.name.serif.x-central-euro: Cambria font.name.serif.x-cyrillic: Cambria font.name.serif.x-unicode: Cambria font.name.serif.x-western: Cambria font.size.fixed.el: 14 font.size.fixed.tr: 14 font.size.fixed.x-baltic: 14 font.size.fixed.x-central-euro: 14 font.size.fixed.x-cyrillic: 14 font.size.fixed.x-unicode: 14 font.size.fixed.x-western: 14 font.size.variable.el: 17 font.size.variable.tr: 17 font.size.variable.x-baltic: 17 font.size.variable.x-central-euro: 17 font.size.variable.x-cyrillic: 17 font.size.variable.x-unicode: 17 font.size.variable.x-western: 17 mail.openMessageBehavior.version: 1 mail.winsearch.enable: true mail.winsearch.firstRunDone: true mail.winsearch.global_reindex_time: 1285706875 mailnews.database.global.datastore.id: b7a97a55-882e-4b38-b472-478d2babcd2 network.cookie.prefsMigrated: true places.database.lastMaintenance: 1349400840 places.history.expiration.transient_current_max_pages: 104858 places.history.expiration.transient_optimal_database_size: 167772160 security.disable_button.openCertManager: false Graphics Adapter Description: NVIDIA GeForce GTX 560 Ti Vendor ID: 10de Device ID: 1200 Adapter RAM: 1023 Adapter Drivers: nvd3dumx,nvwgf2umx,nvwgf2umx nvd3dum,nvwgf2um,nvwgf2um Driver Version: 9.18.13.602 Driver Date: 8-22-2012 Direct2D Enabled: false DirectWrite Enabled: false (6.1.7601.17789) ClearType Parameters: ClearType parameters not found WebGL Renderer: Google Inc. -- ANGLE (NVIDIA GeForce GTX 560 Ti ) -- OpenGL ES 2.0 (ANGLE 1.0.0.1041) GPU Accelerated Windows: 0/1
click on the "Memory Use: about:memory" entry and attach the output as file
Seems it crashed or Windows killed it after consuming so much memory. Shall I wait a little while for it to consume a couple of hundred of extra megabytes before pasting, or will it not particularly matter?
The output is only useful if TB already consumes the memory.
Okay, it seems roughly 90-100MB has been consumed since launch. Here's the output. I'll leave it running overnight just in case it's inconclusive and it ought to have gone up by around 200MB by the time I get up. Main Process Explicit Allocations 305.88 MB (100.0%) -- explicit ├──249.45 MB (81.55%) ── heap-unclassified ├───30.02 MB (09.81%) -- js-non-window │ ├──23.76 MB (07.77%) -- compartments │ │ ├──22.23 MB (07.27%) -- non-window-global │ │ │ ├──18.83 MB (06.16%) ++ (196 tiny) │ │ │ └───3.39 MB (01.11%) ++ compartment([System Principal], resource:///modules/gloda/suffixtree.js) │ │ └───1.53 MB (00.50%) ++ no-global │ ├───4.62 MB (01.51%) -- gc-heap │ │ ├──4.24 MB (01.39%) ── decommitted-arenas │ │ └──0.38 MB (00.12%) ++ (3 tiny) │ └───1.64 MB (00.54%) ++ runtime ├───10.36 MB (03.39%) -- window-objects │ ├───6.04 MB (01.97%) -- top(chrome://messenger/content/messenger.xul, id=3)/active │ │ ├──4.54 MB (01.48%) ++ window(chrome://messenger/content/messenger.xul) │ │ └──1.49 MB (00.49%) ++ (3 tiny) │ └───4.33 MB (01.41%) ++ (8 tiny) ├────8.36 MB (02.73%) -- storage │ ├──8.36 MB (02.73%) -- sqlite │ │ ├──5.07 MB (01.66%) -- global-messages-db.sqlite │ │ │ ├──4.96 MB (01.62%) ── cache-used │ │ │ └──0.11 MB (00.04%) ++ (2 tiny) │ │ └──3.29 MB (01.08%) ++ (10 tiny) │ └──0.00 MB (00.00%) ── prefixset/all └────7.70 MB (02.52%) ++ (11 tiny) Other Measurements 220 (100.0%) -- js-compartments ├──214 (97.27%) ── system └────6 (02.73%) ── user 36.17 MB (100.0%) -- js-main-runtime ├──29.91 MB (82.70%) -- compartments │ ├──19.38 MB (53.59%) -- gc-heap │ │ ├──11.47 MB (31.71%) ── unused-gc-things │ │ ├───3.20 MB (08.85%) -- objects │ │ │ ├──1.66 MB (04.60%) ── non-function │ │ │ └──1.54 MB (04.25%) ── function │ │ ├───2.84 MB (07.86%) -- shapes │ │ │ ├──1.62 MB (04.48%) ── tree │ │ │ ├──0.65 MB (01.79%) ── base │ │ │ └──0.58 MB (01.59%) ── dict │ │ ├───0.94 MB (02.60%) ── scripts │ │ ├───0.69 MB (01.92%) ── strings │ │ └───0.23 MB (00.65%) ++ (3 tiny) │ ├───3.15 MB (08.71%) ── script-data │ ├───2.24 MB (06.18%) ── analysis-temporary │ ├───1.85 MB (05.11%) -- shapes-extra │ │ ├──0.80 MB (02.21%) ── compartment-tables │ │ ├──0.58 MB (01.60%) ── tree-tables │ │ └──0.47 MB (01.30%) ++ (2 tiny) │ ├───1.37 MB (03.80%) -- objects │ │ ├──1.31 MB (03.63%) ── slots │ │ └──0.06 MB (00.16%) ++ (2 tiny) │ ├───1.26 MB (03.48%) ── string-chars │ ├───0.56 MB (01.56%) ── cross-compartment-wrappers │ └───0.10 MB (00.28%) ++ (3 tiny) ├───4.62 MB (12.77%) -- gc-heap │ ├──4.24 MB (11.73%) ── decommitted-arenas │ ├──0.38 MB (01.04%) ── chunk-admin │ └──0.00 MB (00.00%) ++ (2 tiny) └───1.64 MB (04.53%) ── runtime 19.76 MB (100.0%) -- js-main-runtime-gc-heap-committed ├──11.47 MB (58.04%) -- unused │ ├──11.47 MB (58.04%) ── gc-things │ └───0.00 MB (00.00%) ++ (2 tiny) └───8.29 MB (41.96%) -- used ├──7.79 MB (39.44%) ── gc-things ├──0.38 MB (01.90%) ── chunk-admin └──0.12 MB (00.61%) ── arena-admin 0 (100.0%) -- low-memory-events ├──0 (100.0%) ── physical └──0 (100.0%) ── virtual 4.21 MB (100.0%) -- window-objects ├──2.65 MB (62.82%) -- layout │ ├──1.21 MB (28.83%) ── style-sets │ ├──0.73 MB (17.28%) ── pres-shell │ ├──0.30 MB (07.19%) ── frames │ ├──0.18 MB (04.20%) ── style-contexts │ ├──0.14 MB (03.29%) ── rule-nodes │ ├──0.05 MB (01.27%) ── pres-contexts │ └──0.03 MB (00.77%) ++ (2 tiny) ├──0.96 MB (22.82%) ── style-sheets ├──0.60 MB (14.26%) -- dom │ ├──0.34 MB (08.11%) ── element-nodes │ ├──0.19 MB (04.56%) ── other │ ├──0.05 MB (01.23%) ── text-nodes │ └──0.02 MB (00.36%) ++ (3 tiny) └──0.00 MB (00.10%) ── property-tables 305.89 MB ── explicit 0.00 MB ── gfx-d2d-surfacecache 0.00 MB ── gfx-d2d-surfacevram 0.00 MB ── gfx-d2d-vram-drawtarget 0.00 MB ── gfx-d2d-vram-sourcesurface 0.88 MB ── gfx-surface-win32 0 ── ghost-windows 279.32 MB ── heap-allocated 288.40 MB ── heap-committed 9.07 MB ── heap-committed-unused 3.24% ── heap-committed-unused-ratio 2.46 MB ── heap-dirty 18.68 MB ── heap-unused 0.38 MB ── images-content-used-uncompressed 24.00 MB ── js-gc-heap 0 ── low-commit-space-events 369.68 MB ── private 366.69 MB ── resident 8.36 MB ── storage-sqlite 692.18 MB ── vsize
(Note, this was with version 16 beta. And nets to roughly 700MB increase per day) Over that 69 hour time period in safe mode ... what activities were you doing, or occurred automatically? THere are a few other recent bug reports like this. Irving was checking into one of them, but got nowhere so far. I don't have any ideas at the moment, except to update to TB17 beta in a couple days and see if it still reproduces.
Severity: normal → major
Keywords: perf
Summary: Progressive Memory Leak → Progressive Memory Leak, up to 2.5GB even in safe mode
It is a pity we have som much memory in heap-unclassified. Even running and toying around in a test install of TB19 (almost no messages) produces 60MB.
14 hours later and Thunderbird is now consuming 1GB of memory. Main Process Explicit Allocations 1,004.38 MB (100.0%) -- explicit ├────945.25 MB (94.11%) ── heap-unclassified ├─────29.92 MB (02.98%) -- js-non-window │ ├──24.59 MB (02.45%) -- compartments │ │ ├──23.01 MB (02.29%) ++ non-window-global │ │ └───1.57 MB (00.16%) ++ no-global │ └───5.34 MB (00.53%) ++ (2 tiny) ├─────11.75 MB (01.17%) ++ window-objects ├─────10.47 MB (01.04%) -- storage │ ├──10.47 MB (01.04%) ++ sqlite │ └───0.00 MB (00.00%) ── prefixset/all └──────6.99 MB (00.70%) ++ (11 tiny) Other Measurements 219 (100.0%) -- js-compartments ├──212 (96.80%) ── system └────7 (03.20%) ── user 35.28 MB (100.0%) -- js-main-runtime ├──29.94 MB (84.87%) -- compartments │ ├──19.24 MB (54.54%) -- gc-heap │ │ ├──11.29 MB (32.01%) ── unused-gc-things │ │ ├───3.25 MB (09.22%) -- objects │ │ │ ├──1.73 MB (04.91%) ── non-function │ │ │ └──1.52 MB (04.32%) ── function │ │ ├───2.79 MB (07.91%) -- shapes │ │ │ ├──1.61 MB (04.56%) ── tree │ │ │ ├──0.65 MB (01.84%) ── base │ │ │ └──0.53 MB (01.51%) ── dict │ │ ├───0.95 MB (02.68%) ── scripts │ │ ├───0.73 MB (02.07%) ── strings │ │ └───0.23 MB (00.64%) ++ (3 tiny) │ ├───3.17 MB (08.98%) ── script-data │ ├───2.37 MB (06.71%) ── analysis-temporary │ ├───1.82 MB (05.17%) -- shapes-extra │ │ ├──0.78 MB (02.22%) ── compartment-tables │ │ ├──0.59 MB (01.67%) ── tree-tables │ │ └──0.45 MB (01.28%) ++ (2 tiny) │ ├───1.37 MB (03.88%) -- objects │ │ ├──1.31 MB (03.71%) ── slots │ │ └──0.06 MB (00.17%) ++ (2 tiny) │ ├───1.31 MB (03.71%) ── string-chars │ ├───0.57 MB (01.60%) ── cross-compartment-wrappers │ └───0.10 MB (00.28%) ++ (3 tiny) ├───3.76 MB (10.66%) -- gc-heap │ ├──3.40 MB (09.64%) ── decommitted-arenas │ ├──0.36 MB (01.02%) ── chunk-admin │ └──0.00 MB (00.00%) ++ (2 tiny) └───1.58 MB (04.47%) ── runtime 19.60 MB (100.0%) -- js-main-runtime-gc-heap-committed ├──11.29 MB (57.62%) -- unused │ ├──11.29 MB (57.62%) ── gc-things │ └───0.00 MB (00.00%) ++ (2 tiny) └───8.30 MB (42.38%) -- used ├──7.83 MB (39.93%) ── gc-things ├──0.36 MB (01.83%) ── chunk-admin └──0.12 MB (00.61%) ── arena-admin 0 (100.0%) -- low-memory-events ├──0 (100.0%) ── physical └──0 (100.0%) ── virtual 6.39 MB (100.0%) -- window-objects ├──2.83 MB (44.20%) -- dom │ ├──2.28 MB (35.68%) ── orphan-nodes │ ├──0.29 MB (04.50%) ── element-nodes │ ├──0.22 MB (03.43%) ── other │ └──0.04 MB (00.60%) ++ (3 tiny) ├──2.60 MB (40.69%) -- layout │ ├──1.21 MB (18.99%) ── style-sets │ ├──0.66 MB (10.40%) ── pres-shell │ ├──0.35 MB (05.41%) ── frames │ ├──0.15 MB (02.33%) ── style-contexts │ ├──0.13 MB (01.98%) ── rule-nodes │ ├──0.07 MB (01.05%) ── pres-contexts │ └──0.03 MB (00.52%) ++ (2 tiny) ├──0.96 MB (15.03%) ── style-sheets └──0.01 MB (00.08%) ── property-tables 1,004.32 MB ── explicit 0.00 MB ── gfx-d2d-surfacecache 0.00 MB ── gfx-d2d-surfacevram 0.00 MB ── gfx-d2d-vram-drawtarget 0.00 MB ── gfx-d2d-vram-sourcesurface 0.50 MB ── gfx-surface-win32 0 ── ghost-windows 979.07 MB ── heap-allocated 988.66 MB ── heap-committed 9.58 MB ── heap-committed-unused 0.97% ── heap-committed-unused-ratio 2.47 MB ── heap-dirty 29.92 MB ── heap-unused 0.00 MB ── images-content-used-uncompressed 23.00 MB ── js-gc-heap 0 ── low-commit-space-events 1,069.92 MB ── private 1,053.32 MB ── resident 10.47 MB ── storage-sqlite 1,395.79 MB ── vsize This is merely a suggestion, and may be observational bias rather than true causation, but I happen to play games on the same desktop, and whilst I haven't properly tested this, launching a game (and thus invoking DirectX to capture the display and perhaps mess with Thunderbird's visual style on Windows) seems to provoke the memory consumption. I have a laptop with a similar setup, so I'll keep Thunderbird open on that too and see how it goes.
(In reply to :aceman from comment #8) > It is a pity we have som much memory in heap-unclassified. Even running and > toying around in a test install of TB19 (almost no messages) produces 60MB. That's because mork databases aren't exactly small, and we have no reporters that indicate how much memory they're using.
Same issue on a HP Elitebook 2530p with the same OS type.
can you perhaps narrow the issue to a single account?
Not this week, but perhaps next week. Will require at least a few long-hour periods of waiting before purging/restoring data to provide different sets of circumstance. Is there really no way of tracing how memory is being allocated?
(In reply to Ricky Burgin from comment #14) > Not this week, but perhaps next week. Will require at least a few long-hour > periods of waiting before purging/restoring data to provide different sets > of circumstance. If it's not too confusing, you can actually run multiple tbirds side by side on the same machine, by using multiple profiles https://support.mozillamessaging.com/en-US/kb/using-multiple-profiles you would start thusly thunderbird -no-remote -P so you couldl have your production, default profile running. and another thunderbird running test profille that has just one account :) > Is there really no way of tracing how memory is being allocated? the usual suspicion is folder databases being open, which might be revealed by doing msgdb:5 log per https://wiki.mozilla.org/MailNews:Logging ... it's worth a shot, but the issues I've seen lately don't show up in the log irving might have some idea.
If you're still seeing the problem in 16.0, the Gecko Profiler extension might help. It's available from the Mozilla extensions site; go to Tools / Add-ons, Get Add-ons, and search for "Gecko Profiler". Once you have it installed, you'll see "Disabled" and "Dump Profile" show up at the bottom right of your main mail window. Restart, click on "Disabled" (it will change to "Enabled"), and then do the things that cause memory use to increase. Before you get to the point of crashing, click "Dump Profile" and then "Upload full profile"; you'll get a URL you can attach to this bug. We've had some issues with some vendor video drivers leaking memory in Firefox, but only with acceleration turned on. TB ships with video acceleration off by default, so unless you've modified those preferences it's probably not exactly the same issue. However, if the memory leak really is correlated with running other DirectX applications at the same time, that's a very interesting bit of information. Wayne, you normally run Windows, right? Any chance you could try to reproduce that?
Can confirm the problem still exists in 17 beta.
I have a colleague who also has this issue. Win7_64, TB 16.01. Has issues with keyboard freezing due to Tbird growing in memory size and only resolution is to shutdown and restart Tbird. Memory size goes back down to startup levels (about 100K) and keyboard function returns to normal (w/o freezes) Occurs 2x a day. Not sure if issues are related but they do appear so.
If you go back to version 15, can you still reproduce the problem? http://ftp.mozilla.org/pub/mozilla.org/thunderbird/releases/15.0.1/win32/en-US/Thunderbird%20Setup%2015.0.1.exe
Depends on: 480843
Flags: needinfo?(techsmart.chicago)
Flags: needinfo?(ricky)
(In reply to Ricky Burgin from comment #9) > This is merely a suggestion, and may be observational bias rather than true > causation, but I happen to play games on the same desktop, and whilst I > haven't properly tested this, launching a game (and thus invoking DirectX to > capture the display and perhaps mess with Thunderbird's visual style on > Windows) seems to provoke the memory consumption. I'm not sure I follow. Have you been able to reconfirm this, or confirm it on a different machine? > I have a laptop with a similar setup, so I'll keep Thunderbird open on that > too and see how it goes. results? And, does the machine get slower as memory usage increases?
Flags: needinfo?(ricky)
Sorry! Doesn't seem related. The machine doesn't get noticeably slower as memory usage increases, but is liable to forcing applications to crash if they cannot reliably allocate memory due to the leak. The issue is still there with 18, but I see a 19 beta has recently released, so I'll try that version to see if anything has changed. Sorry for not updating this bug, I have been extremely busy as of late and the free time to test this bug was swept from under me. I'm hoping to have some time soon where I can install a fresh copy of Thunderbird and add all the accounts from scratch so the userdata gets created from scratch to rule out any potential weirdness from my current specific settings and mailboxes. If it still doesn't work, I'll try in a virtual machine with a fresh OS too.
Flags: needinfo?(ricky)
Thanks Ricky. In case you are not aware, you can create additional profiles to test on the same machine, without disturbing your production profille, as described at https://support.mozillamessaging.com/en-US/kb/using-multiple-profiles
Thanks, that's really helpful. Do you know of any huge inbox sizes that people are running without issue? On a single mailbox, I have 36154 emails being pulled over IMAP right now - is that within tested limits?
36154 emails would be fine for POP3 (and I have a test folder with 600 000 where I didn't notice any problems yet). We just have a limit of 4GB for the folder size. But I can't say what the performance is for IMAP4 with that many messages.
(In reply to Ricky Burgin from comment #23) > Thanks, that's really helpful. Do you know of any huge inbox sizes that > people are running without issue? On a single mailbox, I have 36154 emails > being pulled over IMAP right now - is that within tested limits? should be fine unless Inbox gets unexpected redownloads
Seems to be stable at ~260MB of memory after creating a new profile with one of my mail accounts. Will add my additional accounts one at a time and monitor further.
After adding a second IMAP account, it is sat at around 520MB usage.
Flags: needinfo?(techsmart.chicago)
(In reply to Ricky Burgin from comment #27) > After adding a second IMAP account, it is sat at around 520MB usage. which account was that? what results with adding the other accounts?
Flags: needinfo?(ricky)
Whiteboard: [closeme 2013-03-10]
Resolved per whiteboard
Status: UNCONFIRMED → RESOLVED
Closed: 12 years ago
Flags: needinfo?(ricky)
Resolution: --- → INCOMPLETE
Whiteboard: [closeme 2013-03-10]
Whiteboard: [gone after new profile, cause unknown]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: