Open Bug 887610 Opened 11 years ago Updated 8 months ago

Thunderbird often slow releasing memory during shutdown, eg after loading large newsgroups

Categories

(Thunderbird :: General, defect)

22 Branch
x86_64
Windows Vista
defect

Tracking

(Not tracked)

People

(Reporter: robertmilesxyz, Unassigned)

References

(Depends on 1 open bug)

Details

(Keywords: perf)

Attachments

(8 files)

User Agent: Mozilla/5.0 (Windows NT 6.0; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0 (Beta/Release)
Build ID: 20130511120803

Steps to reproduce:

I tried to shut down Thunderbird 22.0 beta a few hours ago, by clicking in the X at the top right.  It was already using over 2.1 GB of memory.


Actual results:

The user window closed immediately, but left the program running in the background with the amount of memory it used jumping around between 1.8 GB and 2.0 GB, and a significant number of hard faults in the memory use.

Hours later, the program still hasn't shut down; the memory use is now jumping around between 1.7 GB and 1.9 GB, with fewer hard faults in the memory use, but no other noticeable change.


Expected results:

A much faster shutdown and memory release.
I tried shutting down BOINC to release the memory it was using.  Not much change; the fluctuating Thunderbird memory use is now down to the 1.6GB to 1.8 GB range; no other change noticed.

The Resource Monitor (see the link under the Performance page of Windows Task Manager) says the hard memory faults are continuing, still mostly from thunderbird.exe but also a smaller number from the much less memory-hungry antivirus program.

Windows Task Manager says that about 4.35 GB of memory is in use, but only about 2.5 GB of that is in use by processes.  Does this indicate a major memory leak within Thunderbird?

Is Thunderbird doing anything useful during this shutdown process that should not be interrupted by killing the process?
the more important question is how did you get to that high level of memory usage in the first place.

> Windows Task Manager says that about 4.35 GB of memory is in use, but only about 2.5 GB of that is in use by processes.  Does this indicate a major memory leak within Thunderbird?

screen shots would be wonderful.
also, how much memory is installed on your system?  (for example if you have only 2gb installed and 4.35gb in use, you can expect decades long response time)

> Is Thunderbird doing anything useful during this shutdown process that should not be interrupted by killing the process?

feel free to kill the process
BOINC, which I have running in the background most of the time, is allowed to use up to 3.2 GB of memory, and often comes close to that limit.

Thunderbird reached about 2.1 GB of memory used after I downloaded all the headers for newsgroup alt.support.diabetes from the news.teranews.com server (over 580000 claimed to be available, but only about 450000 would download).

Screen shots of what?  This was after the Thunderbird window had closed, and it was running only in the background.

8 GB of memory is installed, but if 32-bit processes try to use more than about 3.5 GB, response slows down enough that I cannot start any more programs to examine the situation.  (This does not count the memory used by the SYSWOW64 modules required to run 32-bit programs under 64-bit Windows Vista.)
Only 64-bit processes can push the memory used much beyond 3.5 GB without making it impossible for me to start any programs for checking just how much is used.  For example, some hours earlier, a 64-bit program had pushed the total memory used beyond 6 GB.  I've never seen the total memory used exceed the 8 GB of memory installed, though.
To bug opener.

First of all, clealy isolate "Virtual Memory" and "Real Memory" always, please, because you use "OS which supports Virtual Memory System", and Thunderbirs requests and uses "Virtual Memory", and responsible person to manage "Real Memory" which is allocated or associated to accessed "Virtual Memory" is always Operating system.
And, second, if report on MS Win, read and understand documents at least relevant to "MS Win's page management is Page trimming instead of Page stealing", please.

What happens by "Page trimming" can be observed by following.
(1) config.trim_on_minimize=true, and restart Fx or Tb on MS Win.
(2) Watch Virtual Memory Size used by Fx/Tb, and watch size which looks "allocated Real Memory Size to Fx/Tb", by MS Win's Task Manager or performance related Tool.
(2-1) Use Fx/Tb for a while
(2-2) Minimize Fx/Tb
      => Virtual Memory used by Tb is unchanged
      => Size which looks "allocated Real Memory Size to Fx/Tb"
         decreases. This is by "Page trimming".
(2-3) After a while, activate Fx/Tb window again,
      and do some operations until response of Fx/Tb goes back normal.
(2-4) Do (2-2) and (2-3) several times.
      => Avarage size which looks "allocated Real Memory Size to Fx/Tb"
         is "Working set size which is needed to run normally".
(3) Usually, "Virtual Memory Size"  >
             "Working set size which is needed to run normally"
    "Virtual Memory Size" - "Working set size which is needed to run"
    == "Swapped out memory size" (data is held in HDD file for swap)
(4) Terminate Fx/Tb.
    Fx/Tb does termination work such as, object destory, free resources
    by garbage collector, and freemain.
    At this step, phenomenon like following is observed.
      Even though Fx/Tb simply does do "free resource work" only,
      all allocated "Virtual Memory" is swapped in again,
      (Real Memory is allocated again), 
      then freemain of "Virtual Memory" is executed by OS.
(5) Because "Swap in of all allocated virtual memory" occurs,
    step (4) takes far longer than general user thinks.

Big "Virtual Memory size" is a result of "total size of not-freeed resources is big" + "acual memory leak".
"total size of not-freeed resources is big" = "not-freeed resources which is actually needed is big" + "vacant spaces in allocated virtual memory is not so small". 
"total size of not-freeed resources is big" is normal phenomenon because data like "data held by back and forward cache" is usually large and "data of many many Web pages or many many mails" are usually held by bfcache etc.. These are also applicable to data such as objects associated to Widow, folder/message pane, historical data, held in JavaScript Object or C++ Object.

You are merely looking similar phenomenon to above step (4) with config.trim_on_minimize=true due to OS design, in 64bits Fx/Tb of config.trim_on_minimize=false on 64bits MS Win, aren't you?

Big "acual memory leak by add-on" is not so rare problem.
Do you see your problem with newly created Tb's profile and/or -safe-mode of Tb(thunderbird.exe -safe-mode)?
FYI. document relevan to to "Page trimming" is pointed in Bug 381950 comment #0. That is document is for "Page trimming of Win NT", but the basic design of "Page trimming of Win NT" is surely inherited by newer MS Win.
FYI. See document pointed in bug 381950 comment #6, for phenomenon of "MS Win-XP is lazy in expensive page trimming work".
As far as I can tell, the programs I use for observing memory use show only the Real Memory use.  However, one of them mentions that 16 GB of Virtual Memory is available, without mentioning how much of it is in use.

The other memory-hungry program, BOINC, appears to be using Real Memory only, with no capability of using Virtual Memory.

Firefox (22.0 release) is installed, but not showing any similar problem.
As far as I can tell, the programs I use for observing memory use show only the Real Memory use.  However, one of them mentions that 16 GB of Virtual Memory is available, without mentioning how much of it is in use.

The other memory-hungry program, BOINC, appears to be using Real Memory only, with no capability of using Virtual Memory.

Firefox (22.0 release) is installed, but not showing any similar problem.
1. BOINC complicates diagnosing the thunderbird issue, so you should really take it out of the equation by not running it when reproducing the thunderbird issue.

2. Please use taskmgr program's process tab for showing memory. That the screen shot I was looking for

3. the fundamental issue might not be shutdown, but how you get high memory in the first place.  Is using teranews the only way you can get to this high memory state?
What are the time differentials for this snapshots?
Flags: needinfo?(robertmiles)
(In reply to Wayne Mery (:wsmwk) from comment #17)
> What are the time differentials for this snapshots?

Time differentials not measured.

I produced a test file showing what I remembered about them, but it looks like I deleted it without sending it.

I'll abbreviate the tb23_Capture.JPG names to just the one letter that's different from one to another.

All pictures taken with BOINC not running, and therefore not occupying memory.

Firefox was running in the background, but apparently doing nothing. 

Pictures a, b, and c within a few minutes of each other, early during the download of the headers (over 451000 downloaded).  Picture d about 2 hours later, after the header download was finished.  Pictures e and f within a few minutes of each other, about 1.5 hours after picture d; after I tried to download the bodies of about 5000 posts and actually got about a third of them before the Can't log in messages started for that server because I had exhausted the download limit for the day.

I killed the Tb23.0 beta task, then restarted the program.

Messages on Error Console:

Could not read chrome manifest 'file:///C:/Program%20Files%20(x86)/Mozilla%20Thunderbirdbeta/chrome.manifest'.

While creating services from category 'profile-after-change', could not create service for entry 'Disk Space Watcher Service', contract ID '@mozilla.org/toolkit/disk-space-watcher;1'


Warnings on Error Console:

Timestamp: 7/11/2013 2:12:47 PM
Warning: Unknown property 'box-sizing'.  Declaration dropped.
Source File: https://www.mozilla.org/en-US/thunderbird/beta/start/?uri=/thunderbird/start/&locale=en-US&version=23.0&os=WINNT&buildid=20130702112923
Line: 14, Column: 22
Source Code:
            box-sizing: border-box;

Timestamp: 7/11/2013 2:12:47 PM
Warning: Unknown property '-moz-border-radius'.  Declaration dropped.
Source File: https://www.mozilla.org/en-US/thunderbird/beta/start/?uri=/thunderbird/start/&locale=en-US&version=23.0&os=WINNT&buildid=20130702112923
Line: 135, Column: 25
Source Code:
       -moz-border-radius: 5px;

Timestamp: 7/11/2013 2:12:47 PM
Warning: Unknown property '-moz-border-radius'.  Declaration dropped.
Source File: https://www.mozilla.org/en-US/thunderbird/beta/start/?uri=/thunderbird/start/&locale=en-US&version=23.0&os=WINNT&buildid=20130702112923
Line: 168, Column: 25
Source Code:
       -moz-border-radius: 5px 5px 0 0;


No errors seen on Error Console.


Some of the downloaded posts from February, 2008 now appear in the local folder, but not all of them.


Something that might or might not be relevant:

On my previous downloads from this newsgroup on this server, about 90% of the posts were duplicates of posts I had previously downloaded from the newsgroup on other news servers.  I don't want the duplicates, so I deleted them soon after the duplicate remover add-on found them.  About another 5% had mismatches between the header information shown on the messages list and the header information shown by opening the message.  For about 1%, the body of the messages was not downloaded at all.  The remining 4% or so was actually messages missing from my saved messages folders.
Flags: needinfo?(robertmiles)
The high memory usage appears to be partly due to downloading so many headers at once (about 1.6 GB) and partly due to some use of memory can only show as part of the total physical memory in use, but not as attached to any program in particular (about 2 GB more).
...some use of memory that Windows Task Manager can only show...
I can't reproduce your issue - tested using my ISP's news server.  (never before used - yay ptd!)

650,000 article headers downloaded cleanly in about 13 minutes during which time TB was completely unusable. Memory grew quickly but predictably by 800MB, ending at 1,065MB. Then as it was building the thread pane UI it by dropped 400MB to end at 672MB. alt.support.diabetes.msf is 275mb

about:memory
855.63 MB (100.0%) -- explicit
├──885.81 MB (103.53%) -- maildb [?!]
│  ├──422.63 MB (49.39%) ── database(news://usenet.ptd.net:119/alt.support.diabetes)
│  ├──232.17 MB (27.13%) ── database(imap://vseerror@mail.lehigh.edu/Moz)
│  ├──151.40 MB (17.69%) ── database(imap://wsm0@mail.lehigh.edu/INBOX)
│  ├───27.30 MB (03.19%) ── database(imap://wayne.mery@imap.gmail.com/INBOX)
│  ├───25.28 MB (02.95%) ── database(imap://vseerror@mail.lehigh.edu/INBOX)
│  ├───19.75 MB (02.31%) ── database(imap://vseerror@mail.lehigh.edu/moz-lxmac/moz-FF)
│  └────7.29 MB (00.85%) ++ (9 tiny)
├──203.98 MB (23.84%) -- workers
│  ├──198.72 MB (23.22%) -- workers(mozilla.com)/worker(js/parserWorker.js, 0x8adec00)
│  │  ├──166.36 MB (19.44%) -- zone(0x8c8e000)
│  │  │  ├──103.55 MB (12.10%) -- compartment(web-worker)
│  │  │  │  ├───78.42 MB (09.17%) -- objects-extra
│  │  │  │  │   ├──77.52 MB (09.06%) ── elements/non-asm.js
│  │  │  │  │   └───0.90 MB (00.11%) ── slots


In short, I'm not seeing the memory usage at the extreme you are seeing, even with more articles.  And my shutdown time was short/normal.  A guess, perhaps cancelling or interrupted newsgroup download is corrupting such that your alt.support.diabetes.msf gets hosed.

Restarted Thunderbird - loading the newsgroup takes ~40 seconds and 400MB memory.
Wayne Mery,

You appear to have a much faster internet connection than I do - every time I ask my ISP if they can upgrade the current 3 MB speed, they say "not at present."  Could this suggest a timing problem that does not occur with a faster connection?

Every time I need to download the headers again, I must delete the alt.support.diabetes.msf file first, or barely any of them will download.  Any download interruption of those headers is coming from the server, not from anything I do.

Do you happen to be using 64-bit Windows Vista?  I suspect that some of the memory problems I see are due the the way 64-bit Windows Vista (but not 64-bit Windows 7) handle memory allocations for 32-bit programs.

I got roughly the same result from about:memory, but from Firefox, not from Thunderbird.  How do I ask Thunderbird for a similar report, since that's where I see the problem?

Something else that might or might not be relevant:

I've now noticed that for many of the last 5% of the posts downloaded while still running with BOINC also running, the step that was supposed to download the whole message (as seen with View Source) actually got two whole messages instead.  I have not reached a point where I can check if this also happened with the messages downloaded without BOINC running.
I just noticed that the Performance page of Windows Task Manager reported 4.1 GB of memory in use, even with both BOINC and Thunderbird shut down.  The Processes page showed some other programs still running, but nowhere near enough for a 4.1 GB total.

I then rebooted the computer, and both the total memory in use and the memory used by Thunderbird became much less.

To me, this indicates that most of my problem with memory use is something that the Processes page cannot show.


The best I can tell, the programs I use for reporting memory use recognizes use of the real memory only, and do not even mention any use of the virtual memory.

Do you know of some other memory use reporting program that does recognize the difference between real memory and virtual memory?


Wayne Mery, does your ISP's news server have anywhere near as many headers in newsgroup alt.support.diabetes as the news.teranews.com server does?  If not, the problem might be due to the sheer number of headers involved.

Something else that might be affecting the memory use for me - the number of folders under Local Folders.  I probably have a few thousand, but definitely too many to count.  Arranged in multiple levels - most are subfolders of other folders.


When I got two whole messages for each header, usually neither of them matched the header information shown in the messages list.  They did not match each other, either.
Today, I tried running Thunderbird beta for a few hours, with add-ons disabled  and without BOINC running.  The main action was downloading the headers again.  I then shut down Thunderbird beta and compared the total memory usage to the total memory usage before starting it.  The two values were 2.80 GB and 2.79 GB, close enough to indicate that downloading headers under Thunderbird beta is not the source of the increasing total memory in use but not attached to any program.  This left me with so little download quota on that server for today that a similar test on downloading bodies will have to wait until tomorrow.

During this test, the memory usage by Thunderbird beta as shown by the Processes page went up to 1.46 GB.

I then tried running BOINC for a few hours, without Thunderbird beta also running, then shut it down.  For this test, the total memory in use went up to 3.11 GB, indicating that the part of the increased memory use not shown by the Processes page is not from Thunderbird beta, and this thread is not the best place to discuss that part of my problems.

The frequent loss of those headers, and the high memory use while downloading them again, still appears to be due to Thunderbird beta, though.

I've thought that some of the slowdown might be due to my ISP throttling access to port 119, but I don't know a way to test that idea.


WAKA,

Where do I find config.trim_on_minimize and the chance to change it?  So far, I haven't found it.

Where do I find information on how much virtual memory is in use, and how much of that is used by Thunderbird?  So far, I've found only information on what appears to be the amount of real memory in use.
(In reply to Robert Miles from comment #24)
> Where do I find config.trim_on_minimize and the chance to change it?
> So far, I haven't found it.

It sounds that Google is not available on your planet...

Following is web page which is listed at top of Google search result for "config.trim_on_minimize".
> http://kb.mozillazine.org/Config.trim_on_minimize
Link to "how to change setting" is placed in the web page.
config.trim_on_minimize is not shown by about:config of Firefox. It may mean config.trim_on_minimize has been removed in recent Firefox and Thunderbird.

> Where do I find information on how much virtual memory is in use,
> and how much of that is used by Thunderbird?
> So far, I've found only information on what appears to be the amount of real memory in use.

Following is web page which is listed at second/third of Google search result for "ms windows task manager virtual memory size".
> http://windows.microsoft.com/en-my/windows-vista/what-do-the-task-manager-memory-columns-mean
> http://technet.microsoft.com/en-us/library/cc938567.aspx
Column name of Task Manager looks chaged by MS.
  Memory Usage => Private Working Set, Working Set
  Virtual Memory Size => Commit Size
"Commit size" may be "swapped out" part only. Please read documents provided by MS well.

Please see also at least following web pages etc., which are listed at top part of search result of Google search for "ms windows task manager virtual memory size", "ms windows task manager Commit Size" etc., before open bug at B.M.O simply because you saw large "Real Memory" size value for Tb/Fx on MS Windows.
> http://technet.microsoft.com/en-us/magazine/ff382715.aspx
> http://windows.microsoft.com/en-my/windows-vista/see-details-about-your-computers-performance-using-task-manager
> http://stackoverflow.com/questions/1170654/how-to-interpret-windows-task-manager
> http://support.lenovo.com/de_DE/downloads/detail.page?DocID=HT069722
I am on vista 32 bit, oldish dell laptop with 5400rpm disk on wireless and fairly fast cable modem.

You can get good about:memory report using version 24 earlybird 
http://www.mozilla.org/en-US/thunderbird/channel/

> Wayne Mery, does your ISP's news server have anywhere near as many headers in newsgroup alt.support.diabetes as the news.teranews.com server does?

yes. I cited the number. over 600k. and thunderbird's status line confirms >600k unread/unexpired articles in the newsgroup folder.

** I don't know whether your slower connection is a factor. But having to redownload the newsgroup every time is certainly a prime problem that I doubt we'll have a solution for any time soon.  I believe newsgroup is much too big for Thunderbird to handle the full article list unless you've got a state of the art computer with at least 7200 rpm disk on a fast network connection. (it took >90 seconds to just resort the folder on my laptop, with the newsgroup already loaded) 

So my recommendation is to NOT attempt to download the entire newsgroup - only download the most recent several K articles. Further, I would set a retention value and not attempt to sync message bodies for offline use.  If you want see information older than a year, like 1993 (which is the oldest one in my list) then use https://groups.google.com/forum/#!forum/alt.support.diabetes


> The Processes page showed some other programs still running, but nowhere near enough for a 4.1 GB total.

on the process tab make sure you have "Show processes from all users" checked ON.


Last item - I have no interest in data or issues related to testing with trim or BOINC until we have sorted out any Thunderbird's performance.
I'm already using that version of Earlybird on another computer, one using an SSD drive (even faster than a 7200 rpm drive) and near state of the art when I bought it.  Not enough free space on that SSD drive to run similar tests there.  Faster internet connections currently not available for this address, but I often check if that has changed.

That newsgroup has a very active kook, whose messages I rarely want to save or look at again.  Therefore, I want to save my decisions on which messages are worth saving, which is rather difficult on Google Groups.

I've already bought another SSD drive for the computer where I run Thunderbird beta, but am having trouble getting a suitable tray for installing it.

Problem still seen with "Show processes from all users" checked.
At a previous loss of the headers for the newsgroup, I tried
telling Thunderbird to download only 350000 headers instead of
all 585000.

Results so far:  The time until the next loss of headers took
a few days longer.  Perhaps 230000 headers were actually
downloaded, at the most recent end of the range of those
available.

After the latest loss of headers, I told it to download only
250000, trying to narrow it down toward only those for which
I have not downloaded the bodies from this server yet.
Actually got 173726 this time.

These lower numbers of headers downloaded stopped the problem
of multiple messages downloading for just one header, but not
the problem of the full messages being filed under the wrong
header.
Taking this off UNCO so we're not continuously implying we want more info. Because 1. at least half this battle here is bug 185634 - [TB] consumes unbearable amounts of memory in big newsgroups.
2. yeah, when you gobble lots of memory it takes a long time to tear it down.  I'm not sure what we can do about this, and anything that might be done is likely a Core issue, not Thunderbird
Severity: normal → minor
Status: UNCONFIRMED → NEW
Component: Untriaged → General
Depends on: 185634
Ever confirmed: true
Keywords: perf
Summary: Thunderbird beta often very slow at releasing memory when it is shut down → Thunderbird often slow releasing memory during shutdown, eg after loading large newsgroups
May be caused be the new bug I entered today, on the searches for possibly expired messages interfering with viewing messages that were successfully downloaded and also interfering with a normal shutdown of Thunderbird.

As for nightly builds, I looked there and saw no sign of any way to tell which of the several builds listed I should try.
The computer on which I run the beta version uses Windows Vista (64 bits), and part of the problem appears to be due to running 32-bit programs under that version of Windows.  It uses SYSWOW64 to run 32-bit programs, and the memory required to hold the SYSWOW64 modules in use is close to the same as that needed to hold the 32-bit programs that are running.  Therefore, no more than about half of the real memory can be used to hold 32-bit programs that are currently running, and that model of computer uses a 32-bit program to process the keyboard input before passing it along to other programs.

No similar memory restriction problem seen under Windows 7 (64 bits); partly because the SYSWOW64 modules for it are smaller, and usually fewer are in use when running the same 32-bit programs.  The slow release of memory by Earlybird or Thunderbird is still seen under Windows 7 (64 bits), though.
Severity: minor → S4

Robert do you still see this issue?

Flags: needinfo?(robertmilesxyz)

Alfred, do you have ability to test this with newest versions? (preferably newer than 102)

Flags: needinfo?(robertmilesxyz) → needinfo?(infofrommozilla)

I have subscribed to a group on my local server with around 569k articles.

Point A: The download of the headers starts (XOVER / 0 sec).
Point B: The server has transferred the XOVER (30 sec)
Point C: ? probably TB sorts the articles from here on (38 sec)
Point D: The article list is showing up. (90 sec)
I read some articles, but nothing changes in the memory consumption.
Point E: I change to a group with about 100 articles. (160 sec)
The memory consumption starts to increase again.
The old group is still displayed.
Point F: The new group is shown (200 sec).

Flags: needinfo?(infofrommozilla)

TB 118.0a1 (2023-08-05) (64-bit) - Windows 10

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: