Last Comment Bug 540214 - imap STATUS not closing db - mail.check_all_imap_folders_for_new and "check this folder for new messages" causes high memory and error message Unable to open the summary file for xxxxx. Perhaps there was an error on disk, or the full path is too long.
: imap STATUS not closing db - mail.check_all_imap_folders_for_new and "check t...
Status: RESOLVED FIXED
: fixed-seamonkey2.0.4, perf
Product: MailNews Core
Classification: Components
Component: Networking: IMAP (show other bugs)
: Trunk
: x86 Windows XP
: -- major (vote)
: Thunderbird 3.1b1
Assigned To: David :Bienvenu
:
Mentors:
: 516395 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2010-01-16 15:14 PST by Rick Coloccia
Modified: 2010-03-18 06:41 PDT (History)
7 users (show)
mozilla: in‑testsuite+
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
.2-fixed


Attachments
Contents of TB profile directory (115.94 KB, text/plain)
2010-01-17 18:54 PST, Rick Coloccia
no flags Details
Process Monitor Results (25.40 KB, text/plain)
2010-01-22 05:43 PST, Rick Coloccia
no flags Details
process monitor output (165.12 KB, text/plain)
2010-01-24 19:10 PST, Rick Coloccia
no flags Details
pre directory listing (3.19 KB, text/plain)
2010-01-24 19:11 PST, Rick Coloccia
no flags Details
post directory listing (3.66 KB, text/plain)
2010-01-24 19:13 PST, Rick Coloccia
no flags Details
MSGDB Level 5 Log (1.79 MB, application/zip)
2010-01-28 06:26 PST, Rick Coloccia
no flags Details
MSGDB Level 5 Log File (861.68 KB, application/zip)
2010-01-28 07:19 PST, Rick Coloccia
no flags Details
MSGDB Level 5 Log File (912.57 KB, application/x-zip-compressed)
2010-01-28 07:58 PST, Rick Coloccia
no flags Details
proposed fix (1.22 KB, patch)
2010-01-28 14:09 PST, David :Bienvenu
no flags Details | Diff | Review
proposed fix with unit test (3.52 KB, patch)
2010-01-29 12:06 PST, David :Bienvenu
no flags Details | Diff | Review
better error handling (3.61 KB, patch)
2010-02-09 08:32 PST, David :Bienvenu
standard8: review+
standard8: superreview+
standard8: approval‑thunderbird3.0.2+
Details | Diff | Review

Description Rick Coloccia 2010-01-16 15:14:32 PST
User-Agent:       Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 1.1.4322)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0

I experience exactly what you see in bug 381771, which is "resolved incomplete."  I would be happy to "assume ownership" of the bug so it can be reopened and so we can work towards resolution.  I never had this problem with TB2.

I've done everything in this document to resolve the issue:
https://spaces.mtu.edu/clearspace/docs/DOC-1515.pdf;jsessionid=BD56AD952B00E0F658642B2A27B644AA

I've moved my profile to the root of the c-drive, I've compacted folders, run in safe mode, deleted and recreated accounts and directories inside the ImapMail folder, all with no success.

These errors make TB3 pretty much unusable for me. 

Reproducible: Sometimes

Steps to Reproduce:
Sorry, I don't have anything reliable to reproduce it.
It happens in safe mode as well as normal mode.
Comment 1 WADA 2010-01-16 21:28:08 PST
(In reply to comment #0)
> I've moved my profile to the root of the c-drive,

What works did you do for "moved profile to the root of the c-drive"?

Check profiles.ini content(pointer to profile directory).
   At command prompt, NOTEPAD.EXE "%appdata%\Thunderbird\profiles.ini"
A profiles points C:\ ?
If yes, never do it. If you delete the profile which points C:\ with requesting "delete direcory/file", DEL C:\*.* /S /Q is executed according to your request.
Comment 2 Rick Coloccia 2010-01-17 06:14:45 PST
contents of profiles.ini:
[General]
StartWithLastProfile=1

[Profile0]
Name=default
IsRelative=0
Path=c:\Thunderbird\Profiles\2h9l0qyd.default
Comment 3 WADA 2010-01-17 16:49:26 PST
(In reply to comment #2)
> [Profile0]
> Name=default
> IsRelative=0
> Path=c:\Thunderbird\Profiles\2h9l0qyd.default

Check directry strunture by DIR command(at command prompt).
> DIR "c:\Thunderbird\Profiles\2h9l0qyd.default" /S > c:\Thunderbird\DIR.TXT
> NOTEPAD.EXE c:\Thunderbird\DIR.TXT
folder files for mail folder (<folder_name>, <folder_name>.msf) is held in next directory.
> c:\Thunderbird\Profiles\2h9l0qyd.default\Mail\<server_name>\<root_folder>.sbd\...\<parent_folder>.sbd\<folder_name>.msf
> c:\Thunderbird\Profiles\2h9l0qyd.default\ImapMail\<server_name>\<root_folder>.sbd\...\<parent_folder>.sbd\<folder_name>.msf
This path can be seen in "Location:" of Folder Properties. 

For which folder(folder file name) was the error message issued?
Comment 4 Rick Coloccia 2010-01-17 18:54:43 PST
Created attachment 422145 [details]
Contents of TB profile directory

This is the output of the request to check directory structure:

Check directry structure by DIR command (at command prompt).
DIR "c:\Thunderbird\Profiles\2h9l0qyd.default" /S > c:\Thunderbird\DIR.TXT
Comment 5 Rick Coloccia 2010-01-17 18:57:24 PST
I've gotten the error message on a many of the of directories, including:
Inbox
In-ARIN
In-BIND
In-Cisco
In-DHCP
In-EZProxy

It's happening on all folders that I click on right now.
Comment 6 WADA 2010-01-17 19:37:31 PST
(In reply to comment #5)
> I've gotten the error message on a many of the of directories, including:
> Inbox
> In-ARIN
> In-BIND
> In-Cisco
> In-DHCP
> In-EZProxy

> Directory of c:\Thunderbird\Profiles\2h9l0qyd.default\ImapMail\imaps.geneseo.edu
>(snip)
> 01/16/2010  10:52 AM                 0 In-ActiveDir
> 01/16/2010  05:08 PM         9,700,122 In-ACUTA
> 01/16/2010  10:08 AM                 0 In-AOL-SPAM
> 01/16/2010  10:52 AM                 0 In-BB-L
> 01/16/2010  10:52 AM                 0 In-BIND
> 01/17/2010  03:31 PM        31,085,949 In-Cisco
> 01/17/2010  09:04 PM         5,922,326 In-Cron
> 01/17/2010  09:04 PM    <DIR>          In-Cron.sbd
> 01/16/2010  10:52 AM                 0 In-DHCPD
> 01/16/2010  04:35 PM             4,521 In-Eudora
> 01/17/2010  12:14 PM         3,900,029 In-EZProxy
> 01/17/2010  02:17 PM        15,603,035 In-Packeteer
> 01/16/2010  10:52 AM                 0 In-Perl-Win32
> 01/17/2010  09:04 PM        16,783,470 In-REN-ISAC
> 01/17/2010  09:05 PM        30,300,230 In-ResNet-L
> 01/16/2010  10:52 AM                 0 In-Snort
> 01/16/2010  10:52 AM                 0 In-SPAM
> 01/17/2010  09:04 PM         3,553,119 In-Squid
> 01/17/2010  12:12 PM            47,891 In-WinNT-L
> 01/17/2010  09:44 PM       127,608,770 INBOX
> 01/17/2010  09:03 PM            46,785 INBOX.msf
> 01/17/2010  09:45 PM    <DIR>          INBOX.sbd
>(snip)

> In-ARIN    : no In-ARIN.msf, no In-ARIN   
> In-BIND    : no In-BIND.msf 
> In-Cisco   : no In-Cisco.msf
> In-DHCP    : no In-DHCP.msf, no In-DHCP 
> In-EZProxy : no In-EZProxy.msf

Problem on account of "imaps.geneseo.edu"?
What did you do on these folders or folders of no associated ".msf" file?
What did you do on files of no associated ".msf" file?
Are you working in "Work Offline" mode?

1. Disable "Work Offline"(go "Work Online" mode), if "Work Offline" mode".
2. At Server Settings/Advanced, uncheck "Show only subscribed folders".
   If you set "IMAP server directry:", delete specified string.
3. Restart Tb, account's property, Subscribe, check subsciption list
4. Clink Ibox, and view some mails, terminate Tb, restart Tb
Will garbage files be removed?
Will subscribed/unsbscribed folders be displayed?

If problem remains, try next.
5. terminate Tb, rename panacea.dat to pnacea.dat.backup in profile directory.
6. restart Tb
7. Show subscription list, view Inbox content
Comment 7 Rick Coloccia 2010-01-18 10:20:30 PST
Yes, the problem occurs most often on imaps.geneseo.edu, but has happened on all the other accounts in TB.  I use the imaps.geneseo.edu account most, so I 

I took to direct action on any of the files inside the profile folder at all.  I haven't deleted, renamed, etc. any of the folders inside the profile.

I am not working in offline mode.  Never have.  

For all accounts, I have just removed the checkbox next to "show only subscribed folders" and I have removed my "INBOX" prefix from all accounts in "IMAP Server Directory."  As a result, all my folders now appear as subfolders inside my inbox instead of folders at the same level in the tree heirarchy.

I have restarted tb, then clicked on file/subscribe, and for all accounts and all folders, I have resubscribed to everything.

Viewing mail in all folders seems to be OK right now.  

Sometimes the error takes a few minutes to start happening, so I don't want to suggest the problem is resolved when it is not, but right now it seems OK...

I will certainly submit another commend if/when the error returns.
Comment 8 Rick Coloccia 2010-01-18 10:24:07 PST
And now it's back. I again cannot get into folders. Same error, cannot open the summary file for.....

I clicked away from the problem folder to another, and then came back, and it did not show the error again but instead let me into the folder.
Comment 9 Rick Coloccia 2010-01-18 10:29:40 PST
I just closed and reopened TB and it is again doing it.  Cannot open the summary file for...  This time it was In-Perl-Win32.  I clicked away to another folder, then back to In-Perl-Win32 and it let me in and I could see the message in the folder.
Comment 10 WADA 2010-01-18 20:48:47 PST
(In reply to comment #7)
> and I have removed my "INBOX" prefix from all accounts in "IMAP Server Directory."
> As a result, all my folders now appear as subfolders inside my inbox instead of folders at the same level in the tree heirarchy.

It is also reflected to directory/folder structure for .msf and offline-store file. Files are probably newly kept under ...\Inbox.sbd.

(In reply to comment #9)
> I just closed and reopened TB and it is again doing it.
> Cannot open the summary file for...  This time it was In-Perl-Win32.

Probably due to garbage of pointer to old folder path used while "IMAP Server Directory: Inbox".
Rename panacea.dat to panacea.dat.Bacup and restart Tb. Problem remains?
Comment 11 Rick Coloccia 2010-01-19 09:20:57 PST
With TB closed, I renamed panacea.dat.  I started TB.  Things were good for a few minutes, but some 5 minutes later, I clicked on one of my folders, and again, Cannot open the summary file...
Comment 12 WADA 2010-01-19 09:38:46 PST
(In reply to comment #11)

As I requested you to delete "IMAP server directry:" setting in order to make problem analysis simple, and as and you did it, it may cause mismatch between old Copies&Folders settings, Junk settings, Trash settings for IMAP account, and real IMAP folder path settings with no "IMAP server directry:".
Please check all of Copies&Folders settings, Junk settings, Trash setings for IMAP accounts, and choose existent IMAP folder via "Others:" of folder selection UI for them.
Comment 13 Rick Coloccia 2010-01-19 11:25:10 PST
OK I've gone through ALL imap accounts, creating folders where necessary, and making sure that Sent, Drafts, Templates, Archives and Junk all point to valid folders everywhere.  I'm restarting TB3.  We shall see what happens...
Comment 14 Rick Coloccia 2010-01-19 12:21:12 PST
I am still getting the error. It just happened on In-ARIN, In-ACUTA, In-BIND, and any other folder I clicked on.
Comment 15 Rick Coloccia 2010-01-19 12:58:32 PST
I've closed and restarted TB3 a few times, and still, after a moment or so, the error returns.
Comment 16 WADA 2010-01-19 21:17:19 PST
Try next, please.
1. Disable global search and indxer
2. Disable auto-sync of the imap account(Synchronization&Disk space)
3. Check msgFilterRules.dat content. next change may be reuired.
   ...imaps.geneseo.edu/In-ARIN => ...imaps.geneseo.edu/Inbox/In-ARIN
Comment 17 Rick Coloccia 2010-01-20 05:42:20 PST
The global search and indexer has been off (unchecked) all along.
In Synchronization & Storage / Message Synchronizing, I have unchecked "Keep messages for this account..."
I corrected my msgFilterRules dat earlier, when you had me remove the INBOX prefix, so the filter rules file was already correct.  I am restarting TB.
Comment 18 Rick Coloccia 2010-01-20 07:53:07 PST
I am still getting the error.
Comment 19 WADA 2010-01-20 18:13:54 PST
Get Process Monitor and check file I/O of In-ARIN and In-ARIN.msf.
> http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx
> Filter : If Path contains \In-ARIN, Include.
Someone deletes In-ARIN.msf?
Comment 20 Rick Coloccia 2010-01-22 05:42:27 PST
I just downloaded and installed process monitor from the link provided.  Added the filter exactly as you asked (path contains In-ARIN), launched TB3, got into the inbox just fine (as usual), stepped away from the computer, when I returned just a few minutes later, I was unable to access the In-ARIN folder, same error returned. I switched over to Process Monitor, and have the results.  They are attached. No other process is touching the file.
Comment 21 Rick Coloccia 2010-01-22 05:43:10 PST
Created attachment 422959 [details]
Process Monitor Results
Comment 22 WADA 2010-01-22 18:45:39 PST
(In reply to comment #21)
> Process Monitor Results

When(at which log line) was the error message of "Unable to open the summary file for In-ARIN" issued?
If before all attached log lines, the error message may be normal because .msf really doesn't exist and is created by Tb.
Comment 23 Rick Coloccia 2010-01-24 19:08:58 PST
OK here is more information.

With tb3 closed, I took a directory listing, in pre-folders.txt
I started process monitor with the filter for the In-ARIN folder.
I started tb3.
after a few minutes, I got the error again.
at this point, I closed tb3 and stopped the process monitor. Logfile1.csv.
I took another directory listing, post-folders.txt
Comment 24 Rick Coloccia 2010-01-24 19:10:42 PST
Created attachment 423294 [details]
process monitor output
Comment 25 Rick Coloccia 2010-01-24 19:11:29 PST
Created attachment 423295 [details]
pre directory listing
Comment 26 Rick Coloccia 2010-01-24 19:13:25 PST
Created attachment 423296 [details]
post directory listing
Comment 27 Rick Coloccia 2010-01-24 19:23:25 PST
Additional information:

I installed TB3.0.1 on another machine, in a brand new VM with absolutely nothing other than a fully-patched xp sp3 install.

I still get the error.
Comment 28 WADA 2010-01-24 19:43:18 PST
(In reply to comment #24)
> process monitor output

Following is last part of log.
Is Inbox/In-ARIN empty folder? Directory only folder(folder is grayed out)?
If so, change column selection of thread pane(saved in In-ARIN.msf).
Is Inbox.sbd\In-ARIN.msf deleted even after column change?

(Read data from In-ARIN.msf)
> "10:00:02.8580730 PM","thunderbird.exe","2288","WriteFile","...\INBOX.sbd\In-ARIN.msf","SUCCESS","Offset: 2,015, Length: 20"
> "10:00:02.8594984 PM","thunderbird.exe","2288","CloseFile","...\INBOX.sbd\In-ARIN.msf","SUCCESS",""

(CreateFile, Desired Access: Read Attributes, Delete)
> "10:00:44.0733443 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","SUCCESS","CreationTime: 1/24/2010 9:50:45 PM, LastAccessTime: 1/24/2010 10:00:02 PM, LastWriteTime: 1/24/2010 10:00:02 PM, ChangeTime: 1/24/2010 10:00:02 PM, AllocationSize: 4,096, EndOfFile: 2,035, FileAttributes: A"
> "10:00:44.0736029 PM","thunderbird.exe","2288","QueryDirectory","...\INBOX.sbd\In-ARIN.msf","SUCCESS","Filter: In-ARIN.msf, 1: In-ARIN.msf"
> "10:00:44.0739158 PM","thunderbird.exe","2288","CreateFile","...\INBOX.sbd\In-ARIN.msf","SUCCESS","Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened"
> "10:00:44.0740402 PM","thunderbird.exe","2288","QueryAttributeTagFile","...\INBOX.sbd\In-ARIN.msf","SUCCESS","Attributes: A, ReparseTag: 0x0"
> "10:00:44.0741656 PM","thunderbird.exe","2288","SetDispositionInformationFile","...\INBOX.sbd\In-ARIN.msf","SUCCESS","Delete: True"
(Close file => probably In-ARIN.msf is deleted by this close)
> "10:00:44.0742988 PM","thunderbird.exe","2288","CloseFile","...\INBOX.sbd\In-ARIN.msf","SUCCESS",""
> "10:00:44.0746313 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:00:44.0747916 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:00:44.0750249 PM","thunderbird.exe","2288","QueryDirectory","...\INBOX.sbd\In-ARIN.msf","NO SUCH FILE","Filter: In-ARIN.msf"

(In-ARIN.msf is not found, because In-ARIN.msf doesn't exist)
> "10:01:35.0869270 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:01:35.0870739 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:01:35.0872977 PM","thunderbird.exe","2288","QueryDirectory","...\INBOX.sbd\In-ARIN.msf","NO SUCH FILE","Filter: In-ARIN.msf"
> "10:01:35.0875547 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:01:35.0877168 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:01:35.0878897 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:01:35.0881232 PM","thunderbird.exe","2288","QueryDirectory","...\INBOX.sbd\In-ARIN.msf","NO SUCH FILE","Filter: In-ARIN.msf"
> "10:01:35.0889940 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:01:35.0891518 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:01:35.0893832 PM","thunderbird.exe","2288","QueryDirectory","...\INBOX.sbd\In-ARIN.msf","NO SUCH FILE","Filter: In-ARIN.msf"
> "10:01:35.0896508 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:01:35.0898206 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:01:35.0899631 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:01:35.0901930 PM","thunderbird.exe","2288","QueryDirectory","...\INBOX.sbd\In-ARIN.msf","NO SUCH FILE","Filter: In-ARIN.msf"
> "10:01:51.3631962 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:01:51.3635764 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:01:51.3638295 PM","thunderbird.exe","2288","QueryDirectory","...\INBOX.sbd\In-ARIN.msf","NO SUCH FILE","Filter: In-ARIN.msf"
> "10:01:51.3641245 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:01:51.3642960 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:01:51.3647064 PM","thunderbird.exe","2288","QueryOpen","...\INBOX.sbd\In-ARIN.msf","NAME NOT FOUND",""
> "10:01:51.3649640 PM","thunderbird.exe","2288","QueryDirectory","...\INBOX.sbd\In-ARIN.msf","NO SUCH FILE","Filter: In-ARIN.msf"
Comment 29 WADA 2010-01-24 23:23:15 PST
Next log was seen in my Process Monitor log for open of [Gmail]/Drafts.
Similar logs are seen in your log(...\Temp\MozillaMailnews\In-ARIN.msf). 
> 88904	15:55:19.0424651 thunderbird.exe 3404 IRP_MJ_CREATE
> C:\Documents and Settings\wada\Local Settings\Temp\MozillaMailnews\Drafts.msf SUCCESS 
> Desired Access: Read Attributes, Delete, Disposition: Open,
> Options: Non-Directory File, Open Reparse Point, Attributes: n/a,
> ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened
It's probably for .msf update(new version is written in Temp, then .msf is replaced by data in %Temp%).

Next in your log may be for ...\Local Settings\Temp\MozillaMailnews\In-ARIN.msf. (Wrong file handle is used in handling of file in Temp).
I can't understand reason why "Read, Delete" open of original .msf file.
> "10:00:44.0739158 PM","thunderbird.exe","2288","CreateFile","...\INBOX.sbd\In-ARIN.msf","SUCCESS",
> "Desired Access: Read Attributes, Delete, Disposition: Open,
>  Options: Non-Directory File, Open Reparse Point, Attributes: n/a,
>  ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened"

Another concern.
As ...\Local Settings\Temp\MozillaMailnews\Drafts.msf was used for [Gmail]/Drfats (...\[Gmail].sbd\Drafts.msf), problem may occur if same folder exists under different parent folders/accounts and they are opened concurrently.
Even if suffix is used for file in Temp, a/x and b/x can produce temp file of x, x-1, and c/x may use x-2, and c/x-1 may use x-3, and d/x-3 may uses x-4, ...
Comment 30 Rick Coloccia 2010-01-25 04:27:46 PST
You asked:
Is Inbox/In-ARIN empty folder? Directory only folder(folder is grayed out)?
If so, change column selection of thread pane(saved in In-ARIN.msf).
Is Inbox.sbd\In-ARIN.msf deleted even after column change?



No, it is not a greyed-out empty folder, it is a folder into which messages are filtered.
Comment 31 Wayne Mery (:wsmwk, NI for questions) 2010-01-26 14:26:14 PST
(In reply to comment #0)
> I never had this problem with TB2.

should = regression (at least tentatively)
Comment 32 WADA 2010-01-26 20:05:28 PST
Can you get NSPR log with timestamp,MSGDB:5 using Tb 3.1 nightly build?
> https://wiki.mozilla.org/Thunderbird:Testing:Memory_Usage_Problems
Comment 33 Rick Coloccia 2010-01-28 06:15:37 PST
This is the nightly I downloaded and installed:
thunderbird-3.1b1pre.en-US.win32.installer.exe

This is the batch file I used to start the logging:
set NSPR_LOG_MODULES=MSGDB:5
set NSPR_LOG_FILE=c:\msgdb.log
"C:\Program Files\Lanikai\thunderbird.exe"


I launched the batch file, it properly found my profile.

Right away I was getting the error again.

After only a minute, I closed TB31b1.

A 34.3 MB log file was generate, I am attaching it now.
Comment 34 Rick Coloccia 2010-01-28 06:26:08 PST
Created attachment 423989 [details]
MSGDB Level 5 Log
Comment 35 WADA 2010-01-28 07:07:05 PST
(In reply to comment #33)
> thunderbird-3.1b1pre.en-US.win32.installer.exe
Download win32.zip build, if you are not testing installer of Tb intentionally.
You can test by UNZIP only with win32.zip build.
Comment 36 Rick Coloccia 2010-01-28 07:18:35 PST
OK, I did it again.  I downloaded thunderbird-3.1b1pre.en-US.win32.zip.

Batch file:

set NSPR_LOG_MODULES=MSGDB:5
set NSPR_LOG_FILE=C:\msgdb.log
"C:\Documents and Settings\coloccia.GENESEO\Desktop\tb3 bug\thunderbird-3.1b1pre.en-US.win32\thunderbird\thunderbird.exe"

logfile is zipped and attached.
Comment 37 Rick Coloccia 2010-01-28 07:19:21 PST
Created attachment 423995 [details]
MSGDB Level 5 Log File
Comment 38 David :Bienvenu 2010-01-28 07:47:37 PST
Using mozilla error lookup, http://silver.warwickcompsoc.co.uk/mozilla/misc/nserror we see that 0x80550006 is NS_MSG_ERROR_FOLDER_SUMMARY_MISSING, and 0x80520006 is NS_ERROR_FILE_TARGET_DOES_NOT_EXIST.

We should not be opening so many db's (unless you delete panacea.dat), or at least, we should not be leaving so many of them open. You may be running into a limit of the number of open file handles (though I thought Windows had a much higher limit).

One question is what's opening all the databases. You might try turning off imap autosync (using the config editor, toggle mail.server.default.autosync_offline_stores to false and restart). Can you attach a log with the results of a run with autosync turned off?
Comment 39 David :Bienvenu 2010-01-28 07:48:42 PST
BTW, thanks very much for working with us on this.
Comment 40 Rick Coloccia 2010-01-28 07:57:13 PST
You're welcome, I'm glad to help out!

I disabled mail.server.default.autosync_offline_stores, it is now bold and set to false.

The error still happened right away. I'll attach the log file now.  Thanks again!
Comment 41 Rick Coloccia 2010-01-28 07:58:00 PST
Created attachment 424005 [details]
MSGDB Level 5 Log File

with autosymc_offline_stores off.
Comment 42 David :Bienvenu 2010-01-28 08:24:19 PST
OK, I think we need to add some more logging to get a better picture of what's going on when the db's are opened.

set NSPR_LOG_MODULES=MSGDB:5,timestamp,IMAP:5,MsgPurge:5

The imap logging might cause some private information to get into the log, (e.g., message contents), so you might want to sanitize it). It might also make the log too big to attach, in which case you can e-mail it to me (bienvenu@mozillamessaging.com)

You might also try turning off gloda and win search integration, if either of them are turned on - tools | options | advanced, general tab, uncheck "enable global search and indexer" and "Allow Windows Search to search messages". I believe both of them are well-behaved in terms of closing db's they open, but someone is opening all the db's...
Comment 43 Rick Coloccia 2010-01-28 10:12:47 PST
"enable global search and indexer" is unchecked.  That was a troubleshooting step I was asked to take early on, so that's already off.

I have modified the batch file to include this:
set NSPR_LOG_MODULES=MSGDB:5,timestamp,IMAP:5,MsgPurge:5
instead of 
set NSPR_LOG_MODULES=MSGDB:5

I don't see the "Allow windows search to search messages" on the tools | options | advanced, general tab.  Maybe because I don't have windows search on the laptop? In any event, I looked in the config editor, and found that mail.winsearch.enable is already set to false.

I am suddenly, and very frustratingly, unable to reproduce the problem.  It usually happens so suddenly.
Comment 44 Rick Coloccia 2010-01-28 10:47:20 PST
Good, It's happening again. I'll get a log file prepared soon.
Comment 45 Rick Coloccia 2010-01-28 11:18:05 PST
Ok I finally have a log file. it is 373 MB uncompressed, 18MB compressed.  It can be download from here:

http://status.geneseo.edu/~coloccia/msgdb.zip

TB3 didn't start throwing the error until just before I quit, so the errors may be towards the end of the file.
Comment 46 David :Bienvenu 2010-01-28 13:17:21 PST
Rick, do you have Thunderbird set to check all your imap folders for new mail, either through setting the hidden pref mail.check_all_imap_folders_for_new or by checking the box in the folder properties for each folder? I'm trying to figure out which code is responsible for doing the STATUS commands on your folders. I suspect that's what's leaving the db's open...
Comment 47 David :Bienvenu 2010-01-28 14:09:50 PST
Created attachment 424108 [details] [diff] [review]
proposed fix

From what I can see in the log, I believe this will fix the issue. I need to write a unit test for this. I'd also like to spin up a try server build for Rick to try. Once I do, I'll post a link here.
Comment 48 Rick Coloccia 2010-01-28 14:18:24 PST
Yes, I do have mail.check_all_imap_folders_for_new set to true.  I'm excited to hear you have some ideas about the issue!
Comment 49 David :Bienvenu 2010-01-29 12:06:29 PST
Created attachment 424291 [details] [diff] [review]
proposed fix with unit test

fix is to close db's after doing status command, as long as the folder isn't open in a window. The unit test verifies that the db's aren't left open.
Comment 50 Rick Coloccia 2010-01-30 05:05:00 PST
Is there a new nightly build I should test the patch with?
Comment 51 David :Bienvenu 2010-01-30 06:20:14 PST
I tried twice to make try server builds for you, but the windows try server box is not happy at the moment.
Comment 52 Rick Coloccia 2010-01-30 07:25:33 PST
Understood... I'll wait for notice via bugzilla that a test build is available.  Thank you for all your efforts!
Comment 53 Mark Banner (:standard8) 2010-01-30 13:18:05 PST
I fixed the broken builder, a windows build based on today's 3.0.2pre source with the patch is available here for testing:

http://s3.mozillamessaging.com/build/try-server/2010-01-30_11:30-bugzilla@standard8.plus.com-bienvenu-540214/bugzilla@standard8.plus.com-bienvenu-540214-mail-try-win32.installer.exe
Comment 54 Wayne Mery (:wsmwk, NI for questions) 2010-01-30 13:56:21 PST
bienvenu, this would fix (much older) bug 217271 comment 6?
Comment 55 Rick Coloccia 2010-01-30 15:55:04 PST
I have just installed the patched 3.0.2.pre as prvovided. I'll use it a bit and report back.
Comment 56 Charles Cox 2010-02-02 12:11:49 PST
I'm seeing the same thing with TB 3.0 and 3.0.1 on Solaris.  I also have the mail.check_all_imap_folders_for_new setting set to true.
Comment 58 David :Bienvenu 2010-02-03 17:29:46 PST
thx, Rick. Standard8, I think this would be a good patch for 3.02, since it fixes one of the known causes of memory bloat.
Comment 59 David :Bienvenu 2010-02-09 08:32:43 PST
Created attachment 425986 [details] [diff] [review]
better error handling
Comment 60 David :Bienvenu 2010-02-09 15:39:55 PST
fixed on trunk.
Comment 61 David :Bienvenu 2010-02-11 08:17:42 PST
fixed for 3.02
Comment 62 Wayne Mery (:wsmwk, NI for questions) 2010-02-16 05:53:30 PST
*** Bug 516395 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.