332119 - IMAP folders: if sum of characters in foldernames to long TB crashes or loops when you access the last folder in chain. [@ nsMsgDatabase::GetTableCreateIfMissing]

Status: RESOLVED FIXED

(MailNews Core :: Database, defect)

Description

[Michael]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1
Build Identifier: Thunderbird 1.5

On IMAP Server TB crashes or loops if your foldername or the sum of the characters of an IMAP-folder reaches a value. On our shared folder its about 70 characters. On my IMAP account (same server) much more - about 115. The characters of the foldernames can be fragmented on a chain of subfolders!

look at "Additional Information" for more important details!

Reproducible: Always

Steps to Reproduce:
1. Use TB Windows-Client and configure an IMAP-account - if possible use "shared folders" on IMAP Server and share it on your account!
2. make a folder 
3. make more subfolders with long names (subfolders of subfolders - not on same level)
4. If sum of characters in the foldername reach a value TB will crash or loop.

Actual Results:  
a.) TB crashes and Talkback Agent starts
b.) TB loops and you have to kill the process

notice: for my feeling a.) occurs only if you click on the "long folder" very quickly after starting TB

Expected Results:  
TB shows the content of the folder

*)How to see a "bad folder" without getting crashed?

You can see if TB has a problem with a folder when showing the properties for the folder: Look at "Default Character Encoding"  if there is "Western ISO..." or somthing its ok.
If it shows  "Arabic IBM..." the foldername in sum is too long and you will crash when clicking on the folder! 
Also the second Properties-window, "Retention Policiy" seems to be damaged and the other properties rights, quota) do not work.
You dont crash by looking the Properties!

*)What di i mean with folder sum?

You can hace one subfolder and this Subfolder has many charactars or you can have a chain of subfolder - so this subfolders on an IMAP-Server are one lomg name what means the name is a sum of all names. 

*)How long can the foldernames or the sum of the foldernames be?

Its not allways the same!
We use shared folders on Courier IMAP Server to share more users with some kind of an mailarchiv.  We have Folders which have in sum about 68 charactars  other crashes at 70 characters.
On my normal IMAP-account i made a subfolder with about 115 characters. 

Examples at shared folders:
.Archiv-EQ2.diverse_Kunden.TEST3.TEST_4.TEST_5.TEST_1234567890123456   ->works
.Archiv-EQ2.diverse_Kunden.TEST3.TEST_4.TEST_5.TEST_12345678901234567  ->crash

*)Its possible to rename a "bad folder" an shorten it - so then it works again.

*)This problems do not exist on Linux or OSX clients.

at least:
someoneelse tested it on an completly different IMAP-server - same result.
i tested it with the rc 1.5.0.2 - same result.
I sent Talback Information with my emailadress.

Comment 1

[David :Bienvenu]

This is almost certainly because of the windows file system limit on the full path name of a file. Do you have incident id's for the talkback reports you filed?

You could also change the local directory for the imap server account under tools | account settings | server settings - local directory, and point it to a directory outside your profile, with a much shorter path.  If you have filters and/or saved searches, you may either lose them or have to copy some files from the old directory to the new directory.

Comment 2

[Michael]

(In reply to comment #1)
> This is almost certainly because of the windows file system limit on the full
> path name of a file. 

ok. its a little bit depressing that its "so easy"

>Do you have incident id's for the talkback reports you
> filed?

i didnt see a id at my Talkbackagent - where can I see this?
i can reproduce when you give information...

> 
> You could also change the local directory for the imap server account under
> tools | account settings | server settings - local directory, and point it to a
> directory outside your profile, with a much shorter path.  If you have filters
> and/or saved searches, you may either lose them or have to copy some files from
> the old directory to the new directory.

thx for quick reply - that helps a lot!

although, a too long folder path should be intercepted with an error message like "path name too long" or what ever.
A crash is no solution and hast cost us a lot of time for debugging.

And at least that cashingsystem should change!
I have a dream, that TB will work one day without confighacking and limitation! :)

Comment 3

[timeless]

this is your crash report, normally you can get it by running talkback later.

Incident ID: 16999528 
Stack Signature nsMsgDatabase::GetTableCreateIfMissing 278ed631 
Product ID Thunderbird15 
Build ID 2005120115 
Trigger Time 2006-03-29 06:53:49.0 
Platform Win32 
Operating System Windows NT 5.1 build 2600 
Module thunderbird.exe + (004fc1f5) 
URL visited  
User Comments  
Since Last Crash 12 sec 
Total Uptime 1158885 sec 
Trigger Reason Access violation 
Source File, Line No. e:/builds/tinderbox/Tb-Mozilla1.8/WINNT_5.0_Depend/mozilla/mailnews/db/msgdb/src/nsMsgDatabase.cpp, line 1557 
Stack Trace  

nsMsgDatabase::GetTableCreateIfMissing  [e:/builds/tinderbox/Tb-Mozilla1.8/WINNT_5.0_Depend/mozilla/mailnews/db/msgdb/src/nsMsgDatabase.cpp, line 1557]
nsMailDatabase::GetAllOfflineOpsTable  [e:/builds/tinderbox/Tb-Mozilla1.8/WINNT_5.0_Depend/mozilla/mailnews/db/msgdb/src/nsMailDatabase.cpp, line 119]
nsImapMailFolder::UpdateImapMailboxInfo  [e:/builds/tinderbox/Tb-Mozilla1.8/WINNT_5.0_Depend/mozilla/mailnews/imap/src/nsImapMailFolder.cpp, line 2607]
XPTC_InvokeByIndex  [e:/builds/tinderbox/Tb-Mozilla1.8/WINNT_5.0_Depend/mozilla/xpcom/reflect/xptcall/src/md/win32/xptcinvoke.cpp, line 102]
EventHandler  [e:/builds/tinderbox/Tb-Mozilla1.8/WINNT_5.0_Depend/mozilla/xpcom/proxy/src/nsProxyEvent.cpp, line 562]
0x778b0c24
nsHTMLEditor::SetFinalSize  [e:/builds/tinderbox/Tb-Mozilla1.8/WINNT_5.0_Depend/mozilla/editor/libeditor/html/nsHTMLObjectResizer.cpp, line 961]
0x029a0292
0x9dba4f24


note that if you want, you can work around this problem by using subst and junction (or net share+net use), see http://sysinternals.com/ they can let you make shorter paths, at least as long as you have something short as a starting point. something like

subst m: "c:\Documents and Settings\your very very long username\Application Data\Mozilla\Profiles\some very very very long profile name\somethingrandom.slt\ImapMail\localhost"

If that isn't good enough, because the path between along the imap directory chain itself is too long, then, you're much more stuck. we could teach nsifile and friends to deal w/ it, it's a bit painful but should be done eventually.

Comment 4

[David :Bienvenu]

that would be excellent if the too long path causes the impossible (for me) to reproduce CreateTableIfMissing bug!

Comment 5

[Michael]

(In reply to comment #3)
thanks for the hint with the path and the subst.
yesterday i changed the profile path to c:\TBP  
so that gave me about 110 characts more.
at the moment the IMAP paths are not too long and i told the users that they shouldn`t make the paths to deep and keep the foldernames short.
so at the moment we can deal with it, but i hope that the problem will be fixed or the cashingsystem changes in a later release of TB :)

ad id`s:
TB16999528Z
TB16992730Y
these are the ids from 29.03.
and these is one from 14.03.( i dont remember if it was the same prob)
TB16346596W

Comment 6

[David :Bienvenu]

Attachment: proposed fix

if creating a new .msf file fails at the lowest level, use NS_ERROR_FILE_TARGET_DOES_NOT_EXIST as a special error code, and propagate that all the way up to the imap code. Have the imap code that updates a folder put up an error message in that one particular case where the user clicks on a folder. 

This should fix that pesky top crash in CreateTableIfMissing.

Comment 7

[David :Bienvenu]

fixed on trunk - I'll land this on the 1.8 branch as well...

Comment 8

[David :Bienvenu]

Comment on attachment 216874 [details] [diff] [review]
proposed fix

I think this fixes a top crash so we might want it for the next 1.8.0.x build

Comment 9

[Tim Riley [:timr]]

Comment on attachment 216874 [details] [diff] [review]
proposed fix

Cannot add strings in a maint release. This is a big L10n impact. If prompt is required, then talk to Axel for special patch.

Comment 10

[David :Bienvenu]

Comment on attachment 216874 [details] [diff] [review]
proposed fix

actually, for 1.8.0.3, I just wanted to check in the part that fixes the crash, i.e., the nsMsgDatabase.cpp change and nothing else.

Comment 11

[David :Bienvenu]

fixed for 2.0 as well.

Comment 12

[Daniel Veditz [:dveditz]]

Comment on attachment 216874 [details] [diff] [review]
proposed fix

approved for 1.8.0 branch only for the nsMsgDatabase.cpp changes (i.e. no localization changes), a=dveditz for drivers

Comment 13

[Dave Liebreich [:davel]]

Please provide guidance on how to verify this fix on the 1.8.0 branch.  I can do the "long folder name" test, but I'm wondering what else calls down through nsMsgDBService::RegisterPendingListener that might be affected.

Comment 14

[David :Bienvenu]

nsMsgDBService::RegisterPendingListener isn't really at issue here - the core issue is that nsMsgDatabase::Open could return success but with a db that wasn't correctly opened, so that a subsequent method on the db would crash (most likely nsMsgDatabase::GetTableCreateIfMissing). The long filename test is probably sufficient - checking the talkback reports to see if crashes in nsMsgDatabase::GetTableCreateIfMissing or nsMailDatabase::GetAllOfflineOpsTable don't appear in builds after this checkin would also be a good verification. believe there are other situations besides long filenames that could have caused the error opening the db, but the fix is general.

Comment 15

[Dave Liebreich [:davel]]

creating or accessing the final folder fails silently on win.  marking verified

Comment 17

[Carsten Book [:Tomcat]]

Verified fixed for 1.8.1.3 using Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.3) Gecko/20070326 Thunderbird/2.0.0.0 Mnenhy/0.7.5.0 ID:2007032620 (Thunderbird 2 RC 1)

I can create, access and use the final folder from the steps to reproduce from comment #0