535125 - crash on copying folders from old pop account to new imap account [@ nsMsgCopyService::ClearRequest]

Status: RESOLVED DUPLICATE of bug 531568

(MailNews Core :: Backend, defect)

Description

[Harald Dunkel]

User-Agent:       Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.5) Gecko/20091109 Firefox/3.5.5
Build Identifier: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.5) Gecko/20091210 Lightning/1.0pre Thunderbird/3.0

Trying to copy a recursive folder structure from my old pop-based account ("keep message on server" is off) to a new imap account Thunderbird first showed an error popup saying 

The current command did not succeed.  The mail server responded:[TRYCREATE] Mailbox doesn't exist: [e-mail]^^test2.test2a.test2b.test2c
.

then it died with SIGSEGV. 

Imap server is dovecot ("mail_location = maildir:/var/spool/imap/%u/Maildir", folder seperator is "." by default).

After a restart thunderbird shows a corrupted mail folder name in the new subfolder (see screenshot). Other subfolders were not created, Most EMails were not copied, either.


Reproducible: Always

Steps to Reproduce:
1.create a folder hierarchy with 2 subfolders on a pop account

[e-mail]
    |
    +-- test1.test1a.test1b
    |
    +-- test2.test2a.test2b.test2c

and put some EMails in every folder

2. use drag-and-drop to copy "[e-mail]" to a new imap account (using maildir, folder separator is ".")

3. watch it die

Comment 1

[Harald Dunkel]

Attachment: screenshot showing the corrupted folder name

Comment 2

[Harald Dunkel]

PS: I can offer a thunderbird log file for this test case in private mail.

Comment 3

[Ludovic Hirlimann [:Usul]]

(In reply to comment #2)
> PS: I can offer a thunderbird log file for this test case in private mail.

What did you log ?

Comment 4

[Kent James (:rkent)]

This may be the same as bug 531568, which also seems to occur for cross-folder moves.

Comment 5

[Harald Dunkel]

(In reply to comment #3)
> 
> What did you log ?

I started Thunderbird with NSPR_LOG_MODULES=imap:5,timestamp and reproduced the problem. Should I try some more options?

Comment 6

[Ludovic Hirlimann [:Usul]]

can you attach the logs here (use the add attachment link above), clean up personnal bits of informations in the logs too. ?

Comment 7

[Harald Dunkel]

(In reply to comment #4)
> This may be the same as bug 531568, which also seems to occur for cross-folder
> moves.

I am not sure about this. AFAICS tb asked the imap server to create some folders and subfolders with corrupted names. When tb tried to push some EMails into the new subfolders the imap server complained and tb died. Doesn't look like a destructor issue to me.

Can you reproduce this problem?

Comment 8

[Harald Dunkel]

Attachment: log file created with NSPR_LOG_MODULES=imap:5,timestamp

Comment 9

[Harald Dunkel]

(In reply to comment #6)
> can you attach the logs here (use the add attachment link above), clean up
> personnal bits of informations in the logs too. ?

Done. Hopefully I did not corrupt the log on editing.

Comment 10

[Harald Dunkel]

I managed to create a crash dump:

(gdb) bt
#0  0x00007fabdac40c91 in nanosleep () from /lib/libc.so.6
#1  0x00007fabdac40ae0 in sleep () from /lib/libc.so.6
#2  0x000000000047712d in ah_crap_handler (signum=11)
    at /home/hdunkel/debian/raw-thunderbird/raw-thunderbird-3.0/comm-1.9.1/mozilla/toolkit/xre/nsSigHandlers.cpp:149
#3  0x000000000047d36a in nsProfileLock::FatalSignalHandler (signo=11) at nsProfileLock.cpp:216
#4  <signal handler called>
#5  0x00007fabe112fcd8 in NS_LogCOMPtrRelease_P () from /usr/lib/thunderbird/libxpcom_core.so
#6  0x00000000011ba591 in ~nsCOMPtr (this=0x7fabc5bad6b8, __in_chrg=<value optimized out>)
    at ../../../mozilla/dist/include/xpcom/nsCOMPtr.h:508
#7  0x00000000013b3ab3 in ~nsCopyRequest (this=0x7fabc5bad6a0, __in_chrg=<value optimized out>)
    at /home/hdunkel/debian/raw-thunderbird/raw-thunderbird-3.0/comm-1.9.1/mailnews/base/src/nsMsgCopyService.cpp:100
#8  0x00000000013b3b65 in nsMsgCopyService::ClearRequest (this=0x7fabd1efd3c0, aRequest=0x7fabc5bad6a0, rv=2147500037)
    at /home/hdunkel/debian/raw-thunderbird/raw-thunderbird-3.0/comm-1.9.1/mailnews/base/src/nsMsgCopyService.cpp:216
#9  0x00000000013b41d6 in nsMsgCopyService::NotifyCompletion (this=0x7fabd1efd3c0, aSupport=0x7fabcbd17000, 
    dstFolder=0x7fabc5c5d440, result=2147500037)
    at /home/hdunkel/debian/raw-thunderbird/raw-thunderbird-3.0/comm-1.9.1/mailnews/base/src/nsMsgCopyService.cpp:664
#10 0x0000000001228f86 in nsImapMailFolder::OnCopyCompleted (this=0x7fabc5c5d400, srcSupport=0x7fabcbd17000, 
    rv=2147500037)
    at /home/hdunkel/debian/raw-thunderbird/raw-thunderbird-3.0/comm-1.9.1/mailnews/imap/src/nsImapMailFolder.cpp:7947
#11 0x0000000001242a29 in nsImapMailFolder::OnStopRunningUrl (this=0x7fabc5c5d400, aUrl=0x7fabc5bc5008, 
    aExitCode=2147500037)
    at /home/hdunkel/debian/raw-thunderbird/raw-thunderbird-3.0/comm-1.9.1/mailnews/imap/src/nsImapMailFolder.cpp:5249
#12 0x00000000014b0bca in nsMsgMailNewsUrl::SetUrlState (this=0x7fabc5bc5008, aRunningUrl=<value optimized out>, 
    aExitCode=2147500037)
    at /home/hdunkel/debian/raw-thunderbird/raw-thunderbird-3.0/comm-1.9.1/mailnews/base/util/nsMsgMailNewsUrl.cpp:135
#13 0x00000000012294f8 in nsImapMailFolder::SetUrlState (this=0x7fabc5c5d400, aProtocol=0x7fabc772a000, 
    aUrl=0x7fabc5bc5008, isRunning=0, statusCode=2147500037)
    at /home/hdunkel/debian/raw-thunderbird/raw-thunderbird-3.0/comm-1.9.1/mailnews/imap/src/nsImapMailFolder.cpp:6596
#14 0x00007fabe1135fb7 in NS_InvokeByIndex_P () from /usr/lib/thunderbird/libxpcom_core.so
#15 0x00007fabe11278e9 in ?? () from /usr/lib/thunderbird/libxpcom_core.so
#16 0x00007fabe111e5e6 in ?? () from /usr/lib/thunderbird/libxpcom_core.so
#17 0x00007fabe10c0d29 in NS_ProcessNextEvent_P(nsIThread*, int) () from /usr/lib/thunderbird/libxpcom_core.so
#18 0x0000000000787405 in nsBaseAppShell::Run (this=0x7fabd7525820)
    at /home/hdunkel/debian/raw-thunderbird/raw-thunderbird-3.0/comm-1.9.1/mozilla/widget/src/xpwidgets/nsBaseAppShell.cpp:170
#19 0x000000000105f67e in nsAppStartup::Run (this=0x7fabd36b2650)
    at /home/hdunkel/debian/raw-thunderbird/raw-thunderbird-3.0/comm-1.9.1/mozilla/toolkit/components/startup/src/nsAppStartup.cpp:193
#20 0x000000000046bbfd in XRE_main (argc=<value optimized out>, argv=<value optimized out>, 
    aAppData=<value optimized out>)
    at /home/hdunkel/debian/raw-thunderbird/raw-thunderbird-3.0/comm-1.9.1/mozilla/toolkit/xre/nsAppRunner.cpp:3321
#21 0x00000000004651a4 in main (argc=1, argv=0x7fff672eba58)
    at /home/hdunkel/debian/raw-thunderbird/raw-thunderbird-3.0/comm-1.9.1/mail/app/nsMailApp.cpp:103


Hope this helps.

Comment 11

[Kent James (:rkent)]

I tried copying a local folder tree, similar to comment 0, to an IMAP server. I got three different results in three trials, but none of them were crashes.

On the first trial, the folders were copied to the same level in the structure, but the names of the subfolders were appended to the name of the parent, with one or two non-ascii characters between them.

On the second trial, all seemed to go well - but then I could delete the copied subfolders, but not the root folder.

On the third trial, all went well, but I cannot now delete ANY of the copied folders.

Delete fails with the message "The current command did not succeed.  The mail server responded:Mailbox exists."

So I think it is clear that something strange is going on here, I'm just not sure what. Because my results are a little different than Harald's, and are not a crash, that will make it harder to debug. But I'll at least confirm this for now as worthy of additional investigation.

Comment 12

[WADA:World Anti-bad-Duping Agency]

(In reply to Comment 8)
> log file created with NSPR_LOG_MODULES=imap:5,timestamp

It looks next.

(1) ensureExists%3E/test%40test.de:  = currentUrl
    /test%40test.de == top level copy target folder.
> 2009-11-16 12:57:59.693937 UTC - 705689872[7fec32905ef0]: > 31396000:dovecot.some.domain.de:NA:ProcessCurrentURL:imap://[e-mail]:143/ensureExists%3E/test%40test.de:  = currentUrl

(2) appendmsgfromfile%3E%5Etest%40test.de:  = currentUrl
    "/" before test%40test.de is probably replaced by "^"(0x5E),
    and escaped in internal URL representation.
> 2009-11-16 12:57:59.783625 UTC - 771471632[7fec313fc260]:
> 31465800:dovecot.some.domain.de:NA:ProcessCurrentURL:imap://[e-mail]:143/appendmsgfromfile%3E%5Etest%40test.de:  = currentUrl

(3) ensureExists%3E%5Etest%40test.de%5Etest2.test2a.test2b.test2c:  = currentUrl)
       %5Etest%40test.de/Etest2.test2a.test2b.test2c    
    => %5Etest%40test.de^Etest2.test2a.test2b.test2c    
    => %5Etest%40test.de%5Etest2.test2a.test2b.test2c
    => create request after unescape becomes;
       [e-mail]^test2.test2a.test2b.test2c    
> 2009-11-16 12:58:00.006190 UTC - 771471632[7fec313fc260]:
>
31465800:dovecot.some.domain.de:A:ProcessCurrentURL:imap://[e-mail]:143/ensureExists%3E%5Etest%40test.de%5Etest2.test2a.test2b.test2c:  = currentUrl
> 3 list "" "[e-mail]^test2.test2a.test2b.test2c"
> 3 OK List completed.
> 4 create "[e-mail]^test2.test2a.test2b.test2c"
> 4 OK Create completed.
> 5 subscribe "[e-mail]^test2.test2a.test2b.test2c"
> 5 OK Subscribe completed.
> 6 list "" "[e-mail]^test2.test2a.test2b.test2c"
> * LIST (\HasNoChildren) "." "[e-mail]^test2.test2a.test2b.test2c"
> 6 OK List completed.

(4) appendmsgfromfile%3E%5Etest%40test.de%5E%5Etest2.test2a.test2b.test2c:  = currentUrl
    /test2.test2a.test2b.test2c is appended to %5Etest%40test.de%5E,
    and "/" before "test2" is replaced by ^(0x5E),
    and "^" is escaped for internal URL.
    After unescape of URL, append to [e-mail]^^test2.test2a.test2b.test2c. 
> 2009-11-16 12:58:00.106342 UTC - 705689872[7fec32905ef0]:
> 31396000:dovecot.some.domain.de:A:ProcessCurrentURL:imap://[e-mail]:143/appendmsgfromfile%3E%5Etest%40test.de%5E%5Etest2.test2a.test2b.test2c:  = currentUrl
> 13 append "[e-mail]^^test2.test2a.test2b.test2c" (\Flagged) "16-Dec-2009

Comment 13

[WADA:World Anti-bad-Duping Agency]

IMAP folder structure in screen shot.
>  test@test (grayed out)
>    de      (not grayed out)
>    de^test2 + 0x0000 (grayed out)
>      test2a (grayed out)
>        test2b (grayed out)
>          test2c (not grayed out)
Server looks to have createed [e-mail], [e-mail]^test2 and folders under it, as requested. Garbage of 0x0000 after de^test2 seems cause of crash. It may be produced by wrong ".de^^test2."(excess ^ in name, longer than real folder name).

Comment 14

[Harald Dunkel]

As a workaround I have changed my local dovecot configuration to use '/' as a separator and to store mail folders in nested subdirectories:

namespace private {
	separator = /
	prefix =
	location = maildir:~/Maildir:LAYOUT=fs
	inbox = yes
}

Using this configuration I could migrate about 100MByte EMails from the local directory to the maildir managed by dovecot.

Comment 15

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #14)
> As a workaround I have changed my local dovecot configuration to use '/' as a
> separator(snip)

As you reported to bug 537591, Dovecot doen't permit creation of folder like [e-mail] and it silently fails if Tb is used. It was not a problem in your workaround? Did you removed "." from all local mail folder structure befor migration?

Comment 16

[Harald Dunkel]

AFAICS Dovecot doesn't have this restriction for using "LAYOUT=fs". Folder names like "[e-mail]" were accepted without problems. See http://wiki.dovecot.org/MailboxFormat/Maildir

But of course this is just a workaround. If you have a fix for Thunderbird then I would be glad to verify it.

Comment 17

[Wayne Mery (:wsmwk)]

Is this bug two issues, the slash issue, plus the crash?

crash looks like a duplicate Mac stack I have in Bug 542431 - crash [@ nsMsgCopyService::ClearRequest(nsCopyRequest*, unsigned int)]. Bug 542431 is a low occurrence crash.

Comment 18

[David :Bienvenu]

(In reply to comment #17)
> Is this bug two issues, the slash issue, plus the crash?
> 
That's correct, and I believe the crash is a dup of bug 531568. The slash might be a dup as well.

Comment 19

[Wayne Mery (:wsmwk)]

Comment on attachment 418151 [details]
log file created with NSPR_LOG_MODULES=imap:5,timestamp

stack lacks symbols

Comment 20

[Wayne Mery (:wsmwk)]

Comment on attachment 418151 [details]
log file created with NSPR_LOG_MODULES=imap:5,timestamp

my mistake (wrong bug)

Comment 21

[Wayne Mery (:wsmwk)]

(In reply to comment #18)
> (In reply to comment #17)
> > Is this bug two issues, the slash issue, plus the crash?
> > 
> That's correct, and I believe the crash is a dup of bug 531568. The slash might
> be a dup as well.

please reopen if you still see this crash in version 3.1