611513 - Moving folder with dot in name causes problems with IMAP

Status: UNCONFIRMED

(MailNews Core :: Networking: IMAP, defect)

Description

[craig]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12 GTB7.1 ( .NET CLR 3.5.30729; .NET4.0C)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.12) Gecko/20101027 Lightning/1.0b2 Thunderbird/3.1.6

A dot is used as a folder separator in the IMAP protocol. There is a bug with TB when moving a folder with a dot in the name.

Reproducible: Always

Steps to Reproduce:
1. Create a folder with a dot in the name.
2. Move the folder with drag and drop to another folder. It will fail because the dot will be sent directly to the IMAP command, instead of escaped.
Actual Results:  
If you move "my.folder", you will get "my/folder"

Expected Results:  
Creating "my.folder" should do exactly that.

Comment 1

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #0)
> If you move "my.folder", you will get "my/folder"

"Move folder between accounts"? Or "Move folder within an account"?
Please note that "Move folder between accounts" is not supported by Tb since initial. "Drag&Drop of a folder between accounts" is always "Copy of already known mail data by Tb in a folder and his subfolders" since initial.  

What folder do you call "my/folder"?
Folder named "my/folder" at copy target IMAP account?
Subfolder named "folder" under "my" folder at copy target IMAP account?

> the dot will be sent directly to the IMAP command

It's current implementation.
When user requests to create "my/folder" or "my.folder" via folder creation UI of Tb, Tb probably issues "create my/folder" or "create my.folder" according to your request regardless of delimiter at IMAP server, because interpretation of "/" or "." is up to IMAP server.
I guess you dragged local folder or IMAP folder(delimiter=/) named "my.folder" to an IMAP account of delimiter=".", and Tb issued "create my.folder", thus subfolder named "folder" under "my" folder was created at copy target IMAP account.

What is "delimiter character of mail folder hierarchy of you IMAP server"?
Can you attach(not paste) IMAP log for diagnosis by developers?
> https://wiki.mozilla.org/MailNews:Logging
> Win sample: SET NSPR_LOG_MODULES=timestamp,imap:5
Replace sensitive data before open log file to public, please.
See IMAP related bugs listed in dependency tree for meta bug 124287, please.

Comment 2

[craig]

The IMAP server is a current version of Dovecot, using Maildir format.

Below is an extensive log of dragging and dropping "_old/test.folder" from server1 to "misc/test.folder" on server2. The resulting folder is "misc/test/folder". I am using "/" as a directory delimiter to explain this. The bug can be seen below when TB subscribes to it.


3520[91094c0]: ImapThreadMainLoop entering [this=ac2f000]
0[3227140]: ac2f000:imap.server1.com:NA:SetupWithUrl: clearing IMAP_CONNECTION_IS_OPEN
3520[91094c0]: ac2f000:imap.server1.com:NA:ProcessCurrentURL: entering
3520[91094c0]: ac2f000:imap.server1.com:NA:ProcessCurrentURL:imap://[e-mail]:143/select%3E/_old/test.folder:  = currentUrl
3520[91094c0]: ReadNextLine [stream=a77b748 nb=63 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:NA:CreateNewLineFromSocket: * OK CommuniGate Pro IMAP Server 5.2.15 at server1.com ready

3520[91094c0]: ac2f000:imap.server1.com:NA:SendData: 1 capability

3520[91094c0]: ReadNextLine [stream=a77b748 nb=209 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:NA:CreateNewLineFromSocket: * CAPABILITY IMAP4 IMAP4REV1 ACL NAMESPACE UIDPLUS IDLE LITERAL+ QUOTA ID MULTIAPPEND LISTEXT CHILDREN BINARY ESEARCH LOGIN-REFERRALS UNSELECT SASL-IR STARTTLS AUTH=LOGIN AUTH=PLAIN AUTH=CRAM-MD5 AUTH=GSSAPI

3520[91094c0]: ReadNextLine [stream=a77b748 nb=16 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:NA:CreateNewLineFromSocket: 1 OK completed

3520[91094c0]: ac2f000:imap.server1.com:NA:SendData: 2 STARTTLS

3520[91094c0]: ReadNextLine [stream=a77b748 nb=28 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:NA:CreateNewLineFromSocket: 2 OK begin TLS negotiation

3520[91094c0]: ac2f000:imap.server1.com:NA:SendData: 3 capability

3520[91094c0]: ReadNextLine [stream=a77b748 nb=200 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:NA:CreateNewLineFromSocket: * CAPABILITY IMAP4 IMAP4REV1 ACL NAMESPACE UIDPLUS IDLE LITERAL+ QUOTA ID MULTIAPPEND LISTEXT CHILDREN BINARY ESEARCH LOGIN-REFERRALS UNSELECT SASL-IR AUTH=LOGIN AUTH=PLAIN AUTH=CRAM-MD5 AUTH=GSSAPI

3520[91094c0]: ReadNextLine [stream=a77b748 nb=16 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:NA:CreateNewLineFromSocket: 3 OK completed

3520[91094c0]: try to log in
3520[91094c0]: IMAP auth: server caps 0x14E7337, pref 0x20000, failed 0x0, avail caps 0x20000
3520[91094c0]: (GSSAPI = 0x1000000, CRAM = 0x20000, NTLM = 0x100000, MSN =  0x200000, PLAIN = 0x1000, LOGIN = 0x2, old-style IMAP login = 0x4)
3520[91094c0]: trying auth method 0x20000
3520[91094c0]: got new password
3520[91094c0]: IMAP: trying auth method 0x20000
3520[91094c0]: MD5 auth
3520[91094c0]: ac2f000:imap.server1.com:NA:SendData: 4 authenticate CRAM-MD5

3520[91094c0]: ReadNextLine [stream=a77b748 nb=48 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:NA:CreateNewLineFromSocket: + PDQ3OTY3OS4xMjg5NjYzODcxQGV2b2x1dHJhLmNvbT4=

3520[91094c0]: ac2f000:imap.server1.com:NA:SendData: Y3JhaWcgNzU3Zjg3NDg4Mzc2MGJlMGQ3NGY1ZWY5MjM5NGUxY2Q=

3520[91094c0]: ReadNextLine [stream=a77b748 nb=16 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:NA:CreateNewLineFromSocket: 4 OK completed

3520[91094c0]: login succeeded
3520[91094c0]: ac2f000:imap.server1.com:A:SendData: 5 select "_old/test.folder"

3520[91094c0]: ReadNextLine [stream=a77b748 nb=116 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:A:CreateNewLineFromSocket: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $MDNSent $Hidden $Media $Forwarded Junk $Label1 $Label2 $Label3)

3520[91094c0]: ReadNextLine [stream=a77b748 nb=138 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:A:CreateNewLineFromSocket: * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $MDNSent $Hidden $Media $Forwarded Junk $Label1 $Label2 $Label3)] limited

3520[91094c0]: ReadNextLine [stream=a77b748 nb=12 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:A:CreateNewLineFromSocket: * 0 EXISTS

3520[91094c0]: ReadNextLine [stream=a77b748 nb=12 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:A:CreateNewLineFromSocket: * 0 RECENT

3520[91094c0]: ReadNextLine [stream=a77b748 nb=37 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:A:CreateNewLineFromSocket: * OK [UIDNEXT 1] predicted next UID

3520[91094c0]: ReadNextLine [stream=a77b748 nb=41 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:A:CreateNewLineFromSocket: * OK [UIDVALIDITY 322415943] UIDs valid

3520[91094c0]: ReadNextLine [stream=a77b748 nb=36 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:A:CreateNewLineFromSocket: 5 OK [READ-WRITE] SELECT completed

3520[91094c0]: ac2f000:imap.server1.com:S-_old/test.folder:SendData: 6 myrights "_old/test.folder"

3520[91094c0]: ReadNextLine [stream=a77b748 nb=39 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:S-_old/test.folder:CreateNewLineFromSocket: * MYRIGHTS _old/test.folder lrswipcda

3520[91094c0]: ReadNextLine [stream=a77b748 nb=16 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:S-_old/test.folder:CreateNewLineFromSocket: 6 OK completed

3520[91094c0]: ac2f000:imap.server1.com:S-_old/test.folder:SendData: 7 getacl "_old/test.folder"

3520[91094c0]: ReadNextLine [stream=a77b748 nb=24 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:S-_old/test.folder:CreateNewLineFromSocket: * ACL _old/test.folder

3520[91094c0]: ReadNextLine [stream=a77b748 nb=16 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:S-_old/test.folder:CreateNewLineFromSocket: 7 OK completed

3520[91094c0]: ac2f000:imap.server1.com:S-_old/test.folder:SendData: 8 getquotaroot "_old/test.folder"

3520[91094c0]: ReadNextLine [stream=a77b748 nb=43 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:S-_old/test.folder:CreateNewLineFromSocket: * QUOTAROOT "_old/test.folder" "#Account"

3520[91094c0]: ReadNextLine [stream=a77b748 nb=52 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:S-_old/test.folder:CreateNewLineFromSocket: * QUOTA "#Account" (STORAGE 4440 9999999999999999)

3520[91094c0]: ReadNextLine [stream=a77b748 nb=16 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:S-_old/test.folder:CreateNewLineFromSocket: 8 OK completed

3520[91094c0]: ac2f000:imap.server1.com:S-_old/test.folder:SendData: 9 IDLE

3520[91094c0]: ReadNextLine [stream=a77b748 nb=10 needmore=0]
3520[91094c0]: ac2f000:imap.server1.com:S-_old/test.folder:CreateNewLineFromSocket: + idling

5716[9109100]: 9998800:mail.server2.ca:A:ProcessCurrentURL: entering
5716[9109100]: 9998800:mail.server2.ca:A:ProcessCurrentURL:imap://[e-mail]:143/ensureExists%3E.misc.test.folder:  = currentUrl
5716[9109100]: 9998800:mail.server2.ca:A:SendData: 4 list "" "misc.test.folder"

5716[9109100]: ReadNextLine [stream=965b2e8 nb=22 needmore=0]
5716[9109100]: 9998800:mail.server2.ca:A:CreateNewLineFromSocket: 4 OK List completed.

5716[9109100]: 9998800:mail.server2.ca:A:SendData: 5 create "misc.test.folder"

5716[9109100]: ReadNextLine [stream=965b2e8 nb=24 needmore=0]
5716[9109100]: 9998800:mail.server2.ca:A:CreateNewLineFromSocket: 5 OK Create completed.

5716[9109100]: 9998800:mail.server2.ca:A:SendData: 6 subscribe "misc.test.folder"

5716[9109100]: ReadNextLine [stream=965b2e8 nb=27 needmore=0]
5716[9109100]: 9998800:mail.server2.ca:A:CreateNewLineFromSocket: 6 OK Subscribe completed.

5716[9109100]: 9998800:mail.server2.ca:A:SendData: 7 list "" "misc.test.folder"

5716[9109100]: ReadNextLine [stream=965b2e8 nb=48 needmore=0]
5716[9109100]: 9998800:mail.server2.ca:A:CreateNewLineFromSocket: * LIST (\HasNoChildren) "." "misc.test.folder"

5716[9109100]: ReadNextLine [stream=965b2e8 nb=22 needmore=0]
5716[9109100]: 9998800:mail.server2.ca:A:CreateNewLineFromSocket: 7 OK List completed.

5716[9109100]: 9998800:mail.server2.ca:A:SendData: 8 logout

5592[9107300]: 8736000:mail.server2.ca:S-INBOX:SendData: DONE

5592[9107300]: 8736000:mail.server2.ca:S-INBOX:SendData: 9 close

5592[9107300]: 8736000:mail.server2.ca:S-INBOX:SendData: 10 logout

4728[9108340]: 999c800:imap.server1.com:S-INBOX:SendData: DONE

4728[9108340]: 999c800:imap.server1.com:S-INBOX:SendData: 18 close

4728[9108340]: 999c800:imap.server1.com:S-INBOX:SendData: 19 logout

4924[91080c0]: 9903800:mail.server2.ca:A:SendData: 9 logout

3520[91094c0]: ac2f000:imap.server1.com:S-_old/test.folder:SendData: DONE

3520[91094c0]: ac2f000:imap.server1.com:S-_old/test.folder:SendData: 10 close

3520[91094c0]: ac2f000:imap.server1.com:S-_old/test.folder:SendData: 11 logout

5716[9109100]: 9998800:mail.server2.ca:A:TellThreadToDie: close socket connection
5592[9107300]: 8736000:mail.server2.ca:S-INBOX:TellThreadToDie: close socket connection
5592[9107300]: ImapThreadMainLoop leaving [this=8736000]
4728[9108340]: 999c800:imap.server1.com:S-INBOX:TellThreadToDie: close socket connection
4728[9108340]: ImapThreadMainLoop leaving [this=999c800]
5716[9109100]: ImapThreadMainLoop leaving [this=9998800]
3520[91094c0]: ac2f000:imap.server1.com:S-_old/test.folder:TellThreadToDie: close socket connection
3520[91094c0]: ImapThreadMainLoop leaving [this=ac2f000]
4924[91080c0]: 9903800:mail.server2.ca:A:TellThreadToDie: close socket connection
4924[91080c0]: ImapThreadMainLoop leaving [this=9903800]

Comment 3

[WADA:World Anti-bad-Duping Agency]

My "attach(not paste)" meant "please attach data file to this bug via 'Add an attachment (proposed patch, testcase, etc.)' link of this bug", and "please never copy&paste long log file data to bug for readability of a bug report at bugzilla.mozilla.org"...

Comment 4

[WADA:World Anti-bad-Duping Agency]

Log for create.
> mail.server2.ca:A:SendData: 5 create "misc.test.folder"
> mail.server2.ca:A:CreateNewLineFromSocket: 5 OK Create completed.
Result after it.
> The resulting folder is "misc/test/folder". I am using "/" as a directory delimiter to explain this.

Probably, "delimiter at IMAP server1" is "/" and "delimiter at IMAP server2" is ".". It's returned as NAMESPACE response from IMAP server.
Please attach log file which contains both flow for NAMESPACE and flow for LIST/CREATE/SUBSCRIBE by drag&drop of a folder, in order that developers can analyze phenomenon without needless gussing on your environment.

It's rediscovery of already reported phenomenon in "drag drop of local mail folder to IMAP folder" case. If local mail folder name contains "." like abc.def, and if "delimiter at IMAP server is .", "create abc.def" is requested by Tb and folder of abc and subfolder of def under abc is created at IMAP server. It's same as phenomenon with "create request of abc.def at UI by user".

Is there any problem or any different Tb's behavior from your expectation in "drag drop of folder from server2 to server1" case? (opposite case to comment #0, copy from server of delimier="." to server of delimiter="/")

Comment 5

[craig]

Attachment: NAMESPACE log lines

Attached is part of the log showing NAMESPACE. I may have made a mistake in reporting this bug. To be clear, Server1 = CommuniGate, Server 2 = Dovecot (recent version).

Comment 6

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #5)
> NAMESPACE log lines

NAMESPACE response.
> server1.com:A:CreateNewLineFromSocket: * NAMESPACE (("" "/")) (("~" "/")) (("~public/" "/"))
> server2.ca:A:CreateNewLineFromSocket: * NAMESPACE (("" ".")) NIL NIL
It seems that my guess was correct.

> Server1 = CommuniGate, Server2 = Dovecot

If you can control Server2=Dovecot, I recommend you to use "/" as delimiter.
Coexistence of "/" and "." tends to produce problem, and problems relate to "." is many than problems relate to "/", because "/" is safer as "/" is used as delimiter of internal URL of Tb and as "/" is delimiter of file path of file system of Unix.
See bugs in dependency tree for meta bug 124287, please.

No problem when next operation in your environment?
  Drag&Drop of folder of "/" in folder name on Server2=Dovecot(delimiter=".")
  to account of Server1=CommuniGate(delimiter="/").
Or your Dovecot doesn't permit creation of folder of "/" in folder name?

Comment 7

[craig]

If I create a folder with the Thunderbird GUI named "foo/bar", it creates a folder called foo, with a subfolder called bar. This happens on either server, CommuniGate or Dovecot.

If I create a folder called "foo.bar", the Dovecot server does as above. The CommuniGate server creates a folder literally named "foo.bar".

Comment 8

[Ludovic Hirlimann [:Usul]]

So your dovecot server consider that . is a folder separator. How is it configured ?

Comment 9

[[:Aureliano Buendía]]

craig, please, could you answer to comment #8?

Comment 10

[craig]

I don't know if I fully understand the question as written. It uses a "." as the separator, even in the filesystem. For example:

/home/mail/domain.com/someuser/.attic.friends/
contains messages for the [e-mail] account, in an IMAP path called "attic/friends/"

Comment 11

[Jori Mäntysalo]

This bug does not even need two accounts. Just make local folder "foo.bar" with some messages in it and try to move it to imap server. It will create folder "foo" and then stops. TB should escape the name somehow, or at least say that it can't move the folder.