773579 - Separator in IMAP-System is used wrong (^ instead of / is used, if folder hiearchy delimiter of IMAP server is ".")

Status: NEW

(MailNews Core :: Networking: IMAP, defect)

Description

[bugzilla]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1129.0 Safari/536.11

Steps to reproduce:

I tried to create a subfolder in a folder.

foo -> bar


Actual results:

I got a new root-folder foo^bar instead of foo/bar (bar as subfolder of foo)


Expected results:

/foo/bar (foo as root-folder, bar as subfolder of foo)

Comment 1

[bugzilla]

Attachment: I started Thunderbird with NSPR_LOG_MODULES=IMAP: this is the relevant logfile-part

Since I wanted to debug the bug, I started Thunderbird with NSPR_LOG_MODULES=IMAP:5  and here is the relevant part.

Comment 2

[WADA:World Anti-bad-Duping Agency]

(In reply to bugzilla from comment #0)
> Steps to reproduce:
> I tried to create a subfolder in a folder.
> foo -> bar
> Actual results:
> I got a new root-folder foo^bar instead of foo/bar (bar as subfolder of foo)

What operation at where did you do for "create foo -> bar"?
What operation at where did you do for following logs for create/rename/movetotrash(==Delete with move to trash model)?
(a) Create root level "foo", then create subfolder "bar" under "foo"
(b) Request "foo/bar" upon new folder creation instead of "foo.bar"

According to log, when (b), Tb looks to escape "/" in your request by "^" in URL string for the create request, because folder hierarchy character of your IMAP server is "." instead of "/" in your case and because "/" is internal delimiter for Tb, and "folder of ^ in Mbox name" is normally created by your server because your server supports "^" in Mbox name.
Once "Example^Ouya" is normally created at server, the Mbox named "Example^Ouya" is valid Mbox, so nothing wrong happens, except "difference from your expectation".

Several phenomena is reported for this kind of unwanted or wrong escaping of "/" by "^", and bug 771487 comment#1 is an example.
Gmail IMAP doesn't support "^" in Mbox name, so error always occurs if "^" is wrongly used in Mbox name by Tb as seen in bug 771487 comment #1.

What string corresponds to "&AAA-" in log did you use in rename? 
See bug 771487 for problem in rename when "&" or non-7bits-ascii is used in rename of IMAP folder.

> (%2E=. %3E=> %40E=@ %5E=^)
:S-abcdef.foobar.bazasd:ProcessCurrentURL:imap://foo%40bar%2Einvalid@127.0.0.1:993/create%3E%5EExample%5EOuya:  = currentUrl
> :S-abcdef.foobar.bazasd:SendData: 1008 create "Example^Ouya"
> :S-abcdef.foobar.bazasd:SendData: 1009 subscribe "Example^Ouya"
> :S-abcdef.foobar.bazasd:SendData: 1010 list "" "Example^Ouya"
> :S-abcdef.foobar.bazasd:CreateNewLineFromSocket: * LIST (\HasNoChildren) "." "Example^Ouya"
>
:S-abcdef.foobar.bazasd:ProcessCurrentURL:imap://foo%40bar%2Einvalid@127.0.0.1:993/select%3E.Example%5E%5EOuya:  = currentUrl
> :S-abcdef.foobar.bazasd:SendData: 1012 select "Example^Ouya" (CONDSTORE)
> :S-Example^Ouya:CreateNewLineFromSocket: 1012 OK [READ-WRITE] Select completed.
>
> :S-Example^Ouya:ProcessCurrentURL:imap://foo%40bar%2Einvalid@127.0.0.1:993/rename%3E.Example%5E%5EOuya%3E.Ouya%26AAA-:  = currentUrl
> :S-Example^Ouya:SendData: 1014 lsub "" "Example^Ouya.*"
> :S-Example^Ouya:CreateNewLineFromSocket: 1014 OK Lsub completed.
> :S-Example^Ouya:SendData: 1015 close
> :S-Example^Ouya:CreateNewLineFromSocket: 1015 OK Close completed.
> :A:SendData: 1016 rename "Example^Ouya" "Ouya&AAA-"
> :A:SendData: 1017 subscribe "Ouya&AAA-"
> :A:SendData: 1018 unsubscribe "Example^Ouya"
>
> :A:ProcessCurrentURL:imap://foo%40bar%2Einvalid@127.0.0.1:993/select%3E.Ouya%26AAA-:  = currentUrl
> :A:SendData: 1019 select "Ouya&AAA-" (CONDSTORE)
>
> :S-Ouya&AAA-:ProcessCurrentURL:imap://foo%40bar%2Einvalid@127.0.0.1:993/movefolderhierarchy%3E%5EOuya%26AAA-%3E%5EExample:  = currentUrl
> :S-Ouya&AAA-:SendData: 1021 lsub "" "Ouya&AAA-.*"
> :S-Ouya&AAA-:CreateNewLineFromSocket: * LSUB () "." "Ouya"
> :S-Ouya&AAA-:CreateNewLineFromSocket: 1021 OK Lsub completed.
> :S-Ouya&AAA-:SendData: 1022 close
> :S-Ouya&AAA-:CreateNewLineFromSocket: 1022 OK Close completed.
> :A:SendData: 1023 rename "Ouya&AAA-" "Example/Ouya&AAA-"
>
> :A:ProcessCurrentURL:imap://foo%40bar%2Einvalid@127.0.0.1:993/movefolderhierarchy%3E%5EOuya%26AAA-%3E%5ETrash:  = currentUrl
> :A:SendData: 1024 lsub "" "Ouya&AAA-.*"
> :A:CreateNewLineFromSocket: * LSUB () "." "Ouya"
> :A:SendData: 1025 rename "Ouya&AAA-" "Trash.Ouya&AAA-"
>
> :A:ProcessCurrentURL:imap://foo%40bar%2Einvalid@127.0.0.1:993/create%3E%5EExample%5EOuya:  = currentUrl
> :A:SendData: 1026 create "Example^Ouya"
> :A:SendData: 1027 subscribe "Example^Ouya"
> :A:SendData: 1028 list "" "Example^Ouya"
> :A:CreateNewLineFromSocket: * LIST (\HasNoChildren) "." "Example^Ouya"

Comment 3

[WADA:World Anti-bad-Duping Agency]

FYI.
Bug 63038 is for "/" in Mbox name when delimier at server is not "/".
"Escaping of / by ^ in URL like request string of Tb" looks done by other bug, but it looks incomplete or insufficient.

Comment 4

[WADA:World Anti-bad-Duping Agency]

Attachment: IMAP log, Tb 13.0.1 on Win-XP, vmail.me, namespace="INBOX.", delimiter="."

Unable to reproduce your problem.

Log for next.
(0) vmail.me free IMAP account, Namespace="INBOX.", delimiter=".".
    Tb 13.0.1 on Win-XP SP3.
    Select Inbox (namespace folder).

(1) Request new folder of "AA" and "AA/BB" under Inbox from UI.
> :S-INBOX:ProcessCurrentURL:imap://[e-mail]:993/create%3E.INBOX.AA:  = currentUrl
> :S-INBOX:SendData: 12 create "INBOX.AA" 	
> :S-INBOX:ProcessCurrentURL:imap://[e-mail]:993/create%3E.INBOX.AA/BB:  = currentUrl	
> :S-INBOX:SendData: 16 create "INBOX.AA.BB" 	

(2) Request new folder of "CC/DD" under Inbox from UI(INBOX.CC doesn't exist).
> :S-INBOX:ProcessCurrentURL:imap://[e-mail]:993/create%3E.INBOX.CC/DD:  = currentUrl
> :S-INBOX:SendData: 20 create "INBOX.CC.DD"

(3) Request new folder of "DD^EE~FF" under Inbox from UI.
> S-INBOX:ProcessCurrentURL:imap://[e-mail]:993/create%3E.INBOX.DD%5EEE%5EFF:  = currentUrl
> :S-INBOX:SendData: 27 create "INBOX.DD/EE/FF"
> :S-INBOX:CreateNewLineFromSocket: 27 NO Invalid mailbox name: INBOX.DD/EE/FF

When NAMESPACE="INBOX.", delimier=".", "INBOX" in normally selected, and Tb 13.0.1 on Win-XP,
(i) "/" in creation request at Tb's UI is normally converted to "." upon sending IMAP create command.
(ii) "^"(==%5E) in creation request at Tb's UI is converted to "/" upon sendin create IMAP command.g

If delimiter="/", creation request of "CC^DD^EE" at Tb's UI is normally produces 'create "CC^DD^EE"' command, and,
(iii) if Gmail IMAP, create CC^DD^EE fails because "^" is not supported.
(iv)  if Yahoo! IMAP, create CC^DD^EE is successful because "^" is supported.

Comment 5

[bugzilla]

Okay, I have to clarify.

In my mind I used / as separator of mailboxes and the hierarchy. In imap-terms, the separator is ".". What I wanted was to reach a);  Create root level "foo", then create subfolder "bar" under "foo"   while foo already exists.
I already have some folders under foo (ba1, ba2, ba3...) so this problem is only on my side since some days and I disovered it now because I wanted to create bar.


And my imap-Server is on my own root-server, it is a dovecot-2.1.7 (Debian Squeeze).

Comment 6

[WADA:World Anti-bad-Duping Agency]

(In reply to bugzilla from comment #1)
> Created attachment 641816 [details]

> :S-abcdef.foobar.bazasd:ProcessCurrentURL:imap://foo%40bar%2Einvalid@127.0.0.1:993/create%3E%5EExample%5EOuya:  = currentUrl
> :S-abcdef.foobar.bazasd:SendData: 1008 create "Example^Ouya"

This log corrsponds to request of "/create>^Example^Ouya".
Did you do following?
(1) Folder of abcdef.foobar.bazasd is opened, and the abcdef.foobar.bazasd is selected at a cached connection.
(2) Click account, in orer to create root level folder and subfolder at once.
(3-a) Request folder of "/Example/Ouya" at foldr creation UI of Tb.
(3-b) Request folder of "Example/Ouya"  at foldr creation UI of Tb.

Comment 7

[bugzilla]

I did (1), then I marked the Example-Folder, went with right-click in the context-menu where I chose "create subfolder". There I choose the next name.

The result was Example^Ouya.

I repeated the same being in Example with the connection being in Example and create the subfolder. Same result.

Comment 8

[WADA:World Anti-bad-Duping Agency]

Confirming per attached IMAP log.

Comment 9

[Tore Anderson]

Attachment: imap.txt

I have a similar issue occur sometimes. My IMAP server is Cyrus using the / as the hierarchy separator. However for some reason Thunderbird occasionally attempts to use ^ instead, which fails as a "mailbox does not exist" UI error message shown to the user.

As you can see in the attached IMAP trace (extracted by Wireshark), Thunderbird tries to do:

10 STATUS "INBOX^Templates" (UIDNEXT MESSAGES UNSEEN RECENT)

...and that's the only place where the character "^" occurs in the entire trace. The correct "INBOX/Templates" are mentioned several places though, so why I don't see why any external reason why it should make this mistake. It can happen with any folder, not necessarily the Templates one, and I don't see any pattern. If I click on the Templates folder after having received the error message, it generally works and I can access the messages contained within just fine.

Tore

Comment 10

[wertstoffe]

I have a a similar Problem. First with cyrus2.2, now with dovecot. 

Always on startup from thunderbird/icedove i get this error-message with the last opened folder. 
Sometimes, the wrong folder is really created... 

But it only happenes on icedove/tunderbird-startup.

jakob

Comment 11

[Dave Wreski]

I've also encountered this problem, and the fix for me was to modify my dovecot config to include 

namespace inbox {
  separator = "$"
  ...
}

More info here

http://wiki2.dovecot.org/MboxChildFolders

Comment 12

[Ben Campbell]

I've just encountered this bug, or something similar.
I set up an IMAP server with "^" as the separator, and a folder named "a^b".
Once I subscribed to "a^b", it appeared in the left-hand folder tree pane in the GUI with the name "a/b".
(It did appear as "a^b" in the folder subscription window).

The IMAP LIST gave:

3 list "" "*"
* LIST () "^" "a^b"
* LIST () "^" "foo\\bar"
* LIST () "^" "wibble"
* LIST () "^" INBOX
3 OK LIST completed

(the "foo\bar" was defined as foo\bar on the server, and appears as foo\bar in the left-hand folder tree pane, so all seems fine there.).

If I change the separator to "Z" on the server, ie:

14 list "" "*"
* LIST () "Z" "Trash"
* LIST () "Z" INBOX
* LIST () "Z" "a^b"
* LIST () "Z" "foo\\bar"
* LIST () "Z" "wibble"
14 OK LIST completed

Then "a^b" appears in the folder subscription dialog as "a^^b" and it appears in the left-hand folder as "a^b", but with a trailing unicode "missing-character" box (containing four zeros, so presumably representing a nul).