Closed Bug 56489 Opened 24 years ago Closed 24 years ago

IMAP backend code does not parse folders using hierarchy delimiter

Categories

(MailNews Core :: Networking: IMAP, defect, P3)

Product:
MailNews Core
Component:
Networking: IMAP
Platform:
x86
Windows NT
Type:
defect

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: wjamrog, Assigned: Bienvenu)

Details

Attachments

(3 files)

Attach the IMAP log I got for Lotus Domino IMAP server
9.82 KB, text/plain
Details
Attached the screen shot, notice "Jeff/Netscape, abc/def, karen/test1" folders
78.46 KB, image/gif
Details
Attached screen shot: notice the subscribed status of "Jeff/Netscape,abc/def,karen/test1"
37.18 KB, image/gif
Details 
We found a serious bug that make Lotus Domino IMAP server's NAMESPACE feature 
unusable.


>         Our hierarchy delimiter is backward slash.  When a folder name
> contains forward slash, you treat it as it has subfolder.  From that
> point, you can't select the folder and so on.  From the log below, I have
> a folder named "Folder/Iris".  You can't select that folder because your
> client think Iris is a subfolder.  This is a very serious bug against our
> server.  
>
> [0103:0004-00E7] S:  2 OK AUTHENTICATE completed<CRLF>
>
> [0103:0004-00E7] R:  3 list "" "%"
> [0103:0004-00E7] S:  * LIST () "\\" Drafts<CRLF>
> [0103:0004-00E7] S:  * LIST () "\\" Folder/Iris<CRLF>
> [0103:0004-00E7] S:  * LIST () "\\" "Group Calendars"<CRLF>
> [0103:0004-00E7] S:  * LIST (\Noinferiors) "\\" INBOX<CRLF>
> [0103:0004-00E7] S:  * LIST () "\\" Inbox.Peaches<CRLF>
> [0103:0004-00E7] S:  * LIST () "\\" "Sent Items"<CRLF>
> [0103:0004-00E7] S:  * LIST () "\\" SentMail<CRLF>
> [0103:0004-00E7] S:  * LIST () "\\" SentMail.Broccoli<CRLF>
> [0103:0004-00E7] S:  * LIST () "\\" SentMail.Spinach<CRLF>
> [0103:0004-00E7] S:  * LIST () "\\" Test<CRLF>
> [0103:0004-00E7] S:  * LIST () "\\" Trash<CRLF>
> [0103:0004-00E7] S:  * LIST () "\\" Untitled<CRLF>
> [0103:0004-00E7] S:  3 OK LIST completed<CRLF>
>
> [0103:0004-00E7] R:  4 list "" "%\\%"
> [0103:0004-00E7] S:  * LIST () "\\" {13}<CRLF>
> [0103:0004-00E7] S:  SentMail\sub1<CRLF>
> [0103:0004-00E7] S:  4 OK LIST completed<CRLF>
>
> [0103:0004-00E7] R:  5 list "" "INBOX"
> [0103:0004-00E7] S:  * LIST (\Noinferiors) "\\" INBOX<CRLF>
> [0103:0004-00E7] S:  5 OK LIST completed<CRLF>
>
> [0103:0004-00E7] R:  6 select "INBOX"
> [0103:0004-00E7] S:  * 13 EXISTS<CRLF>
> [0103:0004-00E7] S:  * 0 RECENT<CRLF>
> [0103:0004-00E7] S:  * OK [UNSEEN 1] Message 1 is first unseen<CRLF>
> [0103:0004-00E7] S:  * OK [UIDVALIDITY 1] UIDs valid<CRLF>
> [0103:0004-00E7] S:  * FLAGS (\Flagged \Seen \Answered \Deleted
> \Draft)<CRLF>
> [0103:0004-00E7] S:  * OK [PERMANENTFLAGS (\Flagged \Seen \Answered
> \Deleted \Draft)] Permanent flags<CRLF>
> [0103:0004-00E7] S:  6 OK [READ-WRITE] SELECT completed<CRLF>
>
> [0103:0004-00E7] R:  7 UID fetch 1:* (FLAGS)
> [0103:0004-00E7] S:  * 1 FETCH (FLAGS () UID 18)<CRLF>
> [0103:0004-00E7] S:  * 2 FETCH (FLAGS () UID 19)<CRLF>
> [0103:0004-00E7] S:  * 3 FETCH (FLAGS () UID 20)<CRLF>
> [0103:0004-00E7] S:  * 4 FETCH (FLAGS () UID 21)<CRLF>
> [0103:0004-00E7] S:  * 5 FETCH (FLAGS () UID 22)<CRLF>
> [0103:0004-00E7] S:  * 6 FETCH (FLAGS () UID 23)<CRLF>
> [0103:0004-00E7] S:  * 7 FETCH (FLAGS (\Seen) UID 24)<CRLF>
> [0103:0004-00E7] S:  * 8 FETCH (FLAGS () UID 25)<CRLF>
> [0103:0004-00E7] S:  * 9 FETCH (FLAGS () UID 26)<CRLF>
> [0103:0004-00E7] S:  * 10 FETCH (FLAGS (\Seen) UID 27)<CRLF>
> [0103:0004-00E7] S:  * 11 FETCH (FLAGS (\Seen) UID 28)<CRLF>
> [0103:0004-00E7] S:  * 12 FETCH (FLAGS (\Seen) UID 29)<CRLF>
> [0103:0004-00E7] S:  * 13 FETCH (FLAGS (\Seen) UID 30)<CRLF>
> [0103:0004-00E7] S:  7 OK FETCH completed<CRLF>
>
> [0103:0005-0129] S:  1 OK AUTHENTICATE completed<CRLF>
>
> [0103:0005-0129] R:  2 list "" "Folder"
> [0103:0005-0129] S:  2 OK LIST completed<CRLF>
>
> [0103:0005-0129] R:  3 list "" "Folder\\%\\%"
> [0103:0005-0129] S:  3 OK LIST completed<CRLF>
>
> [0103:0005-0129] R:  4 select "Folder\\Iris"
> [0103:0005-0129] S:  4 NO SELECT failure, cannot select mailbox: Mailbox
> does not exist<CRLF>
>
> [0103:0004-00E7] R:  xxxx logout
load balancing to david b, cc'ing jefft.

Unfortunately we are down to rtm stopper bugs right now so we may not be able to
get to this until the next release......
Assignee: mscott → bienvenu
QA Contact: esther → huang
Wei-Lee, do you have test account available for external people to use?
Can you provide the test account info here for us to track this problem? 
I need the server name, user/account name and the password. Thanks.
OK, I understand that.  But please give this bug a higher priority for your next 
release.  Honoring server's hierarchy delimiter is one of the basic rules in the 
RFC.  Our new NAMESPACE feature is totally broken because of this bug.  I talked 
to Jeff about this bug long time ago.  I believe he understand what's involved 
to fix this bug.

server name:imap2.iris.com (or IP: 198.112.211.76)
account: netscape6
password: netscap

Let me know if you have problem acsess this account.

Thanks
Wei-Lee
Sorry, the password is netscape.
Confirming.

Gerv
Status: UNCONFIRMED → NEW
Ever confirmed: true
accepting
Status: NEW → ASSIGNED
Also, in the compose window if you save the message as Drafts, nothing goes to 
the server.  Message is not saved and you have to kill the window.  I think this 
is also because our hierarchy delimiter is backward slash.
I cleaned up a bunch of hierarchy delimiter problems yesterday. I'll try against
your server again with my changes. I have not checked in my fixes to the trunk
yet because I have not been able to get permission. When I check in, you might
try downloading a Mozilla daily build and trying it. It will not make Netscape
6.0, however.
There is a folder named "Jeff/Netscape" in your mailfile that you can test.
Please notice the final two lines:
------------------------------------------------------------------------------

189[33d86c0]: 198.112.211.76:S-folder1\folder2:SendData: 4 select "abc\\def"

189[33d86c0]: 198.112.211.76:S-abc\def:CreateNewLineFromSocket: 4 NO SELECT 
failure, cannot select mailbox: Folder not found in IMAP name space
--------------------------------------------------------------------------------
The folder name with "/" problem is not only occur on this specific server.
There was bug 29926 logged for "IMAP Folders' names with "/" problems for 
different platforms".
David, sounds like we need to distinguish in between the canonical delimiter "/"
vs the real "/" character embedded in the folder name. I am thinking may be we
need to find a way to quote the "/", e.g. folder name "ab/cd" becomes "ab%2Fcd"
internally.
I'm not convinced, Jeff, that we have to escape the '/'. I fixed the parsing of
the online name so that we look for the hierarchy delimiter, not '/' when
parsing out sub-folders (in that Cyrus bug I was working on). If the hiearchy
delimiter is '/', then we're fine. In other words, I no longer look for '/'.
This is all part of honoring the hierarchy delimiter. When I get a chance, I'll
try this test account.
the server seems to be down - I can't connect to it, nor can I ping the ip
address. Is this server info still correct?
I can connect well by using the ip address.
But, there was no existing folder(Jeff/Netscape)as Wei-Lee mentioned there.
I just create other folders for testing and it's still working now.....
Hmm....weird. 
I have the same results as David has when I ping the ip address. 
And I just found out that I can create the folders there but I couldn't receive 
sent messages from the Inbox by myself!! 

For security reason, the SMTP on my server is not setup to allow you send 
message.  If you really needs it, I might be able to ask our IS department to 
change it.  Let me know.
I double check the mail file and it does have a folder named Jeff/Netscape.  
Maybe you need to recreate your account to get the latest folders list.  But 
with your own created folder, you can see the problem anyway.
we can't find the folder because the test account is not subscribed to
Jeff/Netscape - I've fixed that.
I have a fix for this - when I get it reviewed, I'll check it in. Also, changing
summary - The LIST command is not the problem.
Summary: LIST doesn't honor server's hierarchy delimiter → IMAP backend code does not parse folders using hierarchy delimiter
Can you also try save a Drafts message against our server?

Thanks for looking into this problem.
Wei-Lee
I believe this is fixed - try downloading the next mozilla (not Netscape) build
that comes out tomorrow and let me know if it works for you.
Status: ASSIGNED → RESOLVED
Closed: 24 years ago
Resolution: --- → FIXED
Sorry that takes me so long to verify this.  Struggling with Netscape 6 for a 
long time....

Part of the problem is fixed.
The folder panel displays correctly now.  (i.e., it doesn't show subfolder)  But 
you can't SELECT the folder.

Based on my server log, I found out you selected a folder that DO contains 
subfolder, "test 123\\sub of test123".  You need to select a folder that 
contains "/" as part of the name.  There are 2 folders like that in your mail 
file.  One is "Jeff/Netscape".  Another one is "abc/def".  Those folders can't 
be selected because your client issue SELECT "Jeff\\Netscape".
yes, that's a separate bug, bug 60197, for future reference.
Thank you.  I'm tracking that bug now.

Wei-Lee
this bug is about not using the server-specified hierarchy seperator, no matter
what it is ('.', '\', '/', etc). bug 60197 is about treating '/' as if it's the
server-specified hierarchy seperator (because we transform mailbox names into an
internal "canonical" format with '/' as the hierarchy separator, but don't do
anything about the case where the mailbox name already contains '/' that *is
not* meant as a hierarchy delimiter. That's what your screen shot is capturing,
and is really bug 60197.
From above two screen shots, I have questions:
1) Why above folders display different on the folder panel after the second 
launch? (e.g. "Jeff/Netscape" folder: not grey out for Jeff, "abc/def" & 
"karen/test1" folder: grey out for abc & karen)
2) From the subscribe dialog, why "Jeff/Netscape" folder: subscribed on Jeff, 
"abc/def" & "karen/test1" folder: not subscribe on abc & karen? Can David answer 
those questions? And I need to know which part I need to verify for this bug? 
Thanks.
I don't know what to tell you, Karen, other than that our handling of folder
names with '/' in them where '/' is not the hierarchy delimiter is completely
broken. It won't have any chance of working until bug 60197 is fixed.
OK. Thanks David.
I am closing this bug based on my verification and David comments. 
The rest problems had been handled on bug 60197 (probably on bug 29926 too)
Marking as verified.
Status: RESOLVED → VERIFIED
Product: MailNews → Core
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: