Closed Bug 154778 Opened 22 years ago Closed 21 years ago

Can't access the content of Courier IMAP server after setting 'IMAP server directory' to "INBOX."

Categories

(MailNews Core :: Networking: IMAP, defect)

x86
All
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: usselmann.m, Assigned: Henry.Jia)

References

(Blocks 2 open bugs)

Details

Attachments

(5 files, 6 obsolete files)

I have the following two problems with my Courier IMAP Server. The issues are
reproducable with Win NT and Linux. Tested with Mozilla Release 1.0 and Mozilla
1.1a+ Gecko/20020626. The problems do not occur with Sylpheed on Linux and
Pegasus Mail on NT:

1. With nested subfolders not all levels are shown, only two.

E.g. in the following example subfolder 3-1 and 3-2 do not appear:

inbox 
 |
  -- subfolder 1
     |
      -- subfolder 2
         |
          -- subfolder 3-1
         |
          -- subfolder 3-2

 

2. When I use the IMAP server directory "INBOX." the folders are no longer
subfolders of the inbox, which is fine, but the content is not visible. Error
message: "The current command did not succeed. The mail server responded:
Invalid Mailbox."

When I leave the server directory empty the subfolders are shown below the inbox
but the content is visible.

Issue 1 happens in both cases.
The Courier IMAP server is running on FreeBSD 4.5 and compiled with

     --enable-workarounds-for-imap-client-bugs
QA Contact: huang → meehansqa
Blocks: 160644
With Gecko/20020809 the issue with the missing subfolders seems to have improved
a little bit: 

The 2. level of subfolders still do not appear automatically but when I manually
add .sbd subdirectories for the missing subfolders to the folder tree in the
cache the subfolders then also appear in the folder tree of the mail window.
The subfolders are gone again after closing and reopening Mozilla and this time
even creating the subfolders in the cache does not work at all. :-(
Confirm issue 2. Issue 1 doesn't exist, even the message in the nested subfolder
is shown. (comment #0) Please verify.

Marking new and taking. Add mscott to CC list.
Assignee: mscott → Henry.Jia
Status: UNCONFIRMED → NEW
Ever confirmed: true
Henry,

> Issue 1 doesn't exist

Well, for me it does. I am attaching two sreenshots, one accessing the IMAP
server with Mozilla and one with Pegasus Mail. Compare the subfolders of folder
'Mailinglisten'. 'cvs', 'FreeBSD' and 'OpenOffice' contain subfolders which do
not appear with Mozilla.

Something else: I just noticed that creation of folders and subfolders does not
work as well. Nothing happens, not even an error message. Is this a known
issue?

Manfred
Manfred,

I'll take more look at this bug. Thx for the feedback.

Henry
Manfred, several things. First of all, you should update to the latest trunk
build if you want to have the latest Courier fixes. Second, are you subscribed
to all those sub-folders? I'm not sure if Pegasus shows you all folders or just
subscribed folders. If you get the latest trunk mozilla build, and bring up the
imap subscribe UI, you should be able to see if you're subscribed to those
sub-folders or not (you need the latest trunk build for that subscribe UI to
work, unfortunately). Or, you could go into the advanced imap server settings
and uncheck the box that says "show only subscribed folders".

If those things don't clear up the problems, an IMAP protocol log might help.
Here are instructions for generating a log:

http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap
bienvenu,

> First of all, you should update to the latest trunk
> build if you want to have the latest Courier fixes.

Do I have the latest trunk, when I use the version from
http://ftp.mozilla.org/pub/mozilla/nightly/latest/?

Every day before testing I install mozilla-win32-installer-sea.exe from above
location.

It this is not the right version how do I get the latest trunk? Do I have to
compile it myself?

> Second, are you subscribed to all those sub-folders?

I do not use the subcribe feature currently.

> If you get the latest trunk mozilla build, and bring up the
> imap subscribe UI, you should be able to see if you're subscribed to those
> sub-folders or not (you need the latest trunk build for that subscribe UI to
> work, unfortunately). 

It is not working here: The folder tree is mostly invisible. Only some + and
lines at the left can be seen. It does not change anything if I  resize tbe
dialogue to make it bigger.

> Or, you could go into the advanced imap server settings
> and uncheck the box that says "show only subscribed folders".

Since I want to see all folders I have it unchecked all the time.

> If those things don't clear up the problems, an IMAP protocol log might help.

Ok, I will switch on the logging.

Manfred
The attached log should contain the refresh of the folder tree because I
deleted the content of C:\Program
Files\mozilla.org\Mozilla\Profiles\Manfred\8c5vhg16.slt\ImapMail\localhost
before starting Mozilla and opening the mail window.

The updated mail folder tree display still showed no subfolders for
Mailinglisten.cvs, Mailinglisten.FreeBSD and Mailinglisten.OpenOffice (The
existing subfolders can be seen on attachment 95088 [details].
Manfred Usselmann,

I can't read your log. I can't recongnize the format. Would you please supply
another one using text/plain format? Thx.

If the issue you described does exist, I think we may file another bug. Because
issue 1 and issue 2 (comment 0) are not the same and don't have much relationship.

Henry
Henry,

the attachment is a zip file.Sorry, I thought adding an attachment would
preserve the file name. Maybe I should have used another mime type (which?). But
I just tested downloading the attachment as log.zip and it worked fine. Could
you please try again?
I agree that it would have been better if I would have created two separate
issues. If you would want to split feel free to do so. Or shall I do it? But
isn't issue 2 (comment 0) already covered by other issues any way?

Thanks,
Manfred
Attachment #95117 - Attachment mime type: application/octet-stream → application/zip
Attachment #95117 - Attachment mime type: application/zip → application/x-zip-compressed
Manfred,

I've changed the mime type of your attachment. :-)

I'll take care to file a bug for issue 2 if there is not one.

Henry
Manfred,

I've confirmed your another issue and filed bug 162644. This issue exists not
only on courier imap.

Change the summary of this bug.

Henry
Summary: IMAP Courier Folder Problems → Can't access the content of Courier IMAP server after setting 'IMAP server directory' to "INBOX."
the problem is, from what I can tell in the log, that we're not listing
sub-folders when you expand a parent folder in the folder pane. That means that
the you more or less need to use imap subscription if you have nested
sub-folders. I suspect this has been broken for a while, if not forever in
mozilla. The fix would be to make sure we list a folder's children and its
children's children when you expand a folder in the folder pane, If and only if
we're not using IMAP subscription.
this bug is not related to 160644 - it's just a general problem with the "don't
use subscription" feature.
No longer blocks: 160644
David,

For the nested sub-folders showing issue, I filed another bug 162644. It's not a
namespace and imap server directory related bug. I also agree with what you said
about the reason. I think we can also simplify the process by list all the
nested sub-folders once. I mean we can use 'list "" "*"' instead of 'list "" "%"
+ list "" "%.%"'. Then we need not care about the issue later. I gave out a
patch for this issue in bug 162644. Would you please take a look at that and if
possible, r=/sr=? Thx.

This bug (bug 154778) is related to imap server directory. Because after setting
the imap server directory 'INBOX.', you will not get the namespace for the
normal folders parallelly with 'INBOX', thus you will not add the imap server
directory for converting conanical folder to server side folder. This is the
reason for this bug. I'll give out a patch soon.

P.S. Would you please sr= bug 161085, bug 137489 and bug 147995 which has passed
r=navin? Thx. They are all related to namespace and imap server directory.

Regards.

Henry
Blocks: 160644
Henry, it's not a namespace issue or a server directory issue. The issue is that
we're not listing sub-folders more than two levels deep for any kinds of
folders. Listing * is not a good solution - that can be very slow.
Yes, the sub-folders issue is not a namespace related issue. See bug 162644.
This bug 154778 is. See the second part of Comment #17.
Henry, How can we get the namespace for the paralled folders of 'INBOX' when
imap server directory is set 'INBOX' (or 'INBOX.a') or etc. ? I only add
'INBOX.' or etc.(which is called onlineDir) when
nsImapProtocol::CreateEscapedMailboxName returned . And thus get a little success .
This new method of nsImapProtocol.cpp can let us access those folders parolled
with INBOX when IMAP server directory is set 'INBOX'. but mabey it is not a
best resolution . I'm working on a new and better one.
Comment on attachment 104620 [details] [diff] [review]
Change to nsImapProtocol::CreateEscapedMailboxName

Index: nsImapProtocol.cpp
===================================================================
RCS file: /cvsroot/mozilla/mailnews/imap/src/nsImapProtocol.cpp,v
retrieving revision 1.439
diff -u -r1.439 nsImapProtocol.cpp
--- nsImapProtocol.cpp	28 Aug 2002 06:29:15 -0000	1.439
+++ nsImapProtocol.cpp	11 Nov 2002 08:55:40 -0000
@@ -2540,7 +2540,55 @@
 // escape any backslashes or quotes.  Backslashes are used a lot with our NT
server
 char *nsImapProtocol::CreateEscapedMailboxName(const char *rawName)
 {
-  nsCString escapedName(rawName);
+  //get onlineDir here
+  nsString aString;
+  nsCAutoString onlineDir;
+  m_hostSessionList->GetOnlineDirForHost(GetImapServerKey(), aString); 
+  onlineDir = (char *)(!aString.IsEmpty() ? ToNewCString(aString) : nsnull);
+  char delimiterToUse, *onlineDelimiter;
+  //m_runningUrl->GetOnlineSubDirSeparator(&onlineDelimiter);
+  if (m_imapMailFolderSink)
+    m_imapMailFolderSink->GetOnlineDelimiter(&onlineDelimiter);
+  delimiterToUse=*onlineDelimiter;
+  //delimiterToUse='.';//???
+  //get onlineDir ok!
+  //create the correct string here!
+  nsCString escapedName("");//ok?
+  
+  int len = onlineDir.Length();
+  if(PL_strncmp(onlineDir.get(),rawName,len))
+  {
+  //we should get namespaces and compare rawName with namespace prefix
+  if(m_hostSessionList)
+  {
+  PRUint32 count=0;
+  m_hostSessionList->GetNumberOfNamespacesForHost(GetImapServerKey(),count);
+  if(count)
+  {
+  for(PRUint32 i=0;i<count;i++)
+  {
+    nsIMAPNamespace * ns=nsnull;
+    m_hostSessionList->GetNamespaceNumberForHost(GetImapServerKey(),i,ns);
+    if(ns)
+    {
+      const char* prefix=ns->GetPrefix();
+      //compare
+      if(PL_strncmp("INBOX",prefix,5))
+      //if prefix != 'INBOX'
+	 goto done;
+    }
+  }//end of for
+  }//end of if(count)
+  }//end of if(m_hostSessionList)	
+
+      
+      onlineDir.ReplaceChar('/',delimiterToUse);
+      if(onlineDir.Last()!=delimiterToUse)
+      onlineDir+=delimiterToUse;
+      escapedName.Append(onlineDir.get());
+  } 
+done:
+  escapedName.Append(rawName);  

   for (PRInt32 strIndex = 0; *rawName; strIndex++)
   {
1.The attachment 104620 [details] [diff] [review] is not good.It only works on some very special conditions.

2.in Courire server, we will create a wrong dommand when we select a folder. (If
a folder 'INBOX.a', and IMAP server directory is set 'INBOX', in Courier server
, mozilla will send 'select a' instead of 'select INBOX.a' when we select it.)


3.There are these attributes included in nsImapMailFolder.
PRUnichar m_hierarchyDelimiter;
PRInt32 m_boxFlags;
nsCString m_onlineFolderName;
nsFileSpec *m_pathName;
nsCString m_ownerUserName;  // username of the "other user," as in
                            // "Other Users' Mailboxes"

nsCString m_adminUrl;   // url to run to set admin privileges for this folder
nsIMAPNamespace  *m_namespace;    // Opaque pointer to the IMAP namespace for
this folder
                                          // Use libnet accessors for various
namespace functionality

For this bug, mozilla will get wrong onlineFolderName, so we can't send correct
command to server.And mozilla will create folder object and get hostname and
other info to create a urlSpec whenever needed. And never save it.

4.Can we add an attribute m_urlSpec in nsImapMailFolder to save the correct
urlSpec which will include hostname ,...,onlinePathName,namespacePrefix all
these informations.

5.I'm not sure if this can work. And I know the question is that we can't save
the object for later use in mozilla. That means we can get correct urlSpec in
the beginning we lsub all folders in server, but we can't keep urlSpec for later
use.But if we can't get correct onlineFolderName , how will this bug be fixed?:)

That's all.:-)
Comment on attachment 104620 [details] [diff] [review]
Change to nsImapProtocol::CreateEscapedMailboxName

1. use diff -u instead of the code itself
2. nsImapProtocol::CreateEscapedMailboxName is not a good place. rawName should
be right before this funciton is called. You may refer to the code for Cyrus
server handling.
3. don't add any new variables, it should be resolved without any new
footprint.

Thx, Philip. Keep on going.
Attachment #104620 - Flags: review-
Comment on attachment 104620 [details] [diff] [review]
Change to nsImapProtocol::CreateEscapedMailboxName

1. use diff -u instead of the code itself
2. nsImapProtocol::CreateEscapedMailboxName is not a good place. rawName should
be right before this funciton is called. You may refer to the code for Cyrus
server handling.
3. don't add any new variables, it should be resolved without any new
footprint.

Thx, Philip. Keep on going.
Comment on attachment 104620 [details] [diff] [review]
Change to nsImapProtocol::CreateEscapedMailboxName

Sorry for the spam. Not sure why there is two same flags.
This bug will also has a perfomance like this:
1.Clear the IMAP server directory setting.
2.create a folder Inbox.Inbox
3.we will see Inbox.Inbox.Inbox folder hierachy now.Save a message AAA to
Inbox.Inbox.Inbox 
4.now set the IMAP server directory ot 'INBOX'
5.refresh the folder pane
6.we will see just Inbox.Inbox folder now
7.select the Inbox.Inbox folder
Result:
  1.we can select it
  2.But the message we get is not AAA,but the message originally exits in Inbox! 
>  2.But the message we get is not AAA,but the message originally exits in Inbox! 

should be :
   2.But the message we get here is not AAA. The message we get here will be
those messages exists originally in Inbox.That means we can't access the content
in Inbox.Inbox.Inbox.
Attached patch add a patch (obsolete) — Splinter Review
Add comment to attachment (id=106487) and this bug below.

1.This bug cause serious error when user configured the IMAP server directory.

2.On UW server and iPlanet server, we prefer the user not set IMAP directory. 
  When the IMAP directory is set on these 2 kinds of server, dispaly will be
wrong( user will can't see those folders parelled with INBOX originally). 
  This is because those Drafts Templates and other many folders we created are
parelled with INBOX first. The setting of IMAP directory will make them dispear.

3.On Courier server and Cyrus server ,user can set IMAP server directory in some
cases.

4.On Cyrus server, the setting of IMAP server directory to 'INBOX' will not
cause error because original code add onlineDir before the foldername we
selected (or used in other case).

5.On Courier server, this bug exists because the need-added onlineDir in not add
before foldername we selected.

6.attachment (id=106487) resolute this question. It add onlienDir before
foldername when necessary.

7.And the test result of this patch is:
  7.1 Whatever IMAP server directory is set 'INBOX' or something like 'INBOX.a'
it all works.
  7.2 If we set PersonalNamespace (or other kinds of namespace) manuelly and
cleared the override_namespace flag , it works well.
  7.3 If we don't cleare the override_namespace flag , it works well.

8.Shortcoming of this patch:
  It doesn't relolute the question in comment #28 and comment #29.

9.While I think, the question in comment #28 and comment #29 will not have a
good resolute if we don't know the exact hirachy level of a folder.
I also think,mabey we shouldn't use 
   rv = hostSessionList->  
GetNamespaceForMailboxForHost(serverKey,onlineMailboxName,ns); 
at all.

For you see, if the prefix of onlineMailboxName is the same with the prefix of a
special namespace (whatever 'Inbox.' or 'user.' or 'shared.'), the code will
cause error and the result will be we still can't access the content of the
folder we selected.

And this is also the reason of question in comment #28 and comment #29.
Comment on attachment 106487 [details] [diff] [review]
add a patch

> When the IMAP directory is set on these 2 kinds of server, dispaly will be
> wrong( user will can't see those folders parelled with INBOX originally).
We can't think the display is wrong. This is because you want to omit the
online part of you directories showed.

After you code goes to
+addonlinedir:
and ns is null, mozilla will crash.
Attachment #106487 - Flags: review-
Attached patch add if(ns) on patch 106487 (obsolete) — Splinter Review
I add 'if(ns)' before 'goto'. This may avoid mozilla's crash when ns is NIL.

A question,henry.

If the folder hierachy is INBOX.a.b.c, how can we know INBOX is at the level 0,
and folder 'a' is at level 1, and 'b' at level 2, 'c' at level 3 (whatever we
set 'INBOX' as the IMAP server directory, they are all should be at this
level)?
Comment on attachment 106552 [details] [diff] [review]
add if(ns) on patch 106487

You shouldn't only get the personal namespace.

The work around for this bug is to add a "" namesapce and uncheck 'Allow server
to override these namespaces' in the 'Advanced IMAP Server Settings'.
This patch considered other user's namespace and public namespace.

We need only the delimiter of certern namespace we get(since there may be
different delimiter for different namespace).
Attachment #104620 - Attachment is obsolete: true
Attachment #106487 - Attachment is obsolete: true
Attachment #106552 - Attachment is obsolete: true
Comment on attachment 107465 [details] [diff] [review]
new patch for test

What do you think if the online directory is set to 'INBOX' instead of
'INBOX.'?

Also, no need to 
+		  if(ns)
+		    rv
=hostSessionList->GetDefaultNamespaceOfTypeForHost(serverKey,
+									 
ns->GetType(),
+									  ns);
Hi, henry,

Thx you!

1.
>  What do you think if the online directory is set to 'INBOX' instead of
>  'INBOX.'?

   I think both 'INBOX' and 'INBOX.' will let us find the namespace because of
these lines in nsIMAPNamespace::MailboxMatchesNamespace.

>   	if (PL_strstr(boxname, m_prefix) == boxname)
>		return PL_strlen(m_prefix);
>
>	// If the boxname is part of the prefix
>	// (Used for matching Personal mailbox with Personal/ namespace, etc.)
>	if (PL_strstr(m_prefix, boxname) == m_prefix)
>		return PL_strlen(boxname);

2.
>Also, no need to 
>+		  if(ns)
>+		    rv
>=hostSessionList->GetDefaultNamespaceOfTypeForHost(serverKey,
>+		                                    ns->GetType(),
>+					            ns);

    I think we may need them because what the 'ns' we get using
nsIMAPNamespaceList:: GetNamespaceForMailbox is the last one who matchs the
mailbox name in namespace list. While the 'ns' we get using
nsIMAPNamespaceList::GetDefaultNamespaceOfType is the first one who matchs the
mailbox name in namespace list. And I think the first to be used as default is
better.

philip
In our firstbase server , namespace is :
    a namespace
    * NAMESPACE (("" "/")) (("Shared Folders/User/" "/")) NIL
    a OK Completed

The patch is ok because of the ("" "/") namespace. 

If the namespaces is like
    * NAMESPACE (("INBOX/" "/")) (("Shared Folders/User/" "/")) NIL
and the onlineDir is set 'Shared Folders' or 'Shared Folders/Users',the patch is
also ok.
This patch disable user's configure to IMAP server directory 'INBOX' or
'INBOX.' etc. when server has a personal namespace 'INBOX.'.

Thus avoid many problems on IMAP server directory setting.

And I think we also need attachment 107465 [details] [diff] [review] because we still keeps IMAP server
setting now. And user still can configure personal namespace and uncheck
override namespace, then configure IMAP server directory again.

Thx for David's help much. :-)

Pls test the patch. We may need some more talk about the architechture.
Instead of comparing both the personal namespace and the online server directory
to inbox, I suggest comparing them to each other, and not special casing INBOX
like this.

And instead of keeping  attachment 107465 [details] [diff] [review], I think we should disable the IMAP
Advanced Server Settings UI to prevent the user from doing what you describe.

Finally, one user who had this problem told me that they set the imap server
directory so that sub-folders of the INBOX will appear indented under the INBOX
as sub-folders, and that this doesn't happen with the personal namespace
approach. So we might want to change that - though I could have sworn I've had
people complain the other way too - what do you think?
Hi,David,
  I will add a new patch to let personal namespace and onlineDir to compare to 
each other soon. And if there are more than 1 personal namespace on the server, 
I will make onlineDir to compare to the defaut one(ie.the first one in the 
namespaceList). If you don't think so, pls let me know.
  And to attachment 107465 [details] [diff] [review], I agree to delete IMAP server directory and not use 
the attachment. And at that time, we may rewrite the method 
AddOnlineDirIfNecessary or delete it.
  Thirdly, I know that IMAP server directory is working now as the filter of 
folder names. If we set it, mozilla will create a different folder struct in 
the client side. The difference of the folder hierarchy in client side and 
server side gives us much trouble. That's why we delete IMAP server directory.
  And if we make personal namespace setting work as IMAP server directory, the 
same question will be still there. So I don't think we need let personal 
namespace setting to act as IMAP server directory. If you don't think so, pls 
tell me too.
philip
Phillip, I think what you're proposing sounds OK, though I'll have to see the
patch to be sure that I'm understanding you correctly.

Also, bear in mind, the biggest reason we have trouble when the user sets the
imap server directory to INBOX. on a server that returns INBOX. as the personal
namespace is that it's simply wrong :-) 
Hi,David.Sorry for the delay. :-) This patch has changed as your suggestion. It
compare onlineDir with default personal namespace and disabled user's configure
to 'INBOX' when server has a personal namespace 'INBOX'. In other conditions,
for exam, on Cyrus server when we set IMAP server directory to INBOX and server
has a personal namespace "", IMAP server direcotry still works to let INBOX's
subfolder listed as parelled with INBOX. Pls see if this patch is ok. Thx. :-)
Attachment #109005 - Attachment is obsolete: true
Add a check if the namespace is null.
Pls see it, David. thx.
Attachment #110053 - Attachment is obsolete: true
How about set server directory directly to "INBOX" by ignoring case in 
nsImapIncomingServer::SetServerDirectory if we really need to fix this bug ?
> How about set server directory directly to "INBOX" by ignoring case in 
> nsImapIncomingServer::SetServerDirectory if we really need to fix this bug ?

Should be:

How about set server directory directly to "INBOX" by ignoring case in 
nsImapIncomingServer::SetServerDirectory if we really need to fix bug 160643? :-)
If anyone needs test account(s) on a Courier IMAP server to test these patches
against (as I don't have the facilities here for rolling my own builds), send me
an email and I can arrange one for you.
+                 PL_strlen(onlineDir.get())))
should be onlineDir.Length()

+         if ( ns )
+           result.AssignWithConversion(PL_strncmp(host->fOnlineDir,
+                           ns->GetPrefix(),PL_strlen(host->fOnlineDir)) ?
+                           host->fOnlineDir: "");
+         else
+           result.AssignWithConversion( host->fOnlineDir );
+       }

this should just be

if (ns && !PL_strncmp(host->fOnlineDir, ns->GetPrefix(),
PL_strlen(host->fOnlineDir))
result.AssignWithConversion("");
else
result.AssignWithConversion(host->fOnlineDir);

or you could even make it one statement, right?
// if onlineDir is same as namespace prefix, return empty onlineDir,
// otherwise, return onlineDir.
+           result.AssignWithConversion(ns && !PL_strncmp(host->fOnlineDir,
+                           ns->GetPrefix(),PL_strlen(host->fOnlineDir)) ? ""
+                           : host->fOnlineDir);
Thanks David! I change it.
Attachment #110190 - Attachment is obsolete: true
attachment 114746 [details] [diff] [review] disable user's configure IMAP server directory to 'INBOX' or
'INBOX.' etc when the IMAP server which we connect has a personal namespace
'INBOX.'.
Blocks: 201332
Comment on attachment 114746 [details] [diff] [review]
patch disable user's configure to 'INBOX' v4

sr=bienvenu, if you fix the brace indentation to be like the rest of the file
e.g.,
if (a)
{
...
}

instead of if (a) {
}

thx
Attachment #114746 - Flags: superreview+
I believe this is fixed now. Marking fixed.
Status: NEW → RESOLVED
Closed: 21 years ago
Resolution: --- → FIXED
Product: MailNews → Core
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: