Open Bug 59704 Opened 19 years ago Updated 6 years ago

support for IMAP Referrals (RFC2221, RFC2193), when server advertises MAILBOX-REFERRALS we should use RLIST and RLSUB (will help with MS Exchange and others)

Categories

(MailNews Core :: Networking: IMAP, enhancement)

enhancement
Not set

Tracking

(Not tracked)

People

(Reporter: Bienvenu, Unassigned)

References

(Blocks 2 open bugs)

Details

(Keywords: imap-interop)

Attachments

(1 file)

Add support for IMAP referrals, including using APPEND when copy between folders
fails with a REFERRAL response.
accepting - this will help with MS Exchange interoperability.
Severity: normal → enhancement
Status: NEW → ASSIGNED
Keywords: interop
Adding MS Exchange IMAP on the summary.
Summary: add support for IMAP Referrals → MS Exchange IMAP: add support for IMAP Referrals
IMAP referrals are not limited to MS Exchange. It's a general standard.
Summary: MS Exchange IMAP: add support for IMAP Referrals → add support for IMAP Referrals
p5
Priority: P3 → P5
QA Contact: huang → meehansqa
Summary: add support for IMAP Referrals → MS Exchange IMAP:add support for IMAP Referrals
correcting summary - IMAP referrals are not a proprietary MS Exchange feature.
Summary: MS Exchange IMAP:add support for IMAP Referrals → IMAP:add support for IMAP Referrals
Here's the RFC: http://www.faqs.org/rfcs/rfc2193.html

In essence, if the server advertises the MAILBOX-REFERRALS, then the client is
supposed to use RLIST and RLSUB instead of LIST and LSUB. Any command that takes
a mailbox name can return NO and one or more IMAP URLS that are referrals to the
server and folder to use instead.

 "When a client processes an IMAP mailbox referral, it will open a new
   connection or use an existing connection to the remote server so that
   it is able to issue the commands necessary to process the remote
   mailbox"

So our imap parser would need to be extended to handle getting referral
responses, and acting on them. There are two problems here - one is that we need
to be able to run the new url just as if it was the original url, and the other
is that we need to be able to connect and login to the remote server and folder
w/o their being an account in the acct mgr for the remote server. And we need to
have the connection mgmt that's handled by nsImapIncomingServer to re-use
existing connections, etc. This will all be quite tricky to do cleanly. I don't
want there to be dozens of checks throughout the code that we're handling an
imap referral.

The first question is can we re-use the existing url, or do we need to create a
new url to run the referral? Logically, for the clients of the url, it should
look like we're still running the first url so that the client gets the correct
notifications, etc. The url doesn't know much about the protocol object, so it
is possible we could get/create an nsImapIncomingServer object for the referred
server, get/create a protocol object, and then associate the original url with
the protocol object, and run the url. We'd have to set the authentication info
from the referral on the protocol object, so it would know to use it. We'd need
to clean up the original protocol object appropriately, w/o setting the url state...

Anyway, this is a difficult but fun problem to solve.

I now have to use an exchange server which requires IMAP referrals right at
login. Perhaps we can at least support just the login aspect to start? It makes
it very frustrating to configure a new account - you have to set it up to talk
to the parent server (i.e. ms-exchange.foo.com) then login, and read the error
message, and then edit the server properties to change it (i.e. to
some-referral.foo.com)
Alec, no promises, but can you attach a protocol log of what you're server is
sending you? thx.
Attached file IMAP log
As promised.. (domain replaced with foo.com)
logon referrals sound like a job for a special logon redirector but maybe it
could be done in-line in the imap code.
Product: MailNews → Core
*** Bug 300367 has been marked as a duplicate of this bug. ***
Just to be clear, there are two different classes of IMAP referral -- Login Referrals (as defined by RFC 2221) and Mailbox Referrals (as defined by RFC 2193).

In addition to being useful for Exchange interoperability, widespread support for these standards would also be useful for other IMAP server aggregation schemes.

(At present, the only effective way to aggregate a set of IMAP servers is with a relaying proxy server like Perdition, which introduces obvious security and scalability issues.)

Votes +=1.

Blocks: 370090
improved summary for searchability.

since current behavior violates RFC, would it be appropriate to change severity to something other than ENH?
Status: ASSIGNED → NEW
QA Contact: meehansqa → networking.imap
Hardware: PC → All
Summary: IMAP:add support for IMAP Referrals → IMAP:add support for IMAP Referrals, when server advertises MAILBOX-REFERRALS we should use RLIST and RLSUB (will help with MS Exchange and others)
the current behavior doesn't violate the core IMAP RFC, afaik - we just don't support the REFERRAL extension RFC's.
Summary: IMAP:add support for IMAP Referrals, when server advertises MAILBOX-REFERRALS we should use RLIST and RLSUB (will help with MS Exchange and others) → support for IMAP Referrals (RFC2221, RFC2193), when server advertises MAILBOX-REFERRALS we should use RLIST and RLSUB (will help with MS Exchange and others)
Duplicate of this bug: 401435
Duplicate of this bug: 378713
Duplicate of this bug: 472109
Product: Core → MailNews Core
This may not be obvious, but the Referral RFCs, of which there are two, RFC 2193 and RFC2221 (login vs. mailbox referrals) are not specifically for supporting Microsoft products, such as Exchange. These RFCs would be a scaling win for mail server operators if clients supported the RFCs. 

Pine/Alpine support these RFCs, and the Outlook suite of products do as well. Its unfortunte that Thunderbird does not.

The dovecot IMAP server does support these RFCs, but without client support in Thunderbird it isn't very useful.
+1 to 2221, I don't really mind about MAILBOX-REFERRALS, but LOGIN-REFERRAL should be fairly simple and very useful.
Actually looks like Outlook, Outlook Express and Exchange do NOT support these.

http://support.microsoft.com/kb/217388
http://technet.microsoft.com/en-us/library/bb310763.aspx
rkent would you have an interest in this?
Assignee: mozilla → nobody
Priority: P5 → --
Comment on attachment 147913 [details]
IMAP log

>0[294518]: 1e36c28:ms-exchange.foo.com:NA:SetupWithUrl: clearing IMAP_CONNECTION_IS_OPEN
>3364[1cbdd60]: ImapThreadMainLoop entering [this=1e36c28]
>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:NA:ProcessCurrentURL: entering
>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:NA:ProcessCurrentURL:imap://aflett@ms-exchange.foo.com:143/select%3E%5EINBOX:  = currentUrl
>3364[1cbdd60]: ReadNextLine [stream=1cf71f0 nb=98 needmore=0]
>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:NA:CreateNewLineFromSocket: * OK Microsoft Exchange IMAP4rev1 server version 5.5.2657.74 (ex-bridge-02.foo.com) ready

>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:NA:SendData: 1 capability

>3364[1cbdd60]: ReadNextLine [stream=1cf71f0 nb=98 needmore=0]
>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:NA:CreateNewLineFromSocket: * CAPABILITY IMAP4 IMAP4rev1 IDLE LITERAL+ LOGIN-REFERRALS MAILBOX-REFERRALS NAMESPACE AUTH=NTLM

>3364[1cbdd60]: ReadNextLine [stream=1cf71f0 nb=28 needmore=0]
>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:NA:CreateNewLineFromSocket: 1 OK CAPABILITY completed.

>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:NA:SendData: Logging suppressed for this command (it probably contained authentication information)
>3364[1cbdd60]: ReadNextLine [stream=1cf71f0 nb=82 needmore=0]
>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:NA:CreateNewLineFromSocket: 2 OK [REFERRAL imap://aflett;AUTH=*@s2009exm01.foo.com/] LOGIN completed.

>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:A:SendData: 3 namespace

>3364[1cbdd60]: ReadNextLine [stream=1cf71f0 nb=54 needmore=0]
>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:A:CreateNewLineFromSocket: * NAMESPACE (("" "/")) NIL (("Public Folders/" "/"))

>3364[1cbdd60]: ReadNextLine [stream=1cf71f0 nb=39 needmore=0]
>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:A:CreateNewLineFromSocket: 3 OK NAMESPACE completed successfully

>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:A:SendData: 4 lsub "" "*"

>3364[1cbdd60]: ReadNextLine [stream=1cf71f0 nb=119 needmore=0]
>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:A:CreateNewLineFromSocket: 4 NO [REFERRAL imap://aflett;AUTH=*@s2009exm01.foo.com/] The requested mailbox is not available on this server

>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:A:SendData: 5 lsub "" "Public Folders/*"

>3364[1cbdd60]: ReadNextLine [stream=1cf71f0 nb=119 needmore=0]
>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:A:CreateNewLineFromSocket: 5 NO [REFERRAL imap://aflett;AUTH=*@s2009exm01.foo.com/] The requested mailbox is not available on this server

>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:A:SendData: 6 list "" "INBOX"

>3364[1cbdd60]: ReadNextLine [stream=1cf71f0 nb=59 needmore=0]
>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:A:CreateNewLineFromSocket: 6 NO There is no replica for that mailbox on this server.

>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:A:SendData: 7 list "" "Trash"

>3364[1cbdd60]: ReadNextLine [stream=1cf71f0 nb=59 needmore=0]
>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:A:CreateNewLineFromSocket: 7 NO There is no replica for that mailbox on this server.

>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:A:SendData: 8 create "Trash"

>3364[1cbdd60]: ReadNextLine [stream=1cf71f0 nb=59 needmore=0]
>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:A:CreateNewLineFromSocket: 8 NO There is no replica for that mailbox on this server.

>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:A:SendData: 9 select "INBOX"

>3364[1cbdd60]: ReadNextLine [stream=1cf71f0 nb=59 needmore=0]
>3364[1cbdd60]: 1e36c28:ms-exchange.foo.com:A:CreateNewLineFromSocket: 9 NO There is no replica for that mailbox on this server.

>0[294518]: 1e36c28:ms-exchange.foo.com:A:SendData:TellThreadToDie: 10 logout

>0[294518]: 1e36c28:ms-exchange.foo.com:A:TellThreadToDie: close socket connection
>3364[1cbdd60]: ImapThreadMainLoop leaving [this=1e36c28]
Attachment #147913 - Attachment is patch: false
(In reply to Wayne Mery (:wsmwk) from comment #21)
> rkent would you have an interest in this?

Probably not. I've never attempted to work in the IMAP protocol area.
Also see Bug 495318. LIST-EXTENDED (RFC 5258) provides an alternate way to get the same information.

This bug might have my name on it!
Blocks: RFC5258
You need to log in before you can comment on or make changes to this bug.