37827 - IMAPMS Exchange: Message disappear from the message body

Status: VERIFIED FIXED

(SeaMonkey :: MailNews: Message Display, defect, P3)

Description

[Karen Huang]

Message disappear from the message body

Used 05-01-11-M16 commercial build:

1) Login to MS Exchange IMAP Mail Account.
2) Select the Inbox messages
3) Actual Results: From the Inbox's messages, I can see the existing message 
displaying on the thread pane, but the existing messages are disappear from the 
message body 

Expected results: Should display the existing Inbox message from the message 
body as normal.

Comment 1

[scottputterman]

I'm reassigning this to mscott since I think it has to do with a problem 
displaying an Imap message.  

Comment 2

[Scott MacGregor]

sounds like MS Exchange stuff just broke all over the place recently. A PR_Log
might help.

Comment 3

[Karen Huang]

Attachment: attached 5/1 build MSE IMAP log

Comment 4

[Karen Huang]

Please notice the following lines:
--------------------------------------------------------------------------------
266[3313ec0]: poisonoak.mcom.com:S-INBOX:CreateNewLineFromSocket: * 3 FETCH (UID 
111 RFC822.SIZE 559 FLAGS (\Seen) ENVELOPE ("Mon, 01 May 2000 13:22:07 -0700" 
"test  send" (("test02" NIL "test02" "
266[3313ec0]: poisonoak.mcom.com:S-INBOX:STREAM:OPEN Size: 559: Begin Message 
Download Stream
266[3313ec0]: poisonoak.mcom.com:S-INBOX:PARSER:Internal Syntax Error: %s: * 3 
FETCH (UID 111 RFC822.SIZE 559 FLAGS (\Seen) ENVELOPE ("Mon, 01 May 2000 
13:22:07 -0700" "test  send" (("test02" NIL "
--------------------------------------------------------------------------------
Log just stop here.

Comment 5

[leger]

Putting on [nsbeta2+] radar for beta2 fix.

Comment 6

[jefft]

Looks like we don't know how to parse the ENVELOPE server response. We don't 
have this problem on 4.7 since we don't use ENVELOPE at all in 4.7. ENVELOPE was 
added lately.

Comment 7

[Scott MacGregor]

adding interop keyword so we can track our interop bugs easily for mail connect.

Comment 8

[jefft]

Is there any MS EXCHANGE servers in house? posisonoak seems not behave well and 
generate wrong response for ENVELOPE. Lisa, do we have more MS EXCHANGE test 
accounts?

Comment 9

[Scott MacGregor]

Jeff, I was going to help you look at this too. Can you elaborate on what you
mean by it generates a wrong envelope response? is the implication that our
verison of the MS Exchange server is out of date and issuing illegal protocol?

Comment 10

[Scott MacGregor]

One more comment...if poisonoak is spittng out incorrect ENVELOPE data, it
sounds like all exchange servers are doing the same thing since I've seen imap
logs from outside users on exchange servers posting similar stack traces (where
we choke in the ENVELOPE field).

Comment 11

[jefft]

Yes, the MS EXCHANGE server may be out of date. The ENVELOPE structure should 
only contain 10 elements. According to poisonoak it returns 12 in one of the 
test message. 4.7 doesn't use ENVELOPE structure. There must be reasons. We 
might use the alternative way.

Comment 12

[Scott MacGregor]

we use the envelope field in 5.0 and not in 4.7 because aol wants us to use it
as part of aol mail. It eases their server load. So getting rid of it isn't an
option I don't think.

We can try to disable envelope support for exchange servers (I don't like this
idea). Or we can try to accomodate the extra 2 requests in the envelope field if
that is possible....Hmmm

Comment 13

[Karen Huang]

Jeff, we only have poisonoak.mcom.com in house for MS Exchange......I will try 
on this server again....

Comment 14

[jefft]

This is really a bad decision. Because using ENVELOPE has more fields (10) to 
fetch from. In theory we only need 5 (From To Cc Subject and Date). It's only 
AOL server which requires this. Not necessary benefits for all other servers, 
performance wise. Besides they are using non-standard XAOL-ENVELOPE command. Why 
not just make an exception to AOL server only? The whole decision really sounds 
silly to me.

Comment 15

[Scott MacGregor]

It turns out there is nothing wrong with our exchange server actually. I spent
some more time looking into this so I can try to get familiar with some of our
interop stuff before mail connect.

We have a bug in the server parser for the method: parse_addresses.

Basically, in the case of issuing an envelope field that contains multiple
addresses, addresses are packaged using parantheses. If a field has multiple
addresses, then the addresses are nested in a set of parantheses.
i.e.

(("Yun Zhao" NIL "tj" "zia.mcom.com"))
or in the multiple case, something like:
((NIL NIL "tj" "zia.mcom.com")
(NIL NIL "test05" "poisonoak.mcom.com"))

In the exchange server case, there is a space between each email address in a
field that has multiple email addresses:
((NIL NIL "tj" "zia.mcom.com")*space*(NIL NIL "test05" "poisonoak.mcom.com"))
In the Netscape server implementation, there is no space:
((NIL NIL "tj" "zia.mcom.com")(NIL NIL "test05" "poisonoak.mcom.com"))

Our parser was not handling the space correctly. This would cause us to not
properly parse the nested email addresses which would in turn screw us up and
make us try to read the 2nd email address as a brand new field in the envelope.
So we would end up not reading the envelope correctly, generating a syntax error.

I'm attaching the fix to this bug. This is going to fix a slew of Exchange
server bugs because we've got a bunch of them with the ENVELOPE field in the
stack trace. I'll put together a list of the bugs I think this will fix.

Jeff, can you review me on this and make sure I'm not doing something stupid
here? thanks man.

Comment 16

[Scott MacGregor]

Attachment: Fix for the exchange server problem with the envelope field.

Comment 17

[Scott MacGregor]

I tested this patch against poison oak and the netscape imap server. But I need
to test it out a bit more b4 I check it in. Assuming jeff gives me the thumbs
up, I'll try to land this tonight.

Comment 18

[lchiang]

good work, Scott!

Comment 19

[jefft]

Nice catch, mscott. This probably will fix a bunch of my MS EXCHANGE bugs. Way 
to go. Go for it. r=jefft.

Comment 20

[Scott MacGregor]

Thanks for the fast review Mr. Jeff! I'll go through and mark fixed the bugs I
think this ended up fixin' for ya!!

Comment 21

[John G. Myers]

RFC 2060 does not allow a space in that part of an ENVELOPE.  The Exchange 
server is violating protocol.

Comment 22

[Karen Huang]

So does it mean MS Exchange is the only exception? Should I still verify this 
bug?

Comment 23

[Scott MacGregor]

Yes you should still verify this bug. We still have to work with MS Exchange
servers even if they aren't issuing valid protocol.

Comment 24

[Karen Huang]

OK. I will verify this bug then.

Comment 25

[Karen Huang]

Verified on WinNT 05-24-09-M16 commercial build:
Great!I see the message display on the message body on MS Exchange IMAP now...
Marking as verified!!