126987 - Crash reading certain IMAP message [@ nsImapMailFolder::NormalEndMsgWriteStream]

Status: VERIFIED FIXED

(MailNews Core :: Networking: IMAP, defect, P2)

Description

[Henrik Gemal]

I seem to crash every time I read a certain IMAP message.
My talkback IDs are:
TB3187197K
TB3187082X

Reproduceable all the time. I just select the IMAP message and I crash.

I can see this in the IMAP log:
:SendData: 5 UID fetch 3 (UID RFC822.SIZE BODY[])
:CreateNewLineFromSocket: * 3 FETCH (UID 3 RFC822.SIZE 2321 BODY[] NIL)
:STREAM:OPEN Size: 2321: Begin Message Download Stream
:CreateNewLineFromSocket: 5 OK FETCH fuldført.
:STREAM:CLOSE: Normal Message End Download Stream
:PARSER:Internal Syntax Error: %s: 5 OK FETCH fuldført.

20020220

Comment 1

[Henrik Gemal]

Perhaps is the "NIL" in the server response:
3 FETCH (UID 3 RFC822.SIZE 2321 BODY[] NIL)
that mozilla doesn't understand. Should there be a size here?
This could happen when the server cant send the message.

My Outlook Express shows the following message is the mesasge pane:
Message could not be displayed
Outlook Express encountered an unexpected problem while displaying this 
message. Check your computer for low memory or low disk space and try again.

Perhaps Mozilla should do something like that...

Comment 2

[Henrik Gemal]

So basiclly the problem is the Mozilla doesn't support the NIL string.
If you have:
3 FETCH (UID 3 RFC822.SIZE 2321 BODY[] NIL)
this means that the server cannot show the BODY. Am I correct?

Comment 3

[Scott MacGregor]

stack trace:
nsImapMailFolder::NormalEndMsgWriteStream
[d:\builds\seamonkey\mozilla\mailnews\imap\src\nsImapMailFolder.cpp, line 3656]
XPTC_InvokeByIndex
[d:\builds\seamonkey\mozilla\xpcom\reflect\xptcall\src\md\win32\xptcinvoke.cpp,
line 106]
EventHandler [d:\builds\seamonkey\mozilla\xpcom\proxy\src\nsProxyEvent.cpp, line
516]

any chance we can get the message causing the crash?

Comment 4

[Scott MacGregor]

*** This bug has been marked as a duplicate of 114570 ***

Comment 5

[Henrik Gemal]

why is this marked dupe? Are we sure it's a dupe. I'm not. I'm sure we crash
because of the NIL

Scott I cant attach the message since the server isn't returning it. It says NIL!

Comment 6

[Jan Carpenter]

*** Bug 123010 has been marked as a duplicate of this bug. ***

Comment 7

[Henrik Gemal]

*** Bug 109327 has been marked as a duplicate of this bug. ***

Comment 8

[Phil Pishioneri]

I don't know if my crash, TB3916632X, is related (build 2002030604 Win32).

I've been trying to forward copies (stored on my IMAP server) of the latest worm
(the one allegedly from Microsoft) back to service providers to help identify
infected systems.

However, with some copies of the worm, I can't do anything useful with them. 
Can't delete them, or view source (get a window that's empty).  If I then try to
select another message, Mozilla crashes.

I've noticed that the body of the email (first mime part) has some nulls
embedded in it, and think that those are causing the trouble.  If I turn off
display of the message (View -> Message), I can successfully delete it.

Comment 9

[David :Bienvenu]

Nulls are not allowed in imap messages per the rfc, and mozilla won't display
messages with nulls in them.

Comment 10

[Phil Pishioneri]

> Nulls are not allowed in imap messages per the rfc

Is Mozilla using imapv2? I hope that's the right one since Mozilla connects to
the port for imapv2.  If it does, a quick read of RFC1176 doesn't appear to
mention anything about nulls, just making references to RFC822.

Glancing at RFC822, it doesn't seem to forbid nulls in the text of a message,
which is where the ones I see are located (they are in a mime part, but to the
imap server, that's just text).

> mozilla won't display messages with nulls in them

If the nulls are the problem (I'm guessing they are), not displaying such a
message is fine with me (as long as Mozilla indicates such).  It's the crashing
part I don't like.

Comment 11

[David :Bienvenu]

we're supporting imap4 - this has come up in other bugs, and basically, if you
look at the grammar in the rfc for imap4, the characters allowed boils down to a
type that doesn't include nulls. I'm trying to find the bug report that explains
it but bugzilla is being really slow today.
I agree with you that the crashing is a problem. We're a bit stuck reproducing
this since we need to convince access to a server returning this nil.

Comment 12

[scottputterman]

reassigning to bienvenu

Comment 13

[David :Bienvenu]

Henrik, can you copy this message into an account that I can look at, perhaps
with another mail client?

Comment 14

[Henrik Gemal]

David: I cant copy the message since it's not returned by the server. That why
the server returns:
3 FETCH (UID 3 RFC822.SIZE 2321 BODY[] NIL)
the body isn't there....! So I cant copy the message.

Comment 15

[David :Bienvenu]

Henrik, can you give me temporary access to the account with this message?

Comment 16

[Henrik Gemal]

sorry. my imap server is behind a firewall

Comment 17

[David :Bienvenu]

Phil, your crashes are unrelated - they're a different bug that has since been
fixed. So I'm left with Henrik's crash where the server says it has an empty
message with 2321 bytes in it :-)

Comment 18

[David :Bienvenu]

Attachment: proposed fix

this is just a guess - it's the only way I can see how we could crash in
NormalEndMsgWriteStream. Basically, mork will return a null hdr in some cases
(e.g., a msg key of -1) but not return an error, so I need to check for that
case in the db.

Comment 19

[Cavin Song]

Comment on attachment 74014 [details] [diff] [review]
proposed fix

r=cavin.

Comment 20

[(not reading, please use [e-mail] instead)]

Comment on attachment 74014 [details] [diff] [review]
proposed fix

sr=sspitzer

Comment 21

[Adam Hauner]

*** Bug 130520 has been marked as a duplicate of this bug. ***

Comment 22

[harrij]

Attachment: Message causing Mozilla crash

Comment 23

[harrij]

Attachment: Another message causing Mozilla crash

Comment 24

[harrij]

I reported this separately as bug 130520 which is marked as dupe os this bug.
I'm seeing discussion about NIL not supported, but in my case the body size is
set ie. it's not NIL and 'zilla still crashes. I'm getting this at end of the log:

327[3a543f0]: mail.kolumbus.fi:S-INBOX:STREAM:OPEN Size: 3976: Begin Message
Download Stream
327[3a543f0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-MessageRFC822: 0
327[3a543f0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-MessageHeaders: 0
327[3a543f0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-Part-Prefetched: 0
327[3a543f0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-Multipart: 0
327[3a543f0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-Boundary: 0
327[3a543f0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-Leaf: 1
327[3a543f0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-MIMEHeader: 1
327[3a543f0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-Part-Inline: 1
327[3a543f0]: mail.kolumbus.fi:S-INBOX:SendData: 11 UID fetch 17463 (BODY[1])

327[3a543f0]: mail.kolumbus.fi:S-INBOX:CreateNewLineFromSocket: * 250 FETCH
(BODY[1] {2721}


327[3a543f0]: mail.kolumbus.fi:S-INBOX:CreateNewLineFromSocket:  UID 17463)

327[3a543f0]: mail.kolumbus.fi:S-INBOX:PARSER:Internal Syntax Error: %s:  UID 17463)

...message body here....

327[3a543f0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-Boundary: 0
327[3a543f0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-Leaf: 2
327[3a543f0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-XHeader: 2
327[3a543f0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-MIMEHeader: 2
327[3a543f0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-Filling: 2
327[3a543f0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-Boundary-Last: 0
327[3a543f0]: mail.kolumbus.fi:S-INBOX:STREAM:CLOSE: Normal Message End Download
Stream

I've created attachments for the crash causing messages.

Comment 25

[David :Bienvenu]

this part is wrong:

"250 FETCH (BODY[1] {2721}" - there should be string literal here and instead
there's just the UID response.

"UID 17463)"

I doubt I'll be able to get my mail server to serve up the same protocol with
that message. What server are you using?

Comment 26

[Asa Dotzler [:asa]]

Comment on attachment 74014 [details] [diff] [review]
proposed fix

a=asa (on behalf of drivers) for checkin to the 1.0 trunk

Comment 27

[David :Bienvenu]

potential fix checked in - please try tomorrow's build, and report any talkback
incident #'s s if you still crash. thx.

Comment 28

[harrij]

The IMAP mailserver is InterMail vM.5.01.03.15 201-253-122-118-115-20011108

Comment 29

[harrij]

I'm not sure if the fix is supposed to be in this build but Mozilla still
crashes with the two attached messages when trying to retrieve them from the server.

Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:0.9.9+) Gecko/20020314 
buildid: 2002031403

Talkback IDs:
TB4067112Q
TB4067163Y

Comment 30

[David :Bienvenu]

I checked it in on 03/14 so tomorrow's build would be 03/15, i.e., today's
build. I'm pretty sure the build you tried would not have the potential fix -
please try the 03/15 build.

Comment 31

[harrij]

Tested with Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:0.9.9+) Gecko/20020317
buildid 2002031708.

It doesn't crash anymore, but after succesfully reading the message that
previously crashed Mozilla now causes the mail client to stop reading messages.
Mail client opens the message view window but doesn't load the message (any
message on the server) at all. Mozilla doesn't freeze or crash, but just stops
reading messages.

Log shows this after reading the error causing message and after several
attempts to open another message after that. Mozilla doesn't seem to react to
those message opening attempts at all after that one message.

307[41543d0]: mail.kolumbus.fi:S-INBOX:CreateNewLineFromSocket:
----------------------------------------

307[41543d0]: mail.kolumbus.fi:S-INBOX:CreateNewLineFromSocket: Microsoft is
registered trademark of Microsoft Corporation.

307[41543d0]: mail.kolumbus.fi:S-INBOX:CreateNewLineFromSocket: Windows and
Outlook are trademarks of Microsoft Corporation.

307[41543d0]: mail.kolumbus.fi:S-INBOX:CreateNewLineFromSocket:  UID 17463)

307[41543d0]: mail.kolumbus.fi:S-INBOX:PARSER:Internal Syntax Error: %s:  UID 17463)

307[41543d0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-Boundary: 0
307[41543d0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-Leaf: 2
307[41543d0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-XHeader: 2
307[41543d0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-MIMEHeader: 2
307[41543d0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-Filling: 2
307[41543d0]: mail.kolumbus.fi:S-INBOX:SHELL:GENERATE-Boundary-Last: 0
307[41543d0]: mail.kolumbus.fi:S-INBOX:STREAM:CLOSE: Normal Message End Download
Stream
307[41543d0]: BODYSHELL:  Adding shell to cache.

Comment 32

[David :Bienvenu]

Is it just in the inbox that you can't read any more messages, or can you not
read imap messages in any folders? What happens if you hit the offline toggle to
go offline and back online again? Can you read messages after that?

Comment 33

[harrij]

It's just the inbox, and it gets resolved by cliking another folder or Get
Msgs-button. Without that Mailnews doesn't load any messages in that folder. 

Comment 34

[Henrik Gemal]

I no longer crash. I've opened bug 133689 to track the problems loading messages
just after getting "Internal Syntax Error" IMAP errors.

v20020326