223062 - Unable to download from a POP mail server - hangs on mail with null bytes

Status: RESOLVED FIXED

(MailNews Core :: Networking: POP, defect)

Description

[JP]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20030925
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20030925

I have a message I received that mozilla is unable to download from my POP mail
server.  The download hangs idefinitely while trying to download the 17K
message.  Any other message works fine.  The message had caused all POP messages
to fail until I isolated the problem to this specific message.

If I use my mail server's web interface I can view the message and it appears to
be fine.

I have found that if I forward the message to myself the forwarded message
causes the same problem.  I can forward the message to anyone that wants to
reproduce the problem.

Reproducible: Always

Steps to Reproduce:
1. Get the bad message to you POP server (I can send it to you)
2. Download your mail
3. Mozilla hangs.

Actual Results:  
mozilla should receive the message.

Expected Results:  
mozilla hangs.

Comment 1

[Christian Eyrich]

Do you have access to another POP account? If yes, what do you get if you
forward it to and download if from there?

Comment 2

[Vidar Haarr (not reading bugmail)]

Reporter: could you attach the message source here ? Also, please try this using
a recent nightly build from
<http://ftp.mozilla.org/pub/mozilla.org/mozilla/nightly/latest/>.

Comment 3

[JP]

The plot thickens:  I tried forwarding the message to another POP server and
mozilla was able to download it just fine.  I then went back to the original POP
server (that mozilla is unable to download from) and was able to get the message
with outlook.  So it looks like it's specfic to Mozilla/this POP server/and this
message.  The service is EIOmail.com

I'll try a nightly build tonight.

Comment 4

[JP]

Attachment: This is the message that is causing the problem

This is the message that is causing the problem - This has been download ith
mozilla from the working POP server.

Comment 5

[David :Bienvenu]

can you try generating a pop3 protocol log of a session trying to download the
message from the server where it fails? thx. This sounds like a server problem,
from what you describe so far

http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#pop

Comment 6

[Christian Eyrich]

JP sent forwarded the mail to my account and fortunately I can't download it too.

At the moment I've no clue why, the mail is terminated by CRLF.CRLF like it
should. I'll run this in the debugger.

Comment 7

[Christian Eyrich]

Attachment: Ethereal log of hang

Comment 8

[Christian Eyrich]

Attachment: Mozilla log of hang

Comment 9

[David :Bienvenu]

out of curiousity, can you resend it to me as well at [e-mail] ? thx.

Comment 10

[David :Bienvenu]

is the server sending us null bytes?

Comment 11

[Christian Eyrich]

Server, yes. The questions are is which one - the senders SMTP or my providers POP3.
And why Mozilla gets confused by it. While getting this message with Eudora and
KMail I can see the nulls too. But they survive it.

Comment 12

[David :Bienvenu]

well, both servers, actually. I don't believe either are supposed to allow nulls
through. My pop3 server won't give me the nulls at all. Can I get access to your
pop3 server that will let them through? I tried to fix the pop3 code to deal
with embedded nulls (or was that the imap code? I can't remember...) but it was
hard to test without a server that would give me nulls.

Comment 13

[Christian Eyrich]

For server access see my mail.

Though I don't the understand the complete code, I think I encountered at least
one of some failures.
On nsMsgLineBuffer.cpp#405 there's code that should replace nulls with spaces.
But in one case it won't work: if numBytesCopied is < m_numBytesInBuffer.

Currently it's
  for (i=m_numBytesInBuffer;i <numBytesCopied;i++)
but should be
  for (j=0,i=m_numBytesInBuffer;j <numBytesCopied;i++,j++)
or
  newbytes = m_numBytesInBuffer + numBytesCopied;
  for (i=m_numBytesInBuffer;i <newbytes;i++)

With nulls in the buffer ReadNextLine()'s code
  endOfLine = PL_strchr(startOfLine, m_lineToken);
will never see the next line end.

But with or without this change I always get
###!!! ASSERTION: OnDataAvailable implementation consumed no data: 'Error', file
nsInputStreamPump.cpp, line 451
That's because the buffer (8192 bytes) is full and can't read a byte. After
extending the buffer to e.g. 9192 bytes and the above change the message is
retreived without problem.

It looks to me as the code to resize/grow the buffer doesn't work, resp. the
code to compute the necessary size.

Comment 14

[Christian Eyrich]

Attachment: proposed patch

Patch according to comment #13 and an additional fix for buffer grow
computation.
This enables me to get the message from my server notwithstanding the nulls. I
hope this also applies to JPs problem.

Comment 15

[David :Bienvenu]

Comment on attachment 133864 [details] [diff] [review]
proposed patch

sr=bienvenu - I don't think I'll try to get this into 1.6a, unless it looks
like 1.6a is going to drag on an d on. Thx for fixing this.

Comment 16

[JP]

Will this be in tonights nightly build?  I could test it if it is.

Comment 17

[David :Bienvenu]

it'll probably be a couple days until the tree opens again for 1.6b development
- I'll let you know when I check this in.

Comment 18

[David :Bienvenu]

reassigning to Christian. I'll check this in today, so it will be in tomorrow's
build.

Comment 19

[David :Bienvenu]

fixed on 1.6b trunk.

Comment 20

[SchugyC]

*** Bug 227198 has been marked as a duplicate of this bug. ***

Comment 21

[jim garner]

I hesitate to reopen this cuase I don't know much about it, but I can duplicate
this with similar emails and I am currently using...
Mozilla 1.6b
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6b) Gecko/20031117

I am getting an email from Flow54.com (spam from an online casino) and everytime
it comes through the Mozilla Mail Get Messages process, Mail stops and I get an
error saying... Well I will attach the gif. (But it says something about cannot
write to the HD.)

I have posted on this in the nesgroups, but saw a response that said this was
fixed. (under Subject.. "Unable to write while DLing email..." -Dated 12/2 on
n.p.m.mail-news)

Thought you should know.

Please email me for more info, or specifics at Jim .at. Leaders.net

Thanx.

Comment 22

[jim garner]

Attachment: gif of Error Dialog

here is the error message in a gif

Comment 23

[Christian Eyrich]

Jim, from the error message it doesn't look like this bug (it didn't produce any
message, just a hang). But it may be helpful to attach a communications log (see
comment #5 for how to do).

Comment 24

[Christian Eyrich]

*** Bug 229468 has been marked as a duplicate of this bug. ***

Comment 25

[Christian Eyrich]

*** Bug 229584 has been marked as a duplicate of this bug. ***

Comment 26

[Christian Eyrich]

*** Bug 230830 has been marked as a duplicate of this bug. ***

Comment 27

[Christian Eyrich]

*** Bug 239452 has been marked as a duplicate of this bug. ***

Comment 28

[timeless]

*** Bug 187828 has been marked as a duplicate of this bug. ***