mail messages and attachments truncated

RESOLVED INCOMPLETE

Status

P1
minor
RESOLVED INCOMPLETE
19 years ago
10 years ago

People

(Reporter: philippe, Unassigned)

Tracking

Trunk
PowerPC
Linux

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: closeme 2009-01-22)

Attachments

(4 attachments)

(Reporter)

Description

19 years ago
mail messages and attachments truncated
 at line 143 (text) and line 38 (?) of HTML with embedded image

Comment 1

19 years ago
Reporter we need more information. What build are you using? OS? Platform? Is it in all mail messages? Is it POP or IMAP? Please provide more information.
(Reporter)

Comment 2

19 years ago
more info.  Build ID 2000120713; Linux on Macintosh.  2 (POP) mail messages were
affected.  I got one message in full, read it, but the next time I opened it
(from my local hard drive) it *appeared* truncated.  My reply to the sender
*also* got truncated.  (I'm using text composition only.)  NOTE: I cannot
reproduce bug (yet).
(Reporter)

Comment 3

19 years ago
should add: checked Inbox-übermessage in ~/.mozilla : text file itself is
actually truncated.

Comment 4

19 years ago
Any luck reproducing it?
(Reporter)

Comment 5

19 years ago
I got a truncation when I moved a message from the Unsent Folder back to the Drafts folder.  My impression is that it truncates when you move a message to the folder and then (?) open it before it has had a chance to copy the whole thing.  Just a guess.

Comment 6

19 years ago
Marking NEW as per comments.
Status: UNCONFIRMED → NEW
Ever confirmed: true

Comment 7

18 years ago
change qa contact to myself,
Add to the cc list
QA Contact: esther → sheelar

Comment 8

18 years ago
reassign->fenella
QA Contact: sheelar → fenella

Comment 9

18 years ago
cc'ing naving.  Navin, is this a dup of the corruption bug you fixed a few weeks
ago?

Comment 10

18 years ago
To Esther ..
QA Contact: fenella → esther

Comment 11

18 years ago
reporter, can you try again with latest nightly builds. If it has do with 
move/copy messages it may have been recently fixed.

Comment 12

18 years ago
I had problems today using 20010227 on WinNT. I sent a pdf to someone who
reported they couldn't open it. I recreate the file thinking it was a settings
issue and tried to send it but could not. The SMTP connection would show
activity for a short time and then stop.  After some delay, the send would time
out. After several tries, I zipped the pdf and sent it _apparently_
successfully.  Recipient could not open it. I sent a copy to myself and found
the attachment had been liposuctioned in chunks of 27 lines each.  Attached below:

Comment 13

18 years ago
Posted file mime e-mail as sent

Comment 15

18 years ago
reassigning to ducarroz. Esther,Fenella, or JF, could you try this out with the
attached mail and see if you can reproduce this.
Assignee: putterman → ducarroz
OK, I did some testing with my 0.8.1 build and it's definitely got a problem.  I
tried sending myself a gzipped file that was about a meg in size after it was
compressed.  The on disk temporary files looked fine and the sent mail was saved
successfully to my imap server 5 states away but the file comes in through in
email truncated.

Upping the priority on this, setting it as an SMTP transport problem since
everything else seems to have worked fine and giving it a 0.8.1 milestone even
though the odds of actually getting it done in that timeframe are very low. 
Corrupting attachments is really really bad.  Let's see if we can track this
down and get it on the right people's radar.
Severity: normal → critical
Component: Mail Window Front End → Networking - SMTP
Target Milestone: --- → mozilla0.8.1
Adding my buddy seth since he's good about getting traction on issues like this
for me.

Comment 18

18 years ago
Jean-Francois or Varada, 

Is this a dup of our other bugs where sending has problems with large amounts of
data, especially on Mac and Linux?  If it is, then it's not going to be able to
be done for mozilla0.8.1. We have numerous people looking into this and haven't
had much luck.  See 70928.
Keywords: mozilla0.8.1
Target Milestone: mozilla0.8.1 → ---

Comment 19

18 years ago
We need to fix this. If it turns out to be a dup we can mark it that way.
Keywords: nsbeta1
Priority: -- → P1
Whiteboard: [nsbeta1+]
Target Milestone: --- → mozilla0.9
*** Bug 72846 has been marked as a duplicate of this bug. ***
OK, here's an update of what I've found.

First of all, the fact that the message was truncated was completely false.  It
wasn't truncated on the server.  But when I used view->message source it looked
truncated in imap.  I'm guessing there's a limit on how much we download or
something.  It looked just fine if I viewed it as a POP account.  This is a
different problem.

However, the attachments _are_ currupted.  Here's a line from the attachment:

TryyuFN29BPLX1IF/YStZPFWXoxlEruItfMi+kbG5fYJ5kgr0XWP0Mymo9nfpmtotvKm/uG8
MVVPszuTIDlho2sgjGDvllNS0cMNXfjzRWALtMCOzejXeHvOpsd1cLw5rhtsfLyRGZOAGS2T9NA2a2GZg+ZXyIAcsNSbC8ksJpZAEa/SUlCY3GaWH9vqRfNHosRY/rt+PMcQ1GXDjfTRRx
LNLyKq9/OC5Y9TiGJcHxpFXBcb2I47CWHk/f2C8cqww4thUmGf9WJjPVEX92NZlLFA+5Buf7

Note the middle line is longer than it is supposed to be.

Where is this translation done?  I can probably debug this if someone points me
at the right file.
I need to verify that this isn't happening at the tranport layer to the SMTP
server.  The on disk file that it starts out as is probably a good place to start.
Is blizzard the only one working on this currently?

/be
Whiteboard: [nsbeta1+] → [nsbeta1+] important for mozilla0.8.1
OK. This looks like a transport problem to me. The on-disk image is just fine
but there is some data lossage when sending to the SMTP server or something. If
I take the resulting base64 encoded data and strip out the headers I end up with:

-rw------- 1 blizzard blizzard 1896198 Mar 22 14:19 before
-rwx------ 1 blizzard blizzard 1845782 Mar 22 14:19 after

That's why we have some long lines in the mime encoded data - some of the
newlines have been stripped out somewhere. Digging more.
This patch fixes the short write problem in |nsSocketBOS::Write| with
non-blocking sockets.  This is where we were loosing data.  With this patch in I
was able to send two 1.3 meg attachments without any corruption.

"PR_Write may do a short write on a non-blocking fd"
"PR_Write may do a short write on a non-blocking fd"
"PR_Write may do a short write on a non-blocking fd"
"PR_Write may do a short write on a non-blocking fd"
"PR_Write may do a short write on a non-blocking fd"
"PR_Write may do a short write on a non-blocking fd"
"PR_Write may do a short write on a non-blocking fd"
"PR_Write may do a short write on a non-blocking fd"
"PR_Write may do a short write on a non-blocking fd"
"PR_Write may do a short write on a non-blocking fd"
"PR_Write may do a short write on a non-blocking fd"
"PR_Write may do a short write on a non-blocking fd"
writte is uselessly initialized to 0, but otherwise I like it. 
r/sr=brendan@mozilla.org, reassigning to dougt.

This bug is from 12/18, but dougt's changes landed on 2/21.  Two bugs, or just
same bug in old code?

/be
Assignee: ducarroz → dougt
s/writte/&n/

/be

Comment 29

18 years ago
[this is not dougt's fault... i wrote this code thinking that even blocking
 writes could be partial, conforming to POSIX's definition of write.]
r=bryner
This is in the 0.8.1 branch.
Fix is in.
Status: NEW → RESOLVED
Last Resolved: 18 years ago
Resolution: --- → FIXED

Comment 33

18 years ago
I THINK WE NEED TO RECONSIDER THE SOLUTION TO THIS BUG!


the statement that the thread invoking this function blocks until *all* data is
written, is wrong.  consider the following code:

while (count) {
  int n = write(fd, buf, count);
  if (n < 0) break;
  buf += n;
  count -= n;
}

the first call to write could return (n > 0 && n < count).  the second call
could return (n == -1).  then this code (taken from PR_Write) would return the
error and mask the fact that some bytes were written.  this seems completely
wrong IMO.  in my opinion this convention is a mistake.  i understand that it is
the way NSPR has always been... but, i have my doubts about whether or not it
sould be the convention for nsIOutputStream::{Write,WriteFrom,WriteSegments}.  
consider nsIOutputStream::WriteFrom, for example.  A caller might say "write 256
bytes from this input stream"... but what if the input stream only has 10 bytes
available.  should WriteFrom block until the input stream has more bytes?  what
if that was EOF on the input stream?  then should WriteFrom return an error
code?  and mask the fact that it was able to write 10 bytes?  i don't think so.
 *no* caller of WriteFrom would want it to block the calling thread until *all*
data was written.

in the case of mailnews's use of the socket transport's blocking output stream,
it should really be the one to sit in the while loop... not the implementation
of nsSocketBOS::Write. 
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 34

18 years ago
sorry to be such a stick in the mud about this, but we are being VERY
inconsistent.
We have a working patch.  Please submit an alternative patch.
Assignee: dougt → darin
Status: REOPENED → NEW

Comment 36

18 years ago
The current fix can/should stay for now... but, when I get a chance I'd like to
go back and work on a patch that fixes mailnews' usage of nsIOutputStream::Write

Comment 37

18 years ago
downgrading severity to minor since we have a working solution.
Severity: critical → minor

Updated

18 years ago
Target Milestone: mozilla0.9 → Future

Comment 38

18 years ago
I am having the same problem with linux build 2001071108.  I have a email message in my imap inbox which I have opened before and the full message was displayed.  Now, I can only view the headers and the first line.



The message is displayed in the message list as having a size of 6KB.  If I go to view source, I can see the headers and only the first line of the body.



I can use webmail and see the entire message still.  Closing the mail windows and reopening it with CTRL-2 doesn't change anything.  I have to fully close mozilla and reopen it, then I can view the full message.  I wasn't doing anything abnormal when it happened.

Updated

18 years ago
Blocks: 104166

Updated

17 years ago
QA Contact: esther → trix

Comment 40

17 years ago
Is this bug still alive for Mozilla 1.0 RC1 ? I am having a problem with it
truncating certain attachments from our fax system that come in tiff format. 
The mail reader does not properly interpret the mime encoding and does not
display the message as having an attachment.  The view source windows displays a
truncated mime encoded block of text.  But when I do a "save as" on the message,
it saves to file with no truncation (but with annoying windoze ^M's ending each
line).

I use IMAP w/ssl (UW 2001a) on Redhat 7.2/intel.  Am I posting this to the right
place?  I will attach a problematic (intact) mail message that mozilla trashes.
Using a Mozilla RC1 build on Windows, this message display correctly and I can
see and open the tiff attachment. I am using POP. Let me check IMAP...
works too from IMAP for me

Comment 43

17 years ago
Well, to clear IMAP of any guilt, I opened the message over IMAP with Netscape
4.79-linux-intel and it sees the attachment just fine.  As the bug was filed
against Linux, does anyone else watching this bug have a Linux/intel test bed to
try that attached message on?

Comment 44

17 years ago
the posted image/tiff attachment which didn't work in RC1 works now in RC2 on my
setup.  

Comment 45

17 years ago
I think I'm experiencing the same problem.  I have a .mbox archive of old
messages, and I'm running Mac OS X (10.1.5) with Mozilla 1.1 released last night
(8/26).  (I had the same problem in 1.0.)  Some of the messages in the mailbox
have attachments, but in the mail program they load for a few seconds and then
show up blank.  Other mail programs display the messages correctly.  I checked
the message source as Mozilla sees it against the actual text of the .mbox file,
and Mozilla is truncating the messages at around 700 lines or just under 48,500
characters (don't know if that's helpful or useless).
The nsbeta1+ and mozilla0.8.1 marking added by putterman and brendan on
2001-03-21  and 2001-03-22, respectively, are no longer useful.

Clearing those.
Whiteboard: [nsbeta1+] important for mozilla0.8.1
Product: MailNews → Core

Updated

13 years ago
Assignee: darin → nobody
QA Contact: stephend
Target Milestone: Future → ---
Is anyone still seeing this? With no duplicates or recent complaints I really doubt that this is a problem any more.
Whiteboard: closeme 2009-01-22
QA Contact: networking.smtp
RESO INCO per lack of response to comment 47. If you feel this change was made in error, please respond to this bug with your reason why.
Status: NEW → RESOLVED
Last Resolved: 18 years ago10 years ago
Resolution: --- → INCOMPLETE
(Assignee)

Updated

10 years ago
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.