mail lost when getting fetchmailed messages

RESOLVED FIXED

Status

--
critical
RESOLVED FIXED
15 years ago
10 years ago

People

(Reporter: kirill, Assigned: pkwarren)

Tracking

Trunk
x86
Linux

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Reporter)

Description

15 years ago
User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040312 Debian/1.6-3
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040312 Debian/1.6-3

I am using X11; U; Linux i686; en-US; rv:1.6) Gecko/20040312 Debian/1.6-3 and I
saw today Mozilla losing email message. 
I am using fetchmail to deliver the mail from 2 POP3 accounts into the single
localhost account. I've got 2 emails which were logged in exim delivery logs.
When I checked the account in Mozilla mail it showed only one new message. The
message shown was in Cyrillic Win1251 encoding (HTML body and subject line). The
lost message was (probably) in Russian KOI-8R: subject line in Cyrillic with
base64 plain text body text. To add insult to injury the delivered message
happen to be a junk message.

The egrep of sender address in ~/.mozilla did not return any matches.
I am using fetchmail 5.9.11+NTLM+SDPS+NLS and Exim version 3.36 on 2.6.4 kernel.

Reproducible: Couldn't Reproduce
Steps to Reproduce:
1.
2.
3.
(Reporter)

Comment 1

15 years ago
Created attachment 145498 [details]
Spool file with 5 test messages
(Reporter)

Comment 2

15 years ago
The problem is reproducable with 
- 1.7b;
- nightly (Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7b) Gecko/20040405)

With nightly the ~/.mozilla was moved out of the way before test.

To reproduce the problem using "Spool file ..." attached:
- create new Movemail account in Mozilla using all defaults;
- decompress the file and place it into /var/spool/mail under corresponding user
name;
- click "GetMsgs" in Mozilla

Mozilla here shows single new message in Inbox.

mailx shows 5 messages: 4 messages with Subject lines in Win1251 encoding and
last message with Unicode subject line (the one which is actually shown by Mozilla)
Severity: major → critical
(Assignee)

Comment 3

15 years ago
I can confirm that this is occurring on a trunk build from today.
Status: UNCONFIRMED → NEW
Ever confirmed: true
(Assignee)

Comment 4

15 years ago
I noticed that there are embedded null characters contained within the sample
spool file - these have caused problems with Mozilla/Movemail in the past. I'm
trying to track down the bug where this problem was supposedly fixed.
(Assignee)

Comment 5

15 years ago
bienvenu: Didn't you work on a bug dealing with embedded null characters before?

Comment 6

15 years ago
yes, I did, but for pop3 and imap, not for movemail. The movemail code probably
needs a similar kind of fix.
(Assignee)

Comment 7

15 years ago
I believe this may be a duplicate of Bug 115179.
(Assignee)

Comment 8

15 years ago
I have a patch I'm working on which uses dougt's suggestion from Bug 115179
(http://bugzilla.mozilla.org/show_bug.cgi?id=115179#c11). In early testing it
appears to fix the issues with null characters.
(Assignee)

Comment 9

15 years ago
Created attachment 145717 [details] [diff] [review]
Patch v1 (cvs diff -uw8)

This patch does a lot of cleanup for the Movemail code, including switching it
to use the nsILineInputStream interface for reading the file, which fixes the
errors with reading null characters in the mail spool files.

Summary of changes:
- Where possible, replaces usage of nsFileSpec with nsILocalFile.
- Splits off the finding of the mail spool file into a separate function for
readability.
- Switches from using nsInputFileStream for parsing of the file to
nsIInputStream/nsILineInputStream.
- Removes allocation of the buffer used in prior reading of file, uses nsRefPtr
for automatic deletion of newMailParser, and moves the locking of the mail
spool until right before it is read. This allows the removal of the
"freebuff_and_unlock" goto.
- Uses nsIFile's SetFileSize instead of nsFileSpec's Truncate function for
truncating the mail spool.

I have tested this code thoroughly on the provided mail spool and movemail is
now working properly with this fix.
(Assignee)

Comment 10

15 years ago
Comment on attachment 145717 [details] [diff] [review]
Patch v1 (cvs diff -uw8)

Seth - can you please look over these changes?
Attachment #145717 - Flags: review?(sspitzer)

Comment 11

15 years ago
Comment on attachment 145717 [details] [diff] [review]
Patch v1 (cvs diff -uw8)

Seth's busy doing non moz stuff these days. Taking over and sr'ing the patch.

I'm happy to have an active module owner for movemail. Go for it Philip.

I'm trusting you to of course test these changes. I don't have the ability to.
Attachment #145717 - Flags: review?(sspitzer) → superreview+
(Assignee)

Comment 12

15 years ago
Should this change wait for 1.8a or should we try and get this in for 1.7? I am 
not sure how often mail spools have null characters embedded in them, but this 
is crucial in those situations to prevent the loss of mail.
Status: NEW → ASSIGNED

Comment 13

15 years ago
Comment on attachment 145717 [details] [diff] [review]
Patch v1 (cvs diff -uw8)

I think we should try for 1.7 since it does cause data loss.
Attachment #145717 - Flags: review+
Attachment #145717 - Flags: approval1.7?

Comment 14

15 years ago
Comment on attachment 145717 [details] [diff] [review]
Patch v1 (cvs diff -uw8)

a=chofmann for 1.7
Attachment #145717 - Flags: approval1.7? → approval1.7+
(Assignee)

Comment 15

15 years ago
Fixed.

Checking in nsMovemailService.cpp;
/cvsroot/mozilla/mailnews/local/src/nsMovemailService.cpp,v  <-- 
nsMovemailService.cpp
new revision: 1.42; previous revision: 1.41
done
Status: ASSIGNED → RESOLVED
Last Resolved: 15 years ago
Resolution: --- → FIXED

Updated

14 years ago
Blocks: 248121

Comment 16

14 years ago
Comment on attachment 145717 [details] [diff] [review]
Patch v1 (cvs diff -uw8)

Bug 260488 appears to be a TB filing of the same bug -- I'm assuming this
hasn't made it into aviary yet...?
Attachment #145717 - Flags: approval-aviary?

Comment 17

14 years ago
Comment on attachment 145717 [details] [diff] [review]
Patch v1 (cvs diff -uw8)

a=asa for aviary checkin.
Attachment #145717 - Flags: approval-aviary? → approval-aviary+

Comment 18

14 years ago
I don't believe this is related to bug 260488 - this fix went in long before the
aviary branch was made, and is a regression between .7.3 and .8
Product: MailNews → Core
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.