576235 - Email messages sometimes glued together

Status: RESOLVED INCOMPLETE

(MailNews Core :: Networking: IMAP, defect)

Description

[vile]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-us) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.10) Gecko/20100512 Thunderbird/3.0.5

Some of the emails from the same google mail account retrieved through IMAP end up glued together. I.e. end part of the first email message is discarded and replaced with contents of the another message, starting with the headers (Delivered-to...).
Both email messages on server are separate and perfectly accessible through other email clients (apple mail, outlook, webmail). 

Reproducible: Sometimes

Steps to Reproduce:
I don't have clear steps to reproduce this problem, only two different installations affected so far and I haven't noticed any similarities between affected e-mail messages (except that they are received in the sequence).
Actual Results:  
Can't reproduce in stable way.
Email messages are glued together.

Expected Results:  
Keep email messages separate.

Message source for the first affected message (in thunderbird) does not contain any mail server headers.

Such behavior introduces significant security risks, especially for cases when large email message is forwarded without reading the whole e-mail.

Example of such message (copied from Ctrl-U window):
<--------------  start --------------->
Congratulations! You're about to evaluate Qt - the cross-platform application and UI framework used to develop world-leading applications such as Skype®, Adobe® Photoshop® Elements, and Google® Earth.

GETTING STARTED
---------------
1.      Choose either the standalone Qt evaluation, or the Qt SDK, or both!
2.      Use the links below to download your evaluation
3.      Install the evaluation package of your choice
4.      Start using Qt!

GET THE MOST OUT OF YOUR EVALUATION
-----------------------------------
1.             Try the Qt SDK - The Qt SDK is a new way to get up and running with your Qt evaluation faster than ever before.  The Qt SDK includes Qt, the new Qt Creator IDE and the Qt development tools.  You can download it below.

2.      Dive into the rich Qt documentation - You have full access to extensive Qt  documentation and resources, available within the Qt Assistant tool included in your evaluation, and online at http://doc.qt.nokia.com/.

3.      Explore the Qt community - You can get in touch with other Qt users via the Qt-interest mailing list (http://lists.qt.nokia.com/qt-interest/) or at the main Qt community site, Qt Centre (http://www.qtcentre.org/).

4.      Write your own code - The best way to experience Qt is to start using it. Modifying and running the examples and tutorials found at http://doc.qt.nokia.com/examples.html#qt-tutorial is a great way to discover what's possible with Qt.

TERMS OF YOUR EVALUATION
------------------------
Your evaluation allows you to try Qt for the next 30 days, ending on 2010-07-31. The evaluation version contains all the functionality of Qt, but is time limited.

Please note that usage is restricted to evaluation only. See the User Licensing Agreement included in each package for details.

WHERE TO DOWNLOAD YOUR EVALUATION
---------------------------------

Qt Standalone Eval:
-------------------
Windows - MS Visual Studio 2003:
http://dist.qt.nokia.com/developer/download/qt-win-commercial-4.6.3-vs2003.exe

Windows - MS Visual Delivered-To: [e-mail]
Received: by 10.216.184.19 with SMTP id r19cs2975wem;
        Wed, 30 Jun 2010 22:02:51 -0700 (PDT)
Received: by 10.204.115.204 with SMTP id j12mr6998434bkq.132.1277960569683;
        Wed, 30 Jun 2010 22:02:49 -0700 (PDT)
Return-Path: <xxx@xxx>
Received: from mail-fx0-f51.google.com (mail-fx0-f51.google.com [209.85.161.51])
        by mx.google.com with ESMTP id w9si16286114bkx.0.2010.06.30.22.02.49;
        Wed, 30 Jun 2010 22:02:49 -0700 (PDT)
Received-SPF: neutral (google.com: 209.85.161.51 is neither permitted nor denied by best guess record for domain of xxx@xxx) client-ip=209.85.161.51;
Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.161.51 is neither permitted nor denied by best guess record for domain of xxx@xxx) smtp.mail=xxx@xxx
Received: by fxm15 with SMTP id 15so937035fxm.10
        for <[e-mail]>; Wed, 30 Jun 2010 22:02:49 -0700 (PDT)
Received: by 10.87.40.32 with SMTP id s32mr14417751fgj.21.1277960569277;
        Wed, 30 Jun 2010 22:02:49 -0700 (PDT)
Return-Path: <xxx@xxx>
Received: from amd-server (ip-83-147-178-14.lmk.metro.digiweb.ie [83.147.178.14])
        by mx.google.com with ESMTPS id 1sm9808265fkt.11.2010.06.30.22.02.48
        (version=SSLv3 cipher=RC4-MD5);
        Wed, 30 Jun 2010 22:02:49 -0700 (PDT)
Message-ID: <[e-mail]>
Date: Thu, 01 Jul 2010 06:03:05 +0100
From: <xxx@xxx>
To: [e-mail]
Subject: some subject here
X-Mailer: <Avensoft Mailer 7 051205-064511>

e-mail text here
<--- Text of the second e-mail omitted --->

Comment 1

[Wayne Mery (:wsmwk)]

Does it happen with v3.1?
If so, please 
* start thunderbird with imap:5 logging https://wiki.mozilla.org/MailNews:Logging
* clear disk cache (Options|Advanced|Network&Disk|Disk Space|Clear now)
* access the affected messages
Report results

Comment 2

[Ludovic Hirlimann [:Usul]]

rebuilding the index on your folder might also help. What size on disk is your inbox ?

Comment 3

[vile]

I'm trying to reproduce it with 3.1, which is quite difficult, because this bug only occurs from time to time.
I have deleted and re-created account and tried to access affected message, but it was working correctly (of course it would, as i had no messages for account on the local storage). I will update this bug as soon as this happens again.

> rebuilding the index on your folder might also help. What size on disk is your inbox ?
My inbox for this account is ~ 1.5 GB. My colleague, who have this problem more often, has 2 inboxes ~5 GB each.

Comment 4

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #0)
> Gecko/20100512 Thunderbird/3.0.5
> Some of the emails from the same google mail account retrieved through IMAP end up glued together.
(In reply to comment #3)
> My colleague, who have this problem more often, has 2 inboxes ~5 GB each.

Bug 532323 is fixed by Tb 3.1.0. So, Bug 537498 can occur with Tb 3.0.x.
If wraparound of offset in offline-store file occurs, shown mail data becomes irrelevant one to the mail.

> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-us)
> My inbox for this account is ~ 1.5 GB.

Limitation of file size of offline-store on Mac was 2GB before fix of Bug 532323.

Comment 5

[vile]

Got this bug reproduced (Win7/Tb 3.1 RC2). There is a log file ~500 MB, what kind of information required?
(I can't upload whole log file, as this will violate company policies/NDA)

Comment 6

[vile]

Here is most suspiciously-looking part of log (message at the begin was glued together with data from next message):
-------snip ----
6824[5844340]: ReadNextLine [stream=999e940 nb=2 needmore=0]
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:CreateNewLineFromSocket:
6824[5844340]: ReadNextLine [stream=999e940 nb=6 needmore=0]
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:CreateNewLineFromSocket: <ul>
6824[5844340]: ReadNextLine [stream=999e940 nb=100 needmore=0]
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:CreateNewLineFromSocket: <li>Author: Sergey Sobolev</li>
BODY[HEADER.FIELDS (Content-Type Content-Transfer-Encoding)] {89}
6824[5844340]: PARSER: CR/LF fell on chunk boundary.
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:STREAM:OPEN Size: 0: Begin Message Download Stream
6824[5844340]: ReadNextLine [stream=999e940 nb=38 needmore=0]
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:CreateNewLineFromSocket: Content-Type: multipart/alternative;
6824[5844340]: ReadNextLine [stream=999e940 nb=49 needmore=0]
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:CreateNewLineFromSocket:  boundary=mimepart_4c4eb3365d289_7e94e39f93c6e5
6824[5844340]: ReadNextLine [stream=999e940 nb=2 needmore=0]
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:CreateNewLineFromSocket:
6824[5844340]: ReadNextLine [stream=999e940 nb=3 needmore=0]
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:CreateNewLineFromSocket: )
6824[5844340]: ReadNextLine [stream=999e940 nb=15 needmore=0]
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:CreateNewLineFromSocket: 71 OK Success
0[2527140]: considering playing queued url:imap://[e-mail]@imap.gmail.com:993/fetch>UID>/INBOX>6193
0[2527140]: creating protocol instance to play queued url:imap://[e-mail]@imap.gmail.com:993/fetch>UID>/INBOX>6193
0[2527140]: playing queued url:imap://[e-mail]@imap.gmail.com:993/fetch>UID>/INBOX>6193
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:ProcessCurrentURL: entering
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:ProcessCurrentURL:imap://[e-mail]:993/fetch%3EUID%3E/INBOX%3E6193:  = currentUrl
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:SendData: 72 UID fetch 6193 (UID RFC822.SIZE BODY.PEEK[])
6824[5844340]: ReadNextLine [stream=999e940 nb=61 needmore=0]
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:CreateNewLineFromSocket: * 5606 FETCH (UID 6193 RFC822.SIZE 4375181 BODY[] {4375181}
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:STREAM:OPEN Size: 4375181: Begin Message Download Stream
6824[5844340]: ReadNextLine [stream=999e940 nb=41 needmore=0]
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:CreateNewLineFromSocket: Delivered-To: [e-mail]
6824[5844340]: ReadNextLine [stream=999e940 nb=54 needmore=0]
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:CreateNewLineFromSocket: Received: by 10.216.228.1 with SMTP id e1cs81409weq;
6824[5844340]: ReadNextLine [stream=999e940 nb=47 needmore=0]
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:CreateNewLineFromSocket:         Mon, 15 Mar 2010 03:18:54 -0700 (PDT)
6824[5844340]: ReadNextLine [stream=999e940 nb=72 needmore=0]
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:CreateNewLineFromSocket: Received: by 10.223.7.91 with SMTP id c27mr679786fac.19.1268648332750;
6824[5844340]: ReadNextLine [stream=999e940 nb=47 needmore=0]
6824[5844340]: 5c54800:imap.gmail.com:S-INBOX:CreateNewLineFromSocket:         Mon, 15 Mar 2010 03:18:52 -0700 (PDT)
---- snip ----

Comment 7

[Wayne Mery (:wsmwk)]

suspect dup.

Comment 8

[Emanuele Bellini]

Attachment: Example of e-mail messages merged together

Comment 9

[Emanuele Bellini]

I've got this issue, too. Opening an e-mail message, its source is shown together with the next one. Double-clicking on the second message in the inbox, I've got it correctly shown.
The attachment is a print of the tab where I should view the content.
Of course, in other e-mail clients I haven't got this issue.

Comment 10

[Wayne Mery (:wsmwk)]

Gene, both users no longer use Thunderbird.

Is there enough information in comment 6 to say whether this issue likely no longer exists?

Comment 11

[gene smith]

Wayne, the log snippet in comment 6 looks pretty normal. Can't tell much about it. Have never seen msgs "glued" together like described. The attachment isn't real helpful either since it is doesn't seem to be a complete .eml file (e.g., no header).

Comment 12

[Wayne Mery (:wsmwk)]

Thanks. In that case, we don't have enough here to be actionable