Closed Bug 468637 Opened 11 years ago Closed 9 years ago

Message body not downloaded, getting "This body part will be downloaded on demand." in synch data.

Categories

(MailNews Core :: Backend, defect, critical)

1.9.1 Branch
defect
Not set
critical

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: baffoni, Unassigned)

References

Details

(Keywords: dataloss)

Attachments

(1 file)

Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.1b3pre) Gecko/20081204 Thunderbird/3.0b1

I have a curious error on the beta1 release.  I'm getting the "This body part will be downloaded on demand." message on an email with an attachment.  Unlike the times I've seen this in TB2 where I get nothing from the message except the body part with that error, this email actually shows the attachment (openable/launchable), but not the body of the email.   Here is the relevant part of the show source:

------_=_NextPart_001_01C95A17.3F240F9C
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_002_01C95A17.3F240F9C"

------_=_NextPart_002_01C95A17.3F240F9C
X-Mozilla-IMAP-Part: 1.1
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

This body part will be downloaded on demand.
------_=_NextPart_002_01C95A17.3F240F9C
X-Mozilla-IMAP-Part: 1.2
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

This body part will be downloaded on demand.
------_=_NextPart_002_01C95A17.3F240F9C--
------_=_NextPart_001_01C95A17.3F240F9C
Content-Type: application/octet-stream;
	name="Spt Upgrade.pdf"
Content-Transfer-Encoding: base64
Content-Description: Spt Upgrade.pdf
Content-Disposition: attachment;
	filename="Spt Upgrade.pdf"

Unselecting the message and back does not download the body part, nor does going offline and online again, nor does closing the client and reopening.  This occurs in safe mode as well as regular mode.  There are no errors in the console log when reading the message, nor were there on first view of the message.

More curious:  when I first read the email, I could see the message body correctly.  After opening the attachment, then clicking on another email, then going back to the original email did I get the message body not downloaded error (not saving to cache correctly?).

This is a first time issue for this error.  This in in the INBOX, and the inbox is selected for offline replication.  If this is like similar issues in TB2, I'm sure that doing a reindex (online or offline) which would force a full download of all message headers and bodies would fix this temporarily.  This looks like a legacy issue of losing the cached body part of the message on write to disk, and/or not doing proper download validation or marking of invalid download and marking for redownload of the message/bodypart.   At least in this instance we are getting the notification that the body has not been downloaded and there is some content in the download cache; in TB2, we would just get a blank window without even visible header information in anything except the message list (I suppose the message download was completely gone and all that existed was the index entry).
David, does this need a log?  Or are there other things to check first?

could be a dupe, depending.
not dataloss, the data is still there.
Component: Mail Window Front End → Backend
Keywords: dataloss
Product: Thunderbird → MailNews Core
QA Contact: front-end → backend
Version: Trunk → 1.8 Branch
it sounds like that text actually made it into the offline store, which should not happen. If so, logging won't help because no imap protocol will be issued for that message. It would be helpful if Michael could look at the offline store with a text editor and see if that text is in the offline store. If so, you could try copying the message to an other imap folder, but one not configured for offline use, and then clicking on the message in the new folder to see if it has the same problem (if so, doing this with imap logging turned on might be useful).
Well, dang, due bug 468248 I lost track of the message (I highlighted with another color), I'll see if I have a backup of it somewhere.
Version: 1.8 Branch → 1.9.1 Branch
Duplicate of this bug: 491088
Making this more descriptive, easier to search.

I'll follow the instructions in comment 2 on Monday, and post results.
Summary: Message body not downloaded, TB3b1 → Message body not downloaded, getting "This body part will be downloaded on demand." in synch data.
OK, the original file I was looking at was inadvertently deleted, so I now have a new test case email.  Looking in the offline storage, it does indeed have that text in the offline cache:

---------------------8<------------------------
From - Fri Jul 24 17:33:59 2009
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Return-Path: <millerr@avinc.com>
Received: from [x.x.x.x] ([x.x.x.x]) by
          hermes.aerovironment.com; Fri, 24 Jul 2009
          16:53:13 -0700 
Message-ID: <4A6A4989.5010007@avinc.com>
Date: Fri, 24 Jul 2009 16:53:45 -0700
From: "Miller, Rebecca" <millerr@avinc.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.17) Gecko/20080829 SeaMonkey/1.1.12
MIME-Version: 1.0
To: "Baffoni Michael" <baffoni@avinc.com>
Subject: Options for Alfred's desktop
Content-Type: multipart/mixed;
 boundary="------------040308000003080503090800"

--------------040308000003080503090800
Content-Type: multipart/alternative;
 boundary="------------000605040602030409020509"

--------------000605040602030409020509
X-Mozilla-IMAP-Part: 1.1
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

This body part will be downloaded on demand.

---------------------8<------------------------

Moving the message to another folder that does not have an offline setting configured also shows the body part message.  Rebuilding the index of that new folder containing the message fixes the message display.  

Interestingly: if I then set my mail client from "move to trash folder" to "mark deleted", and look at the deleted message in the inbox, now I can see the contents of the message.  I confirm that the message body contents are now in the offline cache file for _inbox_ just by viewing the message in the other, uncached, mail folder.

Lastly, is this the same issue as Bug 246415 and should be dupped (to/from it), or should this stay it's own bug?
I think the difference between this bug and bug 246415 is that you see the "downloaded on demand" message as the actual content of the offline copy, therefore I'd say this is a different issue than the direct display from the IMAP server apparently discussed in the other bug.
Duplicate of this bug: 507223
Confirming as i've had this happen for a few messages within a week on two different computers.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Keywords: dataloss
OS: Windows Server 2003 → All
Hardware: x86 → All
Partial mail data in offline-store is easily be observed with test mail for Bug 501253.

[Steps to produce]

(1) Upload 2 copies of HTML mail attached to Bug 501253
    to an IMAP mail folder (say folder-1, say mail-A, mail-B. 175K each)
(2) Auto-Sync=On, offline-use=on : folder-1 only.
    Disable mail check at startup, every NNN minutes.
    View/Message Body As/original HTML
    View/Display Attachments Inline : Disabled
    Mail display : re-use existing window
    (mail is opened in stand-alone window by double-click)
(3) Delete folder-1, folder-1.msf
(4) Restart Tb, click a folder(force login) => folder-1 appears
(5) Click folder-1 => 2 mails appears
(6) Immediately double click mail-A
(7) While "This body part will be downloaded on demand." is displayed,
    and while JPEG is displayed in attachment pane(for <img> in HTML),
    double-click the JPEG attachment => dialog to open(I set IrfanView for JPEG)
    => Display JPEG or cancel
    => 20KB data is written in folder-1 (for mail-A, partial, for JPEG)  
(8) View mail-B => "This body part will be downloaded on demand."
    Wait for auto-sync completion => folder-1 : 195 KB (20KB +175KB)
    Click other mail, click mail-B again => Normally displayed.
Attached mail folder data is back up of folder-1 at this step.
  mail-A : partial, for JPEG
  mail-B : full mail data

Mail structure is as follows. Quirks for such structure is relevant?
 multipart/mixed
   part-1 : multipart/related
     part-1-1 : multipart/related
       part-1-1 : multipart/alternative
         part-1-1-1 : text/plain  (base64)
         part-1-1-2 : text/html   (base64) <img=...> points part-1-2 
     part-1-2 : image/jpeg
   part-2 : application/pdf
   part-3 : text/plain

To all problem reporters:
Simple mail structure? Or not-so-well formed mail like above?
Please ignore next in above structure. (part-1-1 = multipart/alternative)
>     part-1-1 : multipart/related
For one msg

------=_NextPart_000_0007_01CA0F79.E3FE3E30
Content-Type: multipart/alternative;
	boundary="----=_NextPart_001_0008_01CA0F79.E3FE3E30"

------=_NextPart_001_0008_01CA0F79.E3FE3E30
X-Mozilla-IMAP-Part: 1.1
Content-Type: text/plain;
	charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable

This body part will be downloaded on demand.
------=_NextPart_001_0008_01CA0F79.E3FE3E30
X-Mozilla-IMAP-Part: 1.2
Content-Type: text/html;
	charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable

This body part will be downloaded on demand.
------=_NextPart_001_0008_01CA0F79.E3FE3E30--
------=_NextPart_000_0007_01CA0F79.E3FE3E30
Content-Type: application/msword;
	name="SUOSTUMU1.doc"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
	filename="SUOSTUMU1.doc"

0M8R4KGxGuEAAAAAAAAAAAAAAAAAAAAAPgADAP7/CQAGAAAAAAAAAAAAAAA
.................

-----

Not sure about the other one.
(In addition to comment #10)

If "double-click of attachment" at step (7) of comment #10 is changed from JPEG to PDF, mail data in offile-store for mail-A becomes data for PDF. 
> (Modified step 7)  
> (7-X) While "This body part will be downloaded on demand." is displayed,
>     and while JPEG/PDF/Part 1.3 are displayed in attachment pane,
>     double-click PDF attachment => dialog to open by Adobe Reader
>     => 134KB data is written in folder-1 (for mail-A, partial, for PDF)  
>        Data of all parts other than PDF part is ;
>           This body part will be downloaded on demand.
>     Note: After a while, 175KB data for mail-B is added by Auto-Sync.

Why download of each part by mail.imap.mime_parts_on_demand=true writes data(always partial mail data) into offline-store instead of Cache directory?

To all problem reporters:

Did you try to view attachment(real attachment=PDF, Word document, or faked attachment=image which is not used by HTML in multipart/related) before completion of download of whole mail data by auto-sync?
Questions about behavior at step (8) of Comment #10.

After FETCH of mail headers of new mail, "This body part will be downloaded on demand." is continuously displayed as mail body even though "View/Message Body AS/Original HTML", until mail is re-displayed after whole mail is downloaded into offline-store by auto-sync. 
This behaviour is not observed for IMAP folder of offline-store=off. Although incomplete display of HTML can be observed temporarily due to complex/not-so-well structure, HTML mail is automatically rendered as expected sooner or later.

Why behaviour is different between offline-use=yes and offline-use=off?
Why "This body part will be downloaded on demand." is displayed while "FETCH of mail headers" completed but whole mail download into offline-store is not completed yet?
Mismatch between (a) features for mail data download such as auto-sync, mime_parts_on_demand, and (b) features(quirks) for mail of complex or not-so-well structure?
(In reply to comment #13)
> Did you try to view attachment(real attachment=PDF, Word document, or faked
> attachment=image which is not used by HTML in multipart/related) before
> completion of download of whole mail data by auto-sync?

Possibly, can't recall.
No longer blocks: 468595
Depends on: 468595
I'm experiencing this behaviour in TB 3.0 beta 3
I am also experiencing this.  Offline IMAP.  TB 3.0 beta 3.
Bug 468595 is already fixed. (See Bug 468595 Comment #55 for checkin id)
Can you reproduce your problem with newest trunk nightly build?
> http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-comm-1.9.1/
> http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-comm-1.9.1-l10n/
> If MS Win, download win32.zip build. You can test by UNZIP only.
Same here, with the latest beta 3 (shredder).
Windows XP Pro SP3, gmail app account in SSL/TLS, port 993, IMAP. The problem occurs inline or offline for me.

There is HTML and an image in my signature.

When I have this bug I can see the image and the mail like attachments in the mail. 
The status "Loading message..." never ends.

A reply still shows "This body part will be downloaded on demand. ". But a forward or "edit as new" show the whole mails.

HTH.
(In reply to comment #19)
Same phenomenon as or similar phenomenon to bug 516211?
Same mail structure or similar mail structure to test mail of bug 516211?
if truly dataloss, then sev=major or critical
Severity: normal → critical
I'm no longer seeing this in the offline store, probably because of the size sanity check I added, so it's not really critical data loss, I don't think...
I'm seeing this in 3.1b1, on a mail that works fine in 3.0.x. Does that up the priority here again?

(And do you care to see the email?)
FYI, I'm still seeing this in 3.0.4 (Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4), however it seems to mostly stop happening if the folder is set for offline sync.
Haven't seen this since my April report, FWIW.
I have seen this a couple times in the last two months, but to be fair, the users may not have been running the latest versions.  I'll keep an eye out on this.
I've been running latest nightly for quite some time now (over a year) and haven't encountered this. Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.14pre) Gecko/20110105 Lanikai/3.1.8pre
I experienced this problem in the past, but not any more for quite some time now. Using up-to-date versions supplied by Fedora. So I guess this might have been solved somehow meanwhile.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.