Closed Bug 546095 Opened 16 years ago Closed 15 years ago

Behaviour with IMAP attachments utterly broken

Categories

(MailNews Core :: Networking: IMAP, defect)

Product:
MailNews Core
Component:
Networking: IMAP
Platform:
x86
Windows XP
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 543076

People

(Reporter: frank.vondelft, Unassigned)

Details

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729) Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1 TB's behaviour with IMAP attachments is a complete disaster. It downloads attachments again and again, and routinely corrupts its downloaded file headers. This is carnage for the frequent traveller who is offline a lot, and routinely has slow connections. Reproducible: Always Steps to Reproduce: 1. Send a large attachment to your IMAP account. 2. Connect via a slow connection (e.g. airport), click on the message, and watch it download. 3. Click on another (small) message, then click back to your large message. Watch it download again... 4. Double-click on the attachment (to view). Watch it download YET again. 5. Click on another message, disconnect from the internet, and click back to the large message. Now it says "message body not downloaded" (or such like) 6. Connect back to the internet (fast now, if you like). Click on the message and double-click (=open) the attachment: the attachment is now corrupted (90% of time). Actual Results: (See above) Expected Results: Here's what should happen by default: 1) When message first appears, download only the body (all of it), but NOT the attachment; only show its file-size. 2) Download the attachment ONLY when I explicitly ask for it, and allow me to cancel the download, e.g. in the download manager. 3) Leave downloaded attachments in the cache, and for ever after, ALWAYS get it from the cache, whether I'm off- or online. Only download again if I *explicitly* ask for it (e.g. a "re-download" option on the right-click menu).
(In reply to comment #0) > Expected Results: > Here's what should happen by default: > 1) When message first appears, download only the body (all of it), but NOT the > attachment; only show its file-size. (Q1) Tb doesn't have capability to display file size of attachment. Do you use add-on for it? "downloads attachments again and again" by add-on is already reported. Dup of 543076? (Q2) I assume Inbox of the IMAP account. "offline use"=On? Or Off? Check Folder Properties/Synchronization of Inbox. (Q3) What is your View setting? View/Message Body As : Original HTML, Simple HTML, Plain Text View/Attachments Inline : Enabled or Disabled (Q4) HTML mail with embedded images and attachments? Text mail with attachments? In bothe cases, what is mail structure? Simple multipart/mixed or multipart/related or multipat/alternative? Or multipart/xxx are nested? (Q5) If attachment parts in multipart/mixed is involved, displayable(text or image)? Or non-displayable(ZIP, PDF, DOC, ...)? > Expected Results: Tb3 roughly behaves as you say, although Tb doesn't have capability of enhanced behaviour you want, and although "download twice" was observed for attachment when "offline use"=OFF and "Display Attachmens Inline"=OFF and special mail structure. Read Bug 516211 Comment #13 and see attached IMAP log to Bug 516211 Comment #14.
Before I answer the questions: I was trying to define what should be default behaviour, because nothing else makes sense. The behaviour I describe is in the default installation (except add-on). If TB3 is *meant* to behave like this, then this is definitely, absolutely broken -- because it does not do this at all. Take your laptop to the nearest airport and give it a try... (A1) The add-on is called "Attachment Sizes". This really should be incorporated into trunk: file size is *critical* when connections are slow. (A2) Offline = On (the TB3 default). (A3) "Original HTML", "View Inline"=enabled (TB3 defaults) (A4) All attachments behave the same. Embedded text or html is usually small and no problem; the large doc/ppt/wmv are the issue. (A5) Both, but see above. (I don't usually check.) Btw, this happens *all* the time, not just in a few cases.
(In reply to comment #2) > If TB3 is *meant* to behave like this, then this is definitely, absolutely > broken -- because it does not do this at all. Have you read Bug 516211 Comment #13 and seen IMAP log attached to Bug 516211 Comment #14? Really broken at somewhere, but basic behaviour when "offline use=off" is next as you can see in the IMAP log I attached to that bug. Download body only into Disk Cache initially. Download image parts for embeded image into Disk Cache too. Upon double click of attachment, part for the attachment is downloaded, and saved into Disk Cache. (the log is for issue of "fetched twice") If "offline use=on", auto-sync works and whole mail data is saved in offline-store which is completely different from Disk Cache. Please don't confuse offline-store and Disk Cache. > (A2) Offline = On (the TB3 default). Show "Activity Manager" window. Was message like "Inbox is synchronized" displayed before you click mail of large attachment and next small mail? Probably "No". > (A1) The add-on is called "Attachment Sizes". Sorry, I failed to linkify. Same problem as bug 543076(INVALID) probably occurs in your case. If mail is clicked before download completion of whole mail data into offline-store, Tb3 behaves as if "offline use=off" for the mail, and mail data of several parts only is fetched into Disk Cache. So, bug 543076 occurs untill whole mail data is downloaded into "offline-store" by auto-sync for the folder of "offline use=on". Can you check with "Attachment Sizes" uninstalled? > (A3) "Original HTML", "View Inline"=enabled (TB3 defaults) > (A4) All attachments behave the same. Embedded text or html is usually small and no problem; the large doc/ppt/wmv are the issue. As "Embedded text or html is usually small", "mail body text or html" is normally downloaded shortly and saved in Disk Cache upon first mail click. And, some of parts are fetched because "View Inline"=enabled. It probably is interrupted by click of other small mail. So, attachment is re-fetched again until whole data of required parts are fetched into Disk Cache. (and as written in Bug 516211 Comment #13, fetched may be done twice) Log of Bug 516211 Comment #14 for PDF is for similar phenomenon to such case. - fetch of PDF part is interrupted by Tb himself(psuedo inturrupt). - The PDF attachment part is fetched again, due to inturuption or others. If click of other(newer) small mail(not fetched yet) is executed while auto-sync is executing fetch of whole mail data of previous mail of big attachment into offline-store, the fetch by auto-sync may be interrupted. It causes partial data in offline-store, and it probably causes re-fetch of whole mail data into offline-store by auto-sync, or it causesre-fetch of attachment parts into Disk Cache. Problem like next, and problems after that is also reported in other bug. > 2. Connect via a slow connection (e.g. airport), click on the message, and watch it download. > 3. Click on another (small) message, then click back to your large message. > Watch it download again... I'll ask you for getting data(such as IMAP log) for diagnosis by developer. Additional questions before asking data. Even if interruption occurred during fetch into Disk Cache or fetch into offline-store, auto-sync will fetch whole mail data into offline-store sooner or later. Show "Activity Manager" window. (Q6) Does your problem occur even after message like "Inbox is synchronized" is displayed at "Activity Manager" window? (Q7) Does "Download Now" of Folder Properties/Synchronization resolve happend problem? (Q8) Does "Work offline" and download for offline resolve happend problem?
(Q9) When data of a part of mail-1 is being fetched partially yet, I think click of next mail-2(which is not fetched yet too) is request of following by user. Cancel fetch of previous mail-1 and fetch and display mail-2 immediately. Is it wrong? Please note next: Even if all 5 cached connections can be used for pararell fetch of mail-1 to mail-5 in same Inbox, mail client has to kill fetch of one of mail-1 to mail-5, if user clicked mail-6 because it's request of immediate fetch and display of mail-6.
Of course, as user I do not want even to know that there is a difference between Offline Store and Disk Cache -- the thing should just work as described. (A6) I don't see any "Inbox is synchronized" lines in the activity manager (how do I increase the level of detail?) (A7) No: the folder is synchronized already. (A8) When I set "Work Offline" and click on a message with a large attachment, I get nothing at all, not even the message, only "The body of this message...". The following is repeatable: (1) If I click on a downloaded message (with big attachment) the FIRST time, all I get is the green activity meter and in the status bar: "Downloading message..." This does NOT stop until I click on another message. (2) If I click to another message and back to the attachment message, the status bar now shows (in quick succession): "Authenticating...." "Opening folder INBOX..." "Downloading message..." <clear> And then I can view the attachment. I can repeat as often as I like, it happens exactly in this sequence. It happens exactly the same if Attachment Sizes is uninstalled; the only difference is I can no longer see when the download has completed. How do I generate the logfile?
Sorry, to clarify properly: What can be repeated indefinitely (comment #5) are steps (1) and (2) in sequence: the behaviour is apparently constant.
(1) As "Attachment Sizes" causes problem of bug 543076(INVALID), please check with -safe-mode of Tb3. (2) Check next cases, please. (2-1) Create a folder(F1), set offline use=off copy a mail of large embeded iages with big attachments(.doc etc.) copy a small mail (2-2) Create a folder(F2), set offline use=on copy a mail of large embeded iages with big attachments(.doc etc.) copy a small mail (3) Clear Disk Cache (Tools/Options/Advanced/Network&Disk Space, Clear Now > Disk Cache Directory: > C:\Documents and Settings\<user>\Local Settings\Application Data\Thunderbird\Profiles\<prof_name>\Cache (4) Do your step (1), your step (2), your step (3), ... (4-1) at F1(offline use=off), with watching Disk Cache directory and checking content of files in it (4-2) at F2(offline use=on), with watching Activity Manager for, Sychronizing: <account name>) <account name> is up to date 'Total number of messages downloaded: NNN with watching "offline store" file size and checking file content For IMAP log, see next pages. > https://wiki.mozilla.org/MailNews:Logging > http://www.mozilla.org/projects/nspr/reference/html/prlog.html#25328 > SET NSPR_LOG_MODULES=timestamp,imap:5
(In reply to comment #5) > Of course, as user I do not want even to know that there is a difference between Offline Store and Disk Cache > -- the thing should just work as described. As "Disk Cache" is new with Tb3, I believe users don't know about it. (I'm a nightly tester, so I knew it) Although offline-store and auto-sync is provide by Tb2, it was for advanced/limited users only with many restrictions than one by Tb3. So, I believe users usually don't know about it. (I'm a nightly tester, so I knew it) As you know now about offline-store and Disk Cache for IMAP folder, please distinguish them, please. "Disk Cache" concept with "download on demand" of Tb is nearer to your expectation about "cacheing".
Maybe I don't have Disk Cache after all: I don't have a Cache directory in my profile... I also don't remember activating anything like it, come to think of it. As I said, I just used the defaults and installed a bunch of extensions. (I did migrate from TB2, of course.) I've done steps 1/2 (comment #7), imap.log.zip attached in next comment. I did the following: ===> * made a folder in imap root: f1-nooffline-imaptest * selected properties, unchecked "offline use" (or whatever it's called) * dragged across a large (1Mb attachment) message from imap/INBOX * dragged across a small (no attachment) message from imap/INBOX ===> * made a folder in imap root: f2-withoffline-imaptest * selected properties, unchecked and re-checked "offline use" * dragged (another) large (1.2Mb att.) message from imap/INBOX - TB3 threw an error message, something about IMAP server not found. (Timestamps respectively: 2010-01-16 06:43:12.812000 and 2010-01-16 06:43:25.500000) ===> * restarted TB3 * dragged the same large message from imap/INBOX * dragged the small message from imap/INBOX ===> shut it down. The log file is HUGE -- I hope you can make sense of it. It's gone and done something with every *single* folder at least once, and it's pulled down all messages in INBOX several times. ********************************* After shutting down TB3, I generated a log of the sequence I described in comment #5 (imap-cycle-nooffline2.zip). ===> * click on f1-nooffline-imaptest folder immediately * click on large message * click on small message * click on large message * click on small message * etc. * shut down TB3 ===> 1st time I clicked the large message, status bar said "Downloading message", finished quickly, then went into an infinite "Determining which messages to index". 2nd time: authenticated, opened, downloaded, and finished quickly 3rd time: infinite "Downloading message..." 4th time: authenticated, opened, downloaded, and finished quickly 5th time: infinite "Downloading message..." -- still going when I shut it down. ********************************* I repeated the identical steps with the "withoffline" folder (imap-cycle-withoffline.zip)
Component: Message Compose Window → Message Reader UI
The three zipped logfiles (comment #9) are here: http://www.sgc.ox.ac.uk/~loretta/tb3-imap
(In reply to comment #9) > Maybe I don't have Disk Cache after all: I don't have a Cache directory in my > profile... "Disk Cache" is controlled by; Tools/Options/Advanced/Network&Disk Space Disk Space, Use up to..., "Clear Now" You can see "Disk Cache" location by next: Enable Tools/Options/General, "Show start page" option, and set Location: about:cache?device=disk > imap.log.zip attached in next comment. As you saw, log becomes huge. You can discard log file after quick glance of IMAP flow, unless evidence of Tb's fault exists in log file. Can we see "evidence of Tb's fault" in the log file you uploaded? If so, at where in the huge file? > * click on f1-nooffline-imaptest folder immediately > * click on large message > * click on small message > * click on large message > * click on small message > ===> > 1st time I clicked the large message, status bar said "Downloading message", > finished quickly, then went into an infinite "Determining which messages to > index". > 2nd time: authenticated, opened, downloaded, and finished quickly > 3rd time: infinite "Downloading message..." > 4th time: authenticated, opened, downloaded, and finished quickly > 5th time: infinite "Downloading message..." -- still going when I shut it down. Did you test with "-safe-mode" or with "Attachment Sizes" uninstalled? What mail structure? (Save mail as .eml, edit the .eml by text editor, and delete all lines for data==keep message header lines only) > then went into an infinite "Determining which messages to index". This is message by Global Indexer. Disable it for testing of this bug, please. Tools/Options/Advance/General, uncheck "Enable Global Serarch and Indexer" > I repeated the identical steps with the "withoffline" folder What is test result? Was mail/attachment displayed as expected?
Component: Message Reader UI → Networking: IMAP
Product: Thunderbird → MailNews Core
QA Contact: message-compose → networking.imap
Closing as dup of bug 543076. If you'll find evidence that this bug was caused by Tb's fault, please re-open, with attaching the evidence to this bug.
Status: UNCONFIRMED → RESOLVED
Closed: 15 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.