405437 - IMAP mail are not cached for offline use if message size is bigger than mail.imap.mime_parts_on_demand_threshold

Status: RESOLVED WORKSFORME

(MailNews Core :: Networking: IMAP, defect)

Description

[Patrick Chaleat]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9

In IMAP mode, all messages bigger than mail.imap.mime_parts_on_demand_threshold are not cached. They are not readable in off line mode and every time you read them, a new network transfert is done

Reproducible: Always

Steps to Reproduce:
Put some messages in an IMAP folder with config for have messages available for offline mode. 
Some smaller than mail.imap.mime_parts_on_demand_threshold, and some bigger...
In online mode, read them.
Go into offline mode
Try to read the messages.
Actual Results:  
You can read only messages smaller than mail.imap.mime_parts_on_demand_threshold

Expected Results:  
You can read all the messages

You can also see the problem staying in online mode with network analyser...
If you choose a big value for  mail.imap.mime_parts_on_demand_threshold or if you put mail.imap.mime_parts_on_demand false, the caching is good, but we have an other bug: If you clic on a message (with message pane, without open a new window) and then clic to another message before complete transfert, you can never see the message, the cache is broken for the interrupted message.

Comment 1

[Patrick Chaleat]

I made a mistake for the buid identifier.
It is Thundebird version 2.0.0.9 (20071031) (and not firefox...)

Comment 2

[Patrick Chaleat]

This problem occurs only if the message has an attachement. Without attachement, the cache is ok, even if the message is bigger than mime_parts_on_demand_threshold

Comment 3

[Patrick Chaleat]

To avoid this problem, mail.imap.mime_parts_on_demand=false has no effect.
mail.server.default.mime_parts_on_demand=false is effective.
mail.imap.mime_parts_on_demand_threshold with a value bigger than the bigger message a also an effect.

Comment 4

[David :Bienvenu]

This is because if we don't fetch the whole message, we don't store it for offline use. If the part is not inline, then we leave it on the server when we fetch the message. I suppose you could argue that the have messages available for offline use setting should trump the mime parts on demand setting, in the world of high speed internet connections. 

Comment 5

[Patrick Chaleat]

Yes but when the message and the attachment have been downloaded one time, it must  be put in the cache !
The mime part on demand is not bad, it's not the problem. What is bad is that when the message and the attachment has been already downloaded in several parts, the cache doesn't keep it.
We have 70000 clients on slow network (2mb/s for 200 users). We need an effective cache. 

Precision:
Of course, for the tests, when you go offline, you should not choose to "download the messages for offline use". The goal is to see if the previous read messages are already in the cache...

Comment 6

[David :Bienvenu]

I'm not sure I understand - were all the parts inline (e.g., images), so that the whole message was downloaded? Or did you download the attachment separately, by saving it to disk or opening it?

Comment 7

[Patrick Chaleat]

I explain again (sorry for my poor English)
We receive a message in a folder marked to be usable in a offline mode, with the default setting for mime part on demand.
The message have attachment and the size is bigger than mail.imap.mime_parts_on_demand_threshold, so only the body is downloaded (attachment is not visible inline, a pdf for example). No problem.
Then the user copy or read the attachment. So the download of all the message occurs. No problem.
Later, the user wants to read again this mail already downloaded in a folder marker to be usable in a offline mode (so with a cache). The problem is that the attachment is downloaded again, the previous download was not put in the cache.

Comment 8

[David :Bienvenu]

When you read the attachment, we don't download the whole message - we just fetch the mime part that corresponds to that attachment from the server, using the imap protocol that allows us to fetch a single part. Or at least that's the way it's supposed to work.

Comment 9

[Patrick Chaleat]

I understand what is "mime part on demand". 
What we need is not to download again what have already been downloaded, even if the download have been done in several parts. We need an effective imap cache, even with "mime part on demand". 
Perhaps it is not easily possible due to technical limitations (cache structure ?)

Comment 10

[Gerry Healy]

I'm developing an extension called Incommunicado to make TB more friendly in
unreliable/low-bandwidth (or pay per KB) conditions. More info here:

  https://wiki.mozilla.org/MailNews:Minimizing_Bandwidth_Usage

The objective of the extension could be described as:

  "Only download what I ask for and only download it once."

Unfortunately to achieve this behaviour while supporting mime_parts_on_demand is non-trivial, but I hope to tackle this early 2009.

Comment 11

[Magnus Melin [:mkmelin]]

I just tested this on today's linux nightly, WFM now.