1617823 - change fetch_by_chunks disable by default

Status: NEW

(MailNews Core :: Networking: IMAP, enhancement)

Description

[Alberto Salvia Novella]

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36

Steps to reproduce:

fetch_by_chunks is a feature that was created from the very beginning of Thunderbird, when the Internet was 56K and downloading attachments took a while.

But on modern Internet, even from Internet from underdeveloped countries, this feature is just a pain. When enabled the smallest email attachments take notable time to download, when it is disabled it is instant.

So my suggestion is simple, disable this by default.

Comment 1

[gene smith]

Can't say I disagree. Between fetching in chunks and fetching by parts they add a lot complexity and often just turning them off fixes user issues. Maybe turn them off by default and allow them to be switched on in Advanced server settings?

Comment 2

[Alberto Salvia Novella]

If this was my software I would remove the feature all together.

I think that keeping features "just in case" is a bad design practice, because all ends falling under that.

I only keep features that, by demonstration, are needed in non very rare cases. And keep configurability only for aspects that make practical sense, not just because someone "could" be asking for them.

There are two ways of doing things better. One is by adding more of what is truly good, the other is by removing everything else.

Because on the removal of irrelevance, the relevant becomes explicit.

Comment 3

[Wayne Mery (:wsmwk)]

(In reply to gene smith from comment #1)

Can't say I disagree. Between fetching in chunks and fetching by parts they add a lot complexity and often just turning them off fixes user issues.

https://mzl.la/3cdjV8x open bugs may be of interest. https://mzl.la/35DAGXU lists fixed bugs.

Note bug 425849 comment 2 and bug 425849 comment 4

Comment 4

[gene smith]

Yes, bug 425849 comment 4 is the only reason we do fetch by chunks, AFAIK. If you are downloading a 20M attachment over a slow link and you lose interest while waiting and want to look at another email, you just have to wait for the latest chunk to download instead of the whole thing before you can download the other message using the same connection. Often connections to imap servers are throttled down from the optimum speed so this still might be helpful in some cases (for me, Yahoo imap seem much slower than others). So I don't think we want to throw fetch in chunks out completely.
By "user issues" above I'm referring to slow download of attachments. But this is usually not an issue when the user stores messages for offline use which is default.

Comment 5

[Magnus Melin [:mkmelin]]

We also have mail.imap.mime_parts_on_demand which we just disabled by default, and increased the threshold for using it to 500KB (bug 1629292)

Comment 6

[Alberto Salvia Novella]

Perhaps slow Internet users would prefer to sync all the email all together for offline use.

Comment 7

[Magnus Melin [:mkmelin]]

One would think that's the common scenario (or us POP even).
We already use offline storage (cache) messages by default so fetching on demand should not be a big issue nowadays. [Assuming that works correctly.]

Comment 8

[Wayne Mery (:wsmwk)]

(In reply to Magnus Melin [:mkmelin] from comment #7)

fetching on demand should not be a big issue nowadays. [Assuming that works correctly.]

Except for enterprises that don't want to bloat their roaming profiles with cached messages. But then, they should also be able to flip preferences to suit their needs.

Comment 9

[Wayne Mery (:wsmwk)]

This should be greatly helped by Bug 1580480 - IMAP fetch chunk size is always 65536 bytes.

And if it isn't sufficiently helped by bug 1580480 we should investigate why. Because (going back to comment 4) we shouldn't assume everyone everywhere has fast, solid internet. Even in the US, there are many places where the best land-based service you can get is still DSL - and a dodgy one at that. (My mother was one)

Those of us with a decade of great service take it for granted, but there are entire countries with poor service/availability. And even those of us with great service at home are sometimes in hotels, coffee houses and airports with quite questionable service, on hotspots, tethered to phones, etc. Users shouldn't need to change their settings to work better when there is poor internet service - the software should adapt.

Comment 10

[Alberto Salvia Novella]

I suspect this feature isn't targeted to modern third world Internet connections, but rather to 90s 56K ones. And perhaps less than 1% of connections would benefit from it.

I would try to simulate having one of those connections, and seeing what's the difference.

Comment 11

[Wayne Mery (:wsmwk)]

It would be useful to see to what degree this helps topcrash bug 628646. And this also complicates our imap debugging. So I think it's time to implement this.

Any objections?

Comment 12

[Alberto Salvia Novella]

From me, no objection.

Comment 13

[gene smith]

(In reply to Wayne Mery (:wsmwk) from comment #11)

It would be useful to see to what degree this helps topcrash bug 628646. And this also complicates our imap debugging. So I think it's time to implement this.

Any objections?

I didn't completely re-read everything in bug 628646 but I saw a lot of mention of fetching bodystructure and parts on demand affecting the reported crash. Bodystructure and parts on demand have already been removed from the imap code (not sure if removed in 102).
So could you point to a recent crash report that occurs after bodystructure/part on demand were removed and is caused by "chunking".
Since the chunk size is "adaptive" and you have a fast connection you will usually get the whole message in one chunk. So with default prefs, you will only fetch in small chunks if your network is slow.
Also, just because a crash report contains FetchTryChunking() doesn't mean the message is being fetched in chunks since that function handles non-chunking situations too
So, at this time, I have my doubts that chunking is specifically causing a crash (but I could be wrong).