450456 - Consider turning on disk cache for TB

Status: RESOLVED FIXED

(Thunderbird :: Build Config, defect, P1)

Description

[Mark Banner (:standard8)]

Currently we disable the disk cache for TB at compile time (NECKO_DISK_CACHE in confvars.sh). Presumably this was probably done to save code size and remove something we didn't need.

gozer has just been looking at the start page stats for TB these have been showing about 1% 304 responses (unmodified page). So 99% of all requests to the start page are pulling the full page down from the server.

The 1% of requests are probably because we do actually have a memory cache present at the moment.

A disk cache would help both the server (currently approx 298505 hits per hour) and the user if they are on slow connections.

I think first we need to check:

* If we enable the disk cache, what gets stored in it?
- Is it just the start page, or do we get copies of messages etc in the cache which would make the cache useless.

I'm not too fussed about the extra code size, I doubt it will amount to much. Obviously we'd want to set the cache to a reasonable size but not too large.

Setting blocking TB 3.0 as I think we should at least do the investigation before TB 3, even if we end up not enabling the disk cache.

Comment 1

[David :Bienvenu]

Would things like rss messages, which are often basically html pages, start getting cached in the disk cache? I suspect that might be a good thing.

Comment 2

[Dan Mosedale (:dmosedale, :dmose)]

This all sounds right to me.  We should certainly investigate, and I suspect turning on the cache is likely to be the right thing.  If it _doesn't_ cause RSS pages to get cached, I suspect we'll want to fix that.

Comment 3

[rsx11m]

(In reply to comment #0)
> - Is it just the start page, or do we get copies of messages etc in the cache

This is actually what's asked for in bug 439731 for IMAP messages, unless I'm misunderstanding the topic. So, that may be solved in the process then.

Comment 4

[David :Bienvenu]

(In reply to comment #3)

> This is actually what's asked for in bug 439731 for IMAP messages, unless I'm
> misunderstanding the topic. So, that may be solved in the process then.
> 
No, each protocol gets to decide which cache(s) it's going to use and turning on the disk cache won't make imap or news or local mail use it.

Comment 5

[rsx11m]

Ok, thus the bug here (making disk cache available at compile time) would be a prerequisite for using it with IMAP or any other protocol.

Comment 6

[Joe Sabash [:JoeS1]]

In the process, please enable about:cache if possible it would be nice to know what is actually being stored there. (plugin data, & new video tag Ogg files)
BTW bumping browser.cache.memory.capacity way up to 65536 helped a lot with the problem of re-downloading image data via nttp. Most get stored in memory cache, interesting to see what effect a disc cache will have.

Comment 7

[Joe Sabash [:JoeS1]]

Thats nntp of course, and it seems that memory cache is enabled there.

Comment 8

[Mark Banner (:standard8)]

(In reply to comment #4)
> (In reply to comment #3)
> 
> > This is actually what's asked for in bug 439731 for IMAP messages, unless I'm
> > misunderstanding the topic. So, that may be solved in the process then.
> > 
> No, each protocol gets to decide which cache(s) it's going to use and turning
> on the disk cache won't make imap or news or local mail use it.

David, do you know where we do that? I think we should check what they are set to as part of enabling this.

Comment 9

[David :Bienvenu]

They're definitely set to just the memory cache for mailnews, and always have been (otherwise, SeaMonkey, which builds with the disk cache, would cache imap and nntp messages in the disk cache).

See nsImapService::GetCacheSession and nsNntpService::GetCacheSession

Comment 10

[Philippe M. Chiasson (:gozer)]

I've built myself with NECKO_DISK_CACHE=1 and NECKO_PROTOCOLS_DEFAULT += "about" and fiddled around some.

I've changed my start page url to http://www.yahoo.com/ and stop/started a bunch of times.

Then, looking at about:cache, I was able to inspect into the file cache, and notice that there were tons of cache entries, images, css, etc, and that they had been used more than once (Fecth count > 1)

Continued on by using it to view mail, retrieve attachments, browse newsgroups, and none of that content seemed to have made it to the disk cache (as I would have hoped)

Using the RSS reader was interesting, as it ended up getting good caching, external images, css, etc.

Only downside is that the feed itself ended up getting cached in there, and Standard8 pointed out correctly that the News reader keeps its own cache of it as well, so that's duplicated.

Overall, it looks like this might be a good idea. 

Comment 11

[Philippe M. Chiasson (:gozer)]

Attachment: Enable Disk Cache in Thunderbird builds

Moving this along with the actual 1-byte patch that turns this on.

Comment 12

[Mark Banner (:standard8)]

Comment on attachment 334718 [details] [diff] [review]
Enable Disk Cache in Thunderbird builds

Actually, the default is 1, so can you just remove this line instead place.

r=me with that.

It might be an idea just to check the default cache sizes/limits. I don't know what they are without looking and I haven't got time now.

Comment 13

[David :Bienvenu]

Comment on attachment 334718 [details] [diff] [review]
Enable Disk Cache in Thunderbird builds

thx, gozer

Comment 14

[David :Bienvenu]

oh, sorry, read mark's comment, removed checkin-needed

Comment 15

[Philippe M. Chiasson (:gozer)]

Attachment: [checked in] Enable Disk Cache in Thunderbird builds

Simpler to just remove NECKO_DISK_CACHE, as the default is already 1.
As per Mark's comment #12

Comment 16

[Mark Banner (:standard8)]

The tinderboxes have just started another cycle, so I'll check this in in the morning.

Comment 17

[Mark Banner (:standard8)]

I've just checked the defaults for this:

Disk Cache: 50Mb
Compare the cache to the network is set to "When page is out of date"

These are the same as FF. I think the disk cache size may be a little large, for what we need. Any thoughts on this?

Comment 18

[hansen]

Looking forward to this. Thanks for the work.

50MB might be high, but let's give it a try.

My biggest concern is the "physical" position of the cache.
On Windows, will it be placed like Firefox, in Doc&Set/User/Local Settings?
This way, reducing the roaming profiles.

Comment 19

[Mark Banner (:standard8)]

Comment on attachment 334730 [details] [diff] [review]
[checked in] Enable Disk Cache in Thunderbird builds

Checked in changeset id: 152:d960392f1004

Comment 20

[Mark Banner (:standard8)]

(In reply to comment #18)
> My biggest concern is the "physical" position of the cache.
> On Windows, will it be placed like Firefox, in Doc&Set/User/Local Settings?
> This way, reducing the roaming profiles.

It uses the same code as Firefox, therefore it will be placed where ever Firefox stores its cache.

Comment 21

[David Ascher (:davida)]

Switching for b1 flags to target milestones, to avoid flag churn.

Comment 22

[Mark Banner (:standard8)]

Drivers have agreed to keep 50Mb so I'm marking this as fixed.

Comment 23

[Joe Sabash [:JoeS1]]

This seems to have had a positive effect on viewing RSS feeds with multimedia content like YouTubes, as well as multiple large images in newsgroups.
It also seems to have "fixed" some Quicktime plugin crashes as well.

Hard to say how much of the improvement I'm seeing is due to the disk cache, since the about:cache url has not been enabled with this checkin.

Do we need a new bug to have about:cache enabled.

Tested with:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1a2pre) Gecko/20080821144138 Shredder/3.0b1pre ID:20080821144138

Comment 24

[Magnus Melin [:mkmelin]]

> Do we need a new bug to have about:cache enabled.

Yes please.

Comment 25

[Joe Sabash [:JoeS1]]

(In reply to comment #24)
> > Do we need a new bug to have about:cache enabled.
> 
> Yes please.
> 
Opened bug 451987