742560 - Increase Fennec's disk cache size

Status: RESOLVED FIXED

(Core :: Networking: Cache, defect)

Description

[Brad Lassey [:blassey] (use needinfo?)]

We currently use a smaller disk cache than other browsers, and I suspect that may be hurting us.

Comment 1

[Mark Finkle (:mfinkle) (use needinfo?)]

Some data on existing cache sizes:
http://www.stevesouders.com/blog/2012/03/22/cache-them-if-you-can/
(search for 'Galaxy Nexus: 18 MB')

and:
http://www.stevesouders.com/blog/2010/07/12/mobile-cache-file-sizes/

Comment 2

[Brad Lassey [:blassey] (use needinfo?)]

Geoff, do you have a strategy for picking an appropriate size here?

Comment 3

[Geoff Brown [:gbrown]]

I think there is tension between cache effectiveness (cache it if you can / maximize cache size) and user tolerance for loss of usable storage ("firefox is a disk hog -- it's using 50% of my available storage!" / minimize use of storage).

As a starting point, I would like us to be comparable to other mobile browsers, so I am testing several browsers on several devices to get a better idea of cache sizes for other browsers.

Comment 4

[Geoff Brown [:gbrown]]

Oh darn, it's complicated!

Max cache size of the stock browser varies across devices. I monitored the size of the stock browser cache directory on various devices while browsing and noted the approximate maximum size on each:

Nexus 1 - Android 2.2 -         6 MB
Galaxy S - Android 2.2.1 -      6 MB
Galaxy Nexus - Android 4.0.3 - 24 MB
Galaxy Tab - Android 3.1 -     35 MB

The Dolphin browser appears similar: 10 MB on Galaxy S and 30 MB on Galaxy Tab.

We should also keep in mind that other browsers typically use the Android cache directory while Fennec does not - bug 742558.


Storage space varies widely across devices: Newer devices and tablets especially have much more space - and tolerance for storage-intense apps - than older phones. Perhaps the smart-sizing feature could be expanded to accommodate mobile? That might be the best long-term solution.

In the short-term, 10 MB seems pretty reasonable, but perhaps we could make it 20 MB: see how that affects telemetry hit rates, and if it generates any complaints.

Comment 5

[Mark Finkle (:mfinkle) (use needinfo?)]

We could probably check for the amount of RAM on the device and adjust accordingly.

Comment 6

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to Mark Finkle (:mfinkle) from comment #5)
> We could probably check for the amount of RAM on the device and adjust
> accordingly.

Which is what you suggested in comment 4. I'd rather err on the larger size to start with as well.

Comment 7

[Geoff Brown [:gbrown]]

Attachment: increase max disk cache size to 20MB

I reported bug 745340 to encourage a better solution, but anticipate that may take a while to sort out. Meanwhile, let's bump to 20 MB.

Comment 8

[Mark Finkle (:mfinkle) (use needinfo?)]

Comment on attachment 614918 [details] [diff] [review]
increase max disk cache size to 20MB

Let's see what data we get. Reverting is easy.

Comment 9

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/6f44748aa046

Comment 10

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

https://hg.mozilla.org/mozilla-central/rev/6f44748aa046

Comment 11

[Doug Turner (:dougt)]

taras,

I awhile ago we enabled disk cache to collect telemetry.  I understand that the disk cache caused startup and some jank woes.  Do you still have that data?

Reopening to sort out.

Comment 12

[Brad Lassey [:blassey] (use needinfo?)]

closing the bug again, the patch has been pushed and this has been resolved

Comment 13

[(dormant account)]

Our cache has some massive problems...including janking the main thread, blowing itself away on uncleanshutdown, etc. 
I thought we disabled disk cache because it provided more pain than gain...what changed?

Comment 14

[(dormant account)]

I can't find any bug#s to do with disabling the cache, but here is an email I wrote to relevant people:

NETWORK_DISK_CACHE_OPEN is almost never 0(unlike on desktop), indicating that we are doing something expensive there.  A large number of those are 100-3000ms. No other startupy histogram metrics are similarly slow.

On the non-startup side:
Various histograms that measure time to fetch entry from disk are also on the expensive side(ie HTTP_PAGE_CACHE_READ_TIME).
HTTP_OPEN_TO_FIRST_FROM_CACHE is not obviously better than HTTP_OPEN_TO_FIRST_FROM_RECEIVED...infact the median time is worse.

Note it seems like the recent landing of bug 648429 has shifted HTTP_OPEN_TO_FIRST_FROM_CACHE to the left(ie less time spent reading). This effect is quite obvious on fennec, not so obvious in desktop firefox.

Comment 15

[Doug Turner (:dougt)]

I think it was our mistake not disabling the disk cache on m-c after we collected this data.  We should disable the disk cache given this data you collected.

Comment 16

[Geoff Brown [:gbrown]]

Bug 645848 enabled disk cache for fennec. 

Bug 717031 disabled it on aurora only, for FF11, citing 3 bugs: 707436, 713480 and 716293. 707436 and 716293 are resolved/fixed; 713480 applies only to a disabled feature.

NETWORK_DISK_CACHE_OPEN has not improved much, but HTTP_PAGE_OPEN_TO_FIRST_FROM_CACHE is now substantially better than HTTP_PAGE_OPEN_TO_FIRST_RECEIVED (for 13.0, median of approx 100ms vs approx 700 ms.)

Comment 17

[(dormant account)]

I can't argue about this because the telemetry dashboard is not reporting population samples atm :( bug 747192

Comment 18

[JP Rosevear [:jpr]]

Should we re-open this bug until bug 747192 is resolved?

Comment 19

[Doug Turner (:dougt)]

Hey jp,

We spent a few weeks (a few years ago) looking for a major perf regression that turned out to be the disk cache. I think we need *overwhelming* data that suggests that enabing the disk cache is a good idea.  Right now, we do not have that.  If we do, please point me at it.

The disk cache is a major feature that hasn't proved itself yet.  This late in the game, it is best to cut it.  There will be future releases in which we can completely verify that enabling this feature is a good idea.

I will let you and brad sort out which bugs to open or reopen.