800186 - Number of files grow without bounds in cache

Status: RESOLVED DUPLICATE of bug 709297

(Core :: Networking: Cache, defect)

Description

[Will Fiveash]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:16.0) Gecko/20100101 Firefox/16.0
Build ID: 20121005155445

Steps to reproduce:

I notice when the tuaw.com site is displayed the number of files/dirs in /Users/willf/Library/Caches/Firefox/Profiles/dp2ght2h.Firefox7/Cache appears to increase without bound.  Note that I've limited the over Cache size to 250 MB.


Actual results:

Number of files in Cache grew without bounds.


Expected results:

The # of files in the cache should be capped.

Comment 1

[Jo Hermans]

Is the size of the directory larger than 250MB ? The number of files is not important for the cache itself.

Comment 2

[Will Fiveash]

Let me provide more detail.  For the last month I've noticed firefox performance was degraded (slow to respond when I opened a bunch of websites in tabs simultaneously). I was seeing the spinning ball indicating wait for I/O.  I decided to manually delete the Cache dir (after I quit firefox) using rm -rf.  This took 15 minutes to complete which lead me to think that there was something off about the way firefox was managing its cache.  Once the cache was deleted I restarted firefox (which is now at v16) and limited it's cache size to 256MB.  So far I have not seen the total cache usage go beyond this.  I've also noticed that the number of files and dirs in the cache (measured using find . -print | wc -l) now appears to be capped however I notice looking at about:cache that the number of entries stat in the Disk cache device section appears to always increase.  I think someone should test this by repeatedly opening at least 40 different websites in tabs and watch the cache.  Do this with the default cache settings.  Note that this occurs on a Macbook Pro with 16G RAM and a 75GB hard drive with 200G free.

Comment 3

[Will Fiveash]

Also note that this isn't the first time I've manually rm -rf'ed the cache dir with that taking a very long time to complete.

Comment 4

[Jo Hermans]

There's no cap on the number of files in the cache. I don't understand the title of this bug report.

The default of 1GB is indeed to large, and cause diminishing returns, especially on slow filesystems.  And it's very slow when the cache needs to be cleared manually. The automatic clearing after a crash should be gone soon btw.

Note that since bug 709297, the current default cache size (in version 18 I think) is now 350 MB, but for the moment this is only enforced for new profiles. Otherwise you would see lots of I/O for several minutes after an upgrade.

dupe of bug 709297 ?

Comment 5

[Jason Duell]

We don't have a limit on the number of filenames--we got rid of that a while ago and it was a feature (the limit was too small for >50 MB cache).   We now just have a size limit.  It's true that our current implementation takes way too long to delete the cache, but we've got that on a background thread at least.  Eventually I hope we'll have a smarter filesystem architecture that deletes more quickly, but that's another bug.

Thanks for the report, but I don't think there's anything here to fix immediately, and bug 709297 does essentially what the poster has done manually (limit cache size).