691075 - Clear Cache Is Slow

Status: RESOLVED DUPLICATE of bug 670911

(Core :: Networking: Cache, defect)

Description

[David E. Ross]

Mozilla/5.0 (Windows NT 5.1; rv:7.0) Gecko/20110923 SeaMonkey/2.4

When a user requests Clear Cache, the process is now much slower than it was with SeaMonkey 2.0.x.  

On examining my cache file structure before and after clearing, I find a set of 16 primary folders, each with a number of sub-folders.  Many of those sub-folders contain only one file each.  This is an inefficient file structure.  

On clearing my cache, I see that the primary folders are still in place.  The slowness of clearing might thus result from sequentially examining each of the 16 primary folders and deleting its contents.  This too is inefficient.  

Why cannot all the primary folders be deleted at once?  They should then be regenerated -- each only as needed -- with the first requests to place files in them.  

NOTE:  This is NOT a duplicate of bug #670911, which deals with clearing the cache after a crash.  I am seeing this problem when no crash has occurred.

Comment 1

[Jo Hermans]

(In reply to David E. Ross from comment #0)
> NOTE:  This is NOT a duplicate of bug #670911, which deals with clearing the
> cache after a crash.  I am seeing this problem when no crash has occurred.

That's because it's the same routine that tries to delete the cache. The title of bug 670911 doesn't make that clear. Note that a "slow" clearing of the cache (as fixed in bug 670911) was often lasting minutes - are you still seeing that problem ?

Note that you can't delete a directory with 1 instruction. You need to loop over all files first, then delete the directory. And if meet a sub-directory, you have to do it recursively. That how all operating systems works, even though they might hide it by a wrapper function.

There a few bugs out there that want to change the cache directory structure. The latest versions of Gecko does indeed use (much) more sub-directories, but that's for performance boost (avoiding a large number of files in some of those directories). And the size of the cache is also increased substantially (for Firefox at least, I don't know about SeaMonkey), which makes it much more obvious what going on that in earlier versions. Maybe you're confusing with that. The algorithm itself isn't really changed (aside from the regression that was fixed in bug 670911).

Also note that deleting will take place in the background (in SeaMonkey too ?), it shouldn't be a concern at all how long it takes, or with what structure. Which is just seconds by the way.

Comment 2

[David E. Ross]

No, I'm seeing seconds, not minutes.  

Prior to deleting my cache, I requested about:cache.  The display showed (among other values):
Number of entries: 	113
Storage in use: 	598 KiB
It took approximately 3 seconds.  

Prior to the current cache design, larger caches seemed to be deleted instantaneously.

Comment 3

[Jason Duell]

David,

Thanks for the bug report.

Jo is right about the logic for crashes and "clear cache" being the same.  In both cases the problem was a windows-specific one with NTFS permissions slowing down our logic of "rename cache directory, then delete it in the background".  With the NTFS fix you shouldn't experience any noticeable hang.  The fix is coming in Firefox 8.

I've change the description of bug 670911 to avoid future confusion.

> Prior to the current cache design, larger caches seemed to be deleted instantaneously.

That was mainly because they were small (50 MB vs up to a GB now).