593349 - meta: Startup cache writing algorithm causes severe fragmentation

Status: RESOLVED FIXED

(Core :: XPCOM, defect)

Description

[(account disabled for personal attacks)]

User-Agent:       Mozilla/5.0 (Windows NT 6.1; rv:2.0b6pre) Gecko/20100903 Firefox/4.0b6pre
Build Identifier: Mozilla/5.0 (Windows NT 6.1; rv:2.0b6pre) Gecko/20100903 Firefox/4.0b6pre

The way startup cache file gets written to disk causes severe fragmentation, that gets worse and worse over time.

Reproducible: Always

Steps to Reproduce:
1. delete startup cache (profile\startupCache\startupCache.4.little
2. restart Firefox
3. run analyze in some defragmentation program that shows file fragmentation
Actual Results:  
Startup cache is severely fragmented, always 128 fragments when freshly created, and gets worse over time - 238 fragments (and that's for a 2,6MB file only) after 8 hours of usage and several Firefox restarts right now.

Expected Results:  
The writing algo should be optimized to get at least some reasonable levels of fragmentation.

System: Windows 7. Profile folder on a local, NTFS formated, fully defragmented, non-system disk with 400GB of contiguous free space. No system restore, shadow copies or any other unmovable files that could cause higher fragmentation.

Comment 1

[(dormant account)]

Thanks for the excellent report.

Comment 2

[Benedict Hsieh [:benedict]]

Note (mostly) to self: Taras's patch in bug 594172 could help with this.

Comment 3

[Benedict Hsieh [:benedict]]

However, the main thing is to have a better timer for the initial write. Either extend the length of the timer, or (better) reset the timer when the user does some action.

Comment 4

[Brian Carpenter [:geeknik]]

This is a major issue on my system. I can run Firefox, close Firefox, open Diskeeper, see the startup cache is in 2000 or 3000 fragments. So I defrag. Reboot, run Firefox, close Firefox, open Diskeeper and guess what, it's back to being fragmented. Not as bad, but even so, it shouldn't be fragmented at all. Win7 Pro x64.

Comment 5

[(dormant account)]

(In reply to comment #4)
> This is a major issue on my system. I can run Firefox, close Firefox, open
> Diskeeper, see the startup cache is in 2000 or 3000 fragments. So I defrag.
> Reboot, run Firefox, close Firefox, open Diskeeper and guess what, it's back to
> being fragmented. Not as bad, but even so, it shouldn't be fragmented at all.
> Win7 Pro x64.

what about other files in the profile directory? How much free space do you have on the drive with the startupcache?

Comment 6

[Brian Carpenter [:geeknik]]

Fragmentation on the other files in the profile directory are about normal when compared with other directories on the drive. I do defrag daily. As far as space, this is a RAID10 (4x250GB drives), 500GB total, 216GB free.

Comment 7

[(dormant account)]

(In reply to comment #6)
> Fragmentation on the other files in the profile directory are about normal when
> compared with other directories on the drive. I do defrag daily. As far as
> space, this is a RAID10 (4x250GB drives), 500GB total, 216GB free.

You can try a test build from http://ftp.mozilla.org/pub/mozilla.org/firefox/tryserver-builds/[e-mail]-a754436412b3/tryserver-win32/firefox-4.0b5pre.en-US.win32.zip

To see if it reduces your fragmentation. Delete your startup cache first.

Comment 8

[Benedict Hsieh [:benedict]]

Taras did some jar work to reduce fragmentation, next step is probably just to bump the timer timeout to something like 10 minutes.

Comment 9

[Benedict Hsieh [:benedict]]

Attachment: extends buffer timer

Currently, the write algorithm buffers all the writes that happen during startup and writes them all out at once. This patch just extends that window to catch more writes. It does seem to help a bit, especially for the first-run fragmentation.

I did not anticipate that each write with the zipWriter API would cause so much fragmentation, though, so it's worth thinking about whether we can get the zipwriter to allocate some fixed amount of space or other strategies to reduce this fragmentation.

Comment 10

[(account disabled for personal attacks)]

It's a lot better with all the changes. Right now I get about 40 fragments when creating a new cache and it doesn't climb that much during usage.

Comment 11

[Benedict Hsieh [:benedict]]

Attachment: extends buffer timer, plus some testing convenience

Let's do this first, since the fragmentation of the first-run write will affect perf for all other runs as well. Long-term, we should occasionally / periodically defragment, perhaps by copying to a new file.

Comment 12

[[e-mail]]

Comment on attachment 480204 [details] [diff] [review]
extends buffer timer, plus some testing convenience

>diff --git a/startupcache/StartupCache.cpp b/startupcache/StartupCache.cpp

>   // Wait for 10 seconds, then write out the cache.
>-  rv = mTimer->InitWithFuncCallback(StartupCache::WriteTimeout, this, 10000,
>+  rv = mTimer->InitWithFuncCallback(StartupCache::WriteTimeout, this, 600000,
>                                     nsITimer::TYPE_ONE_SHOT);

Stale comment.

r=dwitte

Comment 13

[Benedict Hsieh [:benedict]]

Not sure how to mark this, but will also help Bug 601186 which is a Fennec blocker.

Comment 14

[Benedict Hsieh [:benedict]]

Attachment: extends buffer timer, plus some testing convenience

Comment 15

[Benedict Hsieh [:benedict]]

Attachment: extends buffer timer, plus some testing convenience

Ready to go.

Comment 16

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

http://hg.mozilla.org/mozilla-central/rev/c0087f1d14bb

Comment 17

[Benedict Hsieh [:benedict]]

I think Taras had some ideas on when we can copy the startupcache file to reduce fragmentation further.

Comment 18

[Dave Garrett]

Comment on attachment 481916 [details] [diff] [review]
extends buffer timer, plus some testing convenience

>   // Wait for 10 seconds, then write out the cache.
>-  rv = mTimer->InitWithFuncCallback(StartupCache::WriteTimeout, this, 10000,
>+  rv = mTimer->InitWithFuncCallback(StartupCache::WriteTimeout, this, 600000,
>                                     nsITimer::TYPE_ONE_SHOT);

You might want to also change the comment to match the result: 10 seconds should be 10 minutes now.

Comment 19

[(dormant account)]

the zipwriter fix reduced fragmentation by 10x, bunching up writes better in cache-on-thread bug should correct the rest.

Comment 21

[(dormant account)]

Landed part of 586859 which takes care of abnormally high fragmentation.