692252 - Defrag some of our files on Windows for faster startup and responsiveness

Status: NEW

(Core :: Widget: Win32, defect)

Description

[Brian R. Bondy [:bbondy]]

Certain files (maybe places.sqlite?) get fragmented over time.

It would be good to have the Windows service (or we may not even need the service depending on how we do it) to defrag these files periodically so we have faster startup.

I think we could use this API to do it from the new Windows service component:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa363911(v=vs.85).aspx#defragmenting_a_file

Or perhaps an easy way is to create a new file, allocate it instantly all at once, and then copy the data into it in chunks.

O(1) create a big file:
CreateFile, SetFilePointerEx, SetEndOfFile, and CloseHandle.

Comment 1

[Robert Strong (they/them - no direct email)]

As long as we only target our files though I think we can get the same effect by allocating and then writing as the updater does.
http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/update/updater/updater.cpp#1320

I'd go with which ever one is safest and if both are safe then fastest.

Comment 2

[Brian R. Bondy [:bbondy]]

Ya that's the same as the second way I mentioned.  I'm not sure if that guarantees it will be sequential on disk (maybe on SetEndOfFile it allocates multiple spots on disk), but probably it is close enough though and a much easier way to implement it.

Comment 3

[Robert Strong (they/them - no direct email)]

It doesn't gaurantee that it will be sequential but it does allocate sequentially if there is sequential space available on the disk which is not dissimilar to not being able to find sequential space with the defrag method.

Comment 4

[Brian R. Bondy [:bbondy]]

cool, that should be good enough.

Comment 5

[(dormant account)]

This would be of biggest benefit for files in the profile dir and it will result in a more responsive browser, not just faster startup.

Rob is right, files installed/updated by us should already not be fragmented. It would be useful to have the service examine which files get most fragmented and report that via telemetry.

Comment 6

[Brian R. Bondy [:bbondy]]

I'm not sure if places.sqlite gets capped at some size or not but it is 10MB for me currently in my profile.  This is probably the largest file (I may be wrong on what the largest file is). 

One benefit of the first approach (actual file defrag) is that we wouldn't need twice the largest file's disk space and in general there would be less IO when a file is not very fragmented.  

The second approach though is probably more safe and would ensure no file corruption even if the computer was turned off in the middle of an operation.

Comment 7

[(dormant account)]

I think the size for places is somewhere between 60-100mb(extensions *may* make it bigger). We grow it in 10mb(i think) increments for most people. So it should not fragment to too many pieces, but realistically doing OS-level defrag is the most efficient-way to do this.

There are also extension-specific databases that likely fragment themselves to death. Thus it would be helpful to do a dumb "defrag everything in profile directory" approach.