612623 - profile de-fragmentation on major upgrade

Status: NEW

(Firefox :: General, defect)

Description

[Christopher Blizzard (:blizzard)]

We'd like to do a one-time profile cleanup when someone upgrades the major version of Firefox.  This basically requires copying all the files in the profile to reduce fragmentation.  (Firefox 4 will have larger block allocations in many places to help with this problem over the long term.)

As a bonus we might also vacuum databases before we do the copy to reduce file size even more.  (Have we ever done this?)

Comment 1

[Mike Beltzner [:beltzner, not reading bugmail]]

cc'ing some of the usual candidates for vacuuming and profiles; would touching the files help, or do we actually need to do a full copy?

Comment 2

[(dormant account)]

I don't know about a full profile copy, I think recreating databases on profile-migration(instead of doing a tricky inplace upgrade) would certainly be a nice optimization(sqlite vacuum is basically a "copy to external db" then "copy back" operation). This has the benefit of being able to disable fsyncs completely during profile migration(since existing data is there) and only sync to disk once migration is complete and the browser is up

Comment 3

[Robert Strong (they/them - no direct email)]

On Windows has anyone checked if the built-in Windows defragmenter reduces the fragmentation to an acceptable level? I'd check but I disable the built-in defragmenter and use PerfectDisk instead.

Comment 4

[(dormant account)]

(In reply to comment #3)
> On Windows has anyone checked if the built-in Windows defragmenter reduces the
> fragmentation to an acceptable level? I'd check but I disable the built-in
> defragmenter and use PerfectDisk instead.

yes, when it works, it works great. We could change the updater to defrag files on update(when sufficient permissions are available).

Problem is that one can't rely on it because the defragmenter is disabled for SSDs, virtual machines and sometimes takes a long time to defragment files even if it is enabled.

Comment 5

[Mike Beltzner [:beltzner, not reading bugmail]]

I'm less worried about SSDs than the majority case of spinning platters :)

Comment 6

[Robert Strong (they/them - no direct email)]

(In reply to comment #4)
> (In reply to comment #3)
> > On Windows has anyone checked if the built-in Windows defragmenter reduces the
> > fragmentation to an acceptable level? I'd check but I disable the built-in
> > defragmenter and use PerfectDisk instead.
> 
> yes, when it works, it works great. We could change the updater to defrag files
> on update(when sufficient permissions are available).
So, why do you think it isn't working then? Taking a long time is actually the way I would hope defragmentation would work so it wouldn't interfere with what the user is doing.

The updater has the ability to launch a separate binary on Windows and Mac (it would be relatively simple to add for Linux as well) so if you want to go down that path using a separate binary is the way to go.

Comment 7

[Robert Strong (they/them - no direct email)]

(In reply to comment #6)
> (In reply to comment #4)
> > (In reply to comment #3)
> > > On Windows has anyone checked if the built-in Windows defragmenter reduces the
> > > fragmentation to an acceptable level? I'd check but I disable the built-in
> > > defragmenter and use PerfectDisk instead.
> > 
> > yes, when it works, it works great. We could change the updater to defrag files
> > on update(when sufficient permissions are available).
> So, why do you think it isn't working then? Taking a long time is actually the
> way I would hope defragmentation would work so it wouldn't interfere with what
> the user is doing.
> 
> The updater has the ability to launch a separate binary on Windows and Mac (it
> would be relatively simple to add for Linux as well) so if you want to go down
> that path using a separate binary is the way to go.
Forgot to add that the user account running Firefox has sufficient privileges to the user's Firefox profile so "when sufficient permissions are available" is not an issue. Also, the user account performing the update won't necessarily be able to access other user system profiles in order to defragment them even after going through all the hoops to access them.