567308 - Test Pilot needs to clean up after itself; delete old prefs and data

Status: RESOLVED WONTFIX

(Mozilla Labs Graveyard :: Test Pilot, defect, P1)

Description

[Dave Garrett]

In my current prefs.js extensions.testpilot.experiment.codeFs sucks up 186 KB and contains tests that are not currently in use. This does not need to be in this file and does not need to be loaded on each startup with all the other prefs. Old tests should be cleaned out of here when done.

I also have another 4.8 KB for extensions.testpilot.tab_switch_study.host_hash which contains a list of domains visited during that study, which is not currently ongoing, and this does not clear when wiping history. This is a privacy problem.

There's also quite a few other prefs lingering here that are past their expiration dates. For privacy and performance reasons, please clean these out after they are no longer needed and only keep a basic record of past results.

In my profile dir I also see TestPilotErrorLog.log and TestPilotErrorLog.log.1 totaling 2.9 MB and half a dozen sqlite files for various tests totaling 456 KB. These files are likely less of a problem, but nonetheless litter that doesn't need to be kept here indefinitely.

For reference, this is in my primary profile under Linux currently using Firefox 3.6.5pre and Test Pilot 1.0a2. According to extensions.testpilot.tabsExperiment.startDate I've had Test Pilot installed since at least Sep 26 2009. (possibly earlier; I don't remember)

Comment 1

[Jono Xia]

Hi Dave, thanks for bringing this up.  You are right.

The sqlite files are now (in trunk) getting wiped when the data expires (7 days after the test is done - to give you a window to save your data and charts if you like).  I'm in the process of moving the codeFs stuff out to separate .jar files instead of a pref.  We can make it so the jar file is deleted at the same time , leaving only the metadata about the test.

With the error logs, we are really only interested in the last week or so, so we can figure out some way of periodically deleting all but the last X lines of the error log.

That leaves the host_hash pref, which was an ad-hoc storage created by a particular study and isn't part of the general study API.  I think what we need to do for cases like this is let each study optionally implement a "profile cleanup" function that gets called when its data expires, that can do things like delete any ad-hoc prefs it set.

Comment 2

[Jono Xia]

Attachment: Patch adding handlers for experiment cleanup

This patch (also in Test Pilot Hg as http://hg.mozilla.org/labs/testpilot/rev/c06482a9b4ce ) makes the first step towards proper cleanup, by adding a handler that is called when an experiment finishes or is canceled; experiments can override the default handler to add cleanup code for anything they create.

This change is necessary in order to run the Search study we want to run soon, because without it the search study can't clean up after itself.

Comment 3

[Dave Garrett]

Are you going to eventually be able to handle old data from old tests using this? In particular, extensions.testpilot.tab_switch_study.host_hash needs wiping ASAP. Frankly, it should have never been used as it was as it's a big silent and permanent privacy leak (though at least locally only). If it was actually a hash as stated it would have been better, but instead it's a list of every site the user visited during the test saved for all eternity.

Comment 4

[Dave Townsend [:mossop]]

Landed on mozilla-central: http://hg.mozilla.org/mozilla-central/rev/e560e15ee4a9

Comment 5

[Jono Xia]

Not sure why this was closed; there is still more cleanup to do as Dave pointed out in comment 3.

Comment 6

[JP Rosevear [:jpr]]

Comment on attachment 465892 [details] [diff] [review]
Patch adding handlers for experiment cleanup

Original patch landed, future patches will need new approvals.

Comment 7

[Adam Muntner [:adamm] (use NEEDINFO)]

decommissioned