676608 - Temporary add-ons installed by A/B tests not removed when Test Pilot is disabled

Status: RESOLVED WONTFIX

(Mozilla Labs Graveyard :: Test Pilot, defect)

Description

[Jono Xia]

User reported in an AMO review that disabling the Test Pilot extension didn't revert the change to the blank tab page caused by the New Tab Multivariate study.

We've tested that the temporary add-on is removed if the user opts out of the individual study, but of course a lot of users' first thought would be to disable Test Pilot as a whole through the add-on manager, which doesn't give the cleanup routine a chance to run.

A possible solution is to require that any temporary add-on installed for the purpose of an A/B test must include code that checks on startup whether Test Pilot is running so that it can disable itself if Test Pilot has also been disabled.

Comment 1

[Jono Xia]

A change has been made to the New Tab Multivariate study add-on in version 1.3 to implement this behavior, and deployed to the server.  People who join the study from now on will get this version, which means if they disable/remove Test Pilot their new tab will go back to normal.  Unfortunately there doesn't seem to be any way to deploy 1.3 to users who started the study earlier and got version 1.2.

Comment 2

[Jono Xia]

When we make Test Pilot restartless, we can try to handle this stuff in the disable() and remove() handlers in bootstrap.js.

Comment 3

[Adam Muntner [:adamm] (use NEEDINFO)]

decommissioned