1395234 - [Shield] [Opt-Out] The Opt-Out study is not marked as completed once the study add-on is removed/disabled

Status: RESOLVED INVALID

(Shield :: General, defect)

Description

[Adrian Florinescu [:aflorinescu]]

[Enviroment:]
56.0b7 20170828185355
Windows 10 x64, Ubuntu 16.04 x64

[Preconditions:]

You need access to Admin interface (for this Test Suite, https://normandy-admin.stage.mozaws.net/control
1. Obtain a copy of Firefox with the SHIELD recipe client system add-on installed. You can check about:support to ensure that you have it.
2. Set the extensions.shield-recipe-client.dev_mode preference to true to run recipes immediately on startup.
3. Set the extensions.shield-recipe-client.logging.level preference to 0 to enable more logging.
4. Set the security.content.signature.root_hash preference to DB:74:CE:58:E4:F9:D0:9E:E0:42:36:BE:6C:C5:C4:F6:6A:E7:74:7D:C0:21:42:7A:03:BC:2F:57:0C:8B:9B:90
5. Set the preference value for extensions.shield-recipe-client.api_url set to https://normandy.stage.mozaws.net/api/v1


[Steps:]
1.Open Admin Interface.
2.Open the extension page. (https://normandy-admin.stage.mozaws.net/control/extension/)
3.Click Add New, Upload and save a new Extension.
4.Open control interface.
5.In the Control Interface, Click on Add New.
6.In the Name field, set a recipe name.
7.In the Filter Expression set a filter expression (e.g. normandy.isDefaultBrowser).
8.From the action DropDown, select type "Opt-Out Study"
9.In the Study Name input the Name of the study
8.In the Study Description input text describing what the study does.
9.From the Add-on dropdown select the extension uploaded at step 3.
10.Save Draft.
11.Request Approval, get the recipe approved and enable the recipe.
12.Open a FF set as a default browser having the pre-requisites set and open browser console, about:studies and about:addons
13.Remove/Disable the study add-on.


[Actual Result:]
The study is marked as completed only after FF is restart.

[Expected Result:]
The study is completed and marked as such as soon as the add-on is disabled/removed.

Comment 1

[Adrian Florinescu [:aflorinescu]]

Now that I think about it, for add-ons/extensions we have the following options: disable/remove. Once you remove an add-on/extension you also have undo as an option.

I guess that what we're aiming for is that once the add-on is touched in any way: disabled or removed, we automatically complete the study, right?

Comment 2

[Osmose [:osmose, :mkelly]]

So the root cause is that about:addons does not uninstall add-ons until after you close the tab, so that the Undo link works. When testing the removal of add-ons, we need to include closing about:addons as part of the test steps.

As for disabling, all study add-ons are required to uninstall themselves when they are disabled, as disabling and re-enabling the add-on would invalidate the data we're collecting about how the user interacts with it. So testing the disable button with non-study add-ons isn't valid for our purposes.

I confirmed with Adrian that uninstall + closing about:addons does correctly mark the study as ended, so this bug is invalid.