1672177 - [Experiment] Users are not unenrolled from the experiment if the “Allow Firefox to install and run studies” is unchecked in the “about:preferences#privacy” page on Firefox 82

Status: VERIFIED FIXED

(Firefox :: Messaging System, defect)

Description

[Robert Martin [:romartin], Ecosystem QA]

Attachment: 2020-10-20_15h40_21.gif

[Notes]:

• I have managed to reproduce the issue for the following studies: “Multi-Stage About:Welcome Zero Stage”, “Homepage Remediation: Search Value Props Experiment”, and “Wizardless Password Autocomplete”. However, I think that this issue affects all the Firefox 82 experiments that are being delivered through Nimbus.

[Affected Versions]:

• Firefox RC 82.0 (Build ID: 20201014125134)

[Affected Platforms]:

• Windows 10 x64
• Windows 7 x64

[Prerequisites]:

• Have a new Firefox profile created.
• Have the user.js saved in the profile folder before opening the profile. (There should only be 2 files in the folder, times.json and user.js)
• Have the Terminal opened.

[Steps to reproduce]:

1. Start the Firefox profile from prerequisites using the first-startup flag.
2. Navigate to the “about:preferences#privacy” page.
3. Uncheck the “Allow Firefox to install and run studies” option.
4. Navigate to the “about:studies” page and observe.

[Expected results]:

• The study can be observed in the “Completed studies” section.

[Actual results]:

• The study is still in the “Active studies”.

[Additional Notes]:

• I've verified if other means of making the profile ineligible (from a targeting standpoint) will reproduce the issue and it does not. In order to do this, I've done the following:
  • I've enrolled a new profile in the experiment using a Firefox 82 Release.
  • I've opened the same profile with a Firefox 81 Beta by adding the --allow-downgrade argument in the Terminal.
• The issue is not reproducible for Firefox Beta 83.0b1.
• Attached a screen recording of the issue.

Comment 1

[Punam Dahiya [:pdahiya]]

@romartin Thanks for bringing it up, the issue here is specific to stage RS, we have deleted couple of test recipes ( 'devoted-coral' and 'opposite-tomato' ) which were failing 82 schema and interfering with initialization of RemoteSettingsExperimentLoader, causing failure to move studies to 'Completed'.

Please note once the user opts out, experiment is ended on next recipe update run after a interval or restart of Firefox instance. Thanks!

Comment 2

[Robert Martin [:romartin], Ecosystem QA]

I have verified the following for all 3 studies mentioned in the description using Firefox Release 82.0 en-US (Build ID: 20201014125134) on Windows 10 x64 and Windows 7 x64:

• I unchecked the "Allow Firefox to install and run studies" from the "about:preferences#privacy" page and performed a Firefox browser restart which resulted in the studies to be marked as "Completed" in the "about:studies" page and an unenroll telemetry ping is correctly displayed in the "about:telemetry#events-tab" page.
• I unchecked the "Allow Firefox to install and run studies" from the "about:preferences#privacy" page and waited ~5 minutes which resulted in the studies to be marked as "Completed" in the "about:studies" page and an unenroll telemetry ping is correctly displayed in the "about:telemetry#events-tab" page.