1924686 - [Experiment] Unenrolling from any "sidebar + GenAI" experiment does not revert the experience to the default one

Status: VERIFIED FIXED

(Firefox :: Sidebar, defect)

Description

[Cosmin Muntean [:cmuntean], Ecosystem QA]

Attachment: unenrolling from the experiment.mp4

[Affected versions]:

• Firefox Nightly 133.0a1 - Build ID: 20241014095457
• Firefox Beta 132.0b7 - Build ID: 20241014091808

[Affected Platforms]:

• Windows 10
• macOS 13.4
• Linux Mint 20.1

[Prerequisites]:

• Have following prefs set:
  • browser.search.region = US
  • nimbus.debug = true

[Steps to reproduce]:

1. Open the browser using the profile from the prerequisites.
2. Navigate to the following custom link to force enroll in the treatment-a of the experiment: about:studies?optin_slug=new-users-sidebar-genai&optin_branch=treatment-a&optin_collection=nimbus-preview.
3. Close the tab.
4. Navigate to about:studies.
5. Click the "Remove" button next to the "New users: sidebar + GenAI" experiment.
6. Observe the behavior.

[Expected result]:

• The profile is unenroll from the experiment, and the sidebar and the new tab layout are reset to the old view which is the default.

[Actual result]:

• The profile is unenroll from the experiment, but the new sidebar and new vertical tab layout remain enabled.

[ Notes]:

• The issue is also reproducible after a browser restart.
• The issue is also reproducible if naturally enroll in the experiment.
• The issue is also reproducible for the existing users experiment.
• Attached is a screen recording of the issue.

Comment 1

[Sarah Clements [:sclements]]

Hi Cosmin, we're doing something different in this experiment in that we don't want to unenroll users who change any of the settings which are initially set by nimbus prefs, and for users preferences to persist post experiment (so not to reset to the old sidebar).

See bug 1895932 for some details (and this). Sorry this wasn't communicated to you. For the steps you went through here, if you change the tools (deselect genAI via the Customize Sidebar menu, for example), or disable vertical tabs, do those changes persist after the experiement ends?

Comment 2

[Cosmin Muntean [:cmuntean], Ecosystem QA]

Thank you Sarah for providing the information. I will update the JIRA QA request description to contain this. I will also inform my colleagues who are working on the other experiment for existing users.

For the steps you went through here, if you change the tools (deselect genAI via the Customize Sidebar menu, for example), or disable vertical tabs, do those changes persist after the experiement ends?

• Yes, all changes persist after the experiment ends.

Due to the fact that this behavior is intended and the QA request was updated, I will mark the issue as fixed.