1731617 - [Experiment] The Onboarding modal is shown in the control branch of the experiment after disabling the “Include occasional sponsored suggestions” and “Contextual suggestions” options

Status: VERIFIED FIXED

(Firefox :: Address Bar, defect)

Description

[Simona Rosu [:srosu], Ecosystem QA]

Attachment: OnboardingModal_WronglydisplayedonControlBranch.gif

[Affected versions]:

• Firefox Nightly 94.0a1 (Build ID: 20210920102201)
• Firefox Beta 93.0b7 (Build ID: 20210919190049)

[Affected Platforms]:

• Mac 11.5.2
• Windows 10 x64
• Ubuntu 20.04 x64

[Prerequisites]:

• Have the latest Firefox Nightly browser installed and opened.
• Have browser’s region set to “US”.
• Have the “nimbus.debug” pref set to “true” value on the “about:config” page.

[Steps to reproduce]:

1. Force enroll in the control branch of the experiment using the following link: about:studies?optin_slug=firefox-suggest-offline-vs-online&optin_branch=control&optin_collection=nimbus-preview.
2. Open “about:preferences#privacy-locationBar” and uncheck the “Include occasional sponsored suggestions” option.
3. Uncheck the “Contextual suggestions” option.
4. Restart the browser and observe what happens.

[Expected result]:

• The browser is normally restarted and the Onboarding modal is not shown.

[Actual result]:

• The Onboarding modal is displayed.

[Notes]:

• If the “Allow Suggestions” option from the modal is clicked, after a browser restart, the options are checked and both sponsored and non-sponsored results are displayed.
• Attached is a screen recording of the issue.

Comment 1

[Simona Rosu [:srosu], Ecosystem QA]

After further investigations, we have noticed that the "quickSuggestShouldShowOnboardingDialog" pref is set to "true" in the Nimbus recipe for the Control branch of the experiment. @drew, @mythmon, Shouldn't the pref be set to "false" in the Control branch?

Comment 2

[Drew Willcoxon :adw]

Yes, you're right, it should be false. And actually after bug 1730618 landed, we don't even need to specify quickSuggestShouldShowOnboardingDialog in either control or treatment anymore.

Comment 3

[Michael Cooper [:mythmon]]

I've removed quickSuggestShouldShowOnboardingDialog and quickSuggestEnabled from the Nimbus recipe, as Drew recommended (partially in Slack). Simona, can you re-test this?

Comment 4

[Simona Rosu [:srosu], Ecosystem QA]

I’ve just retested this issue using the latest Firefox Nightly 94.0a1 (Build ID: 20210920212918) and Firefox Beta 93.0b7 (Build ID: 20210919190049) on Windows 10 x64, Mac 11.5.1, Ubuntu 20.04 x64.

• The Onboarding modal is no longer shown in the control branch of the experiment after disabling the “Include occasional sponsored suggestions” and “Contextual suggestions” options. Considering these I’m marking this issue as Resolved Fixed.