1836990 - [Experiment] The same default browser homepage banner is displayed on all treatment branches of the "Mobile default browser homepage banner copy test” experiment

Status: VERIFIED FIXED

(Fenix :: Experimentation and Telemetry, defect)

Description

[Cosmin Muntean [:cmuntean], Ecosystem QA]

[Notes]:

• Based on the "Experiment: Set to default Homepage Banner with ECO body" document, in each treatment branch we should have a different default browser homepage banner. However, I have verified all the branches and the same default browser homepage banner is displayed.
• However, we are not sure if the issue is reproducible because we are using the Preview collection or we are missing some steps.
• Here is the link to the Experimenter page.

[Affected versions]:

• Firefox Release 113.2.0 (Build #2015952075)

[Affected Platforms]:

• Android 13 - OnePlus 8 Pro

[Steps to reproduce]:

1. Open the Firefox release app.
2. Open Firefox Settings -> About Firefox and tap 5 times on the "Firefox Browser" title to enter in debug mode.
3. Return to the "Settings" menu and tap the "Secret Settings" option and enable the "Use Nimbus Preview Collection" option.
4. Close and reopen the browser at least 3 times.
5. Open Firefox Settings -> About Firefox and tap 5 times on the "Firefox Browser" title to enter debug mode.
6. Return to the "Settings" menu and tap the "Nimbus Experiments" option.
7. Tap on the "Mobile default browser homepage banner copy test" experiment.
8. If you are not enrolled in a branch, tap on any of the treatment branches.
9. Close and reopen the browser.
10. Observe the default browser homepage banner.

[Expected results]:

• The specific default browser homepage banner of the enrolled branch is displayed.

[Actual results]:

• The control's default browser homepage banner is displayed even if you are enrolled in a treatment branch.

[Additional Notes]:

• Here is a screen recording of the issue: Gdoc.

Comment 1

[Cosmin Muntean [:cmuntean], Ecosystem QA]

The issue was fixed by the latest changes made on the Experimenter page.

Comment 2

[Cosmin Muntean [:cmuntean], Ecosystem QA]

I have re-verified the issue and is no longer reproducible with Firefox Release 112.2.0, Firefox Release 114.2.0, Firefox Release 115.1.0 and the latest Beta 116.0b7.

• The correct message is displayed after enrolling in the experiment.