1744951 - [Regression] The “More From Mozilla” pane is almost empty after an update to Firefox 96 beta is done while enrolled in the Treatment branch of the Experiment

Status: RESOLVED DUPLICATE of bug 1744888

(Firefox :: Messaging System, defect)

Description

[Robert Martin [:romartin], Ecosystem QA]

Attachment: 2021-12-08_13h41_36.gif

[Notes]:

• The “More From Mozilla” Experiment is currently running on the Beta channel until 16th of December and even though the enrollment is complete, already enrolled users might update to the latest Firefox Beta version and see the almost empty “More From Mozilla” pane.

[Affected Versions]:

• Firefox Beta 96.0b2 (Build ID: 20211207185904)
• Firefox Nightly 97.0a1 (Build ID: 20211207155117)

[Affected Platforms]:

• Windows 10 x64
• macOS 11.6
• Ubuntu Linux 20.04 x64

[Prerequisites]:

• Have Firefox Beta 95.0bx installed.
• Have the “nimbus.debug” pref value set as true.

[Steps to reproduce]:

1. Open the browser with the profile from prerequisites.
2. Navigate to the “about:studies?optin_slug=more-from-mozilla&optin_branch=treatment”
   page.
3. Update the browser to the latest Beta version.
4. Navigate to the “about:preferences” page.
5. Click the “More From Mozilla” pane and observe the page.

[Expected results]:

• The “More From Mozilla” title is displayed
• The “Check out other Mozilla products that work to support a healthy internet.” subtitle is displayed.
• The “Firefox Mobile”, “Mozilla VPN”, and “Mozilla Rally” sections are displayed.

[Actual results]:

• The “More From Mozilla” title is displayed
• The “Check out other Mozilla products that work to support a healthy internet.” subtitle is displayed.

[Regression Window]:

• Since this issue is not reproducible with the Firefox Nightly 96.0a1 version from 2021-12-02, using the Mozregression tool we have managed to find the following regression window:
  Last good revision: 19a1500c08e308d66cf26452ebf66d2049839f8b
  First bad revision: edcfdb2bbe36f9c40c96ac5b8201aaa1e956eca6
  Pushlog:
  https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=19a1500c08e308d66cf26452ebf66d2049839f8b&tochange=edcfdb2bbe36f9c40c96ac5b8201aaa1e956eca6

From the pushlog it seems that bug 1740393 has caused this regression.

[Additional Notes]:

• Attached is a screen recording of the issue.
• The issue is reproducible even if the “browser.preferences.moreFromMozilla” pref value is flipped to true.

@Punam, @Daniel, can you please weigh in on this?

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1740393

Comment 2

[Dan Mosedale (:dmosedale, :dmose)]

I suspect this is bug 1744888; should be a straightforward fix, if I’m right. I’ll have a look this morning. Thanks for finding this!

Comment 3

[Dan Mosedale (:dmosedale, :dmose)]

There was a hope that no-one would hit this in the wild, because the JSON targeting for the experiment would target 95 only. However, that doesn't appear to me to be the case:

The JEXL that is used in the live experiment is version|versionCompare('95.!') >= 0, and putting that in the 97 devtools evaluates to true, so I suspect we're going to have a find a different plan here.

https://experimenter.services.mozilla.com/nimbus/more-from-mozilla/details is the live experiment.

As a result this bug may or may not end up being a DUP of bug 1744888.

Comment 4

[Dan Mosedale (:dmosedale, :dmose)]

I talked this over with Punam, and she said that different JEXL will be used to target the release experiment, so it will prevent this from happening there.

She also said:

beta experiment was mostly for health check, here’s the metrics https://sql.telemetry.mozilla.org/dashboard/more-from-mozilla-beta-95 , only 29 users who saw preferences (bearbeitet) and 135 enrolled in treatment, very little chance any of these users will upgrade and visit settings

So we're not worried about this for users from the 95 beta experiment, and it's tracked for 96 in bug 1744888. Mark this bug as a DUP.

Comment 5

[Dan Mosedale (:dmosedale, :dmose)]

That said, it was a good find; thanks, Robert!