1621288 - An "unenrollFailed" event is sent besides the regular "unenroll" one when unrolling from the QuantumBar v1 experiment by manually modifying a pref

Status: NEW

(Firefox :: Normandy Client, defect, P3)

Description

[Cosmin Muntean [:cmuntean], Ecosystem QA]

Attachment: unenrollFailed.gif

[Affected Versions]:

• Firefox 74.0
• Firefox 75.0b1
• Nightly 76.0a1

[Affected Platforms]

• All Windows
• All Mac
• All Linux

[Prerequisites]
Note: We managed to reproduce this issue only with the recipe for QuantumBar v1 experiment.

• Have an active multi-preference-experiment recipe (Please use the recipe created by us for reproducing the bug).
• Have the recipes set-up in such a way that they target your profile.
• Have the "cosmin-search-test" boolean pref set to "true" in the profile to be targeted by the recipe.
• Have the security.content.signature.root_hash pref set to DB:74:CE:58:E4:F9:D0:9E:E0:42:36:BE:6C:C5:C4:F6:6A:E7:74:7D:C0:21:42:7A:03:BC:2F:57:0C:8B:9B:90.
• Have the app.normandy.api_url pref set to https://stage.normandy.nonprod.cloudops.mozgcp.net/api/v1.
• Have the app.normandy.dev_mode pref set to true.
• Have the app.normandy.logging.level pref set to 0.
• Have the services.settings.server pref set to https://settings.stage.mozaws.net/v1.

[Steps to reproduce]:

1. Open the Firefox browser with the profile from prerequisites.
2. Navigate to about:studies and verify that you are enrolled in the multi-preference-experiment.
3. Navigate to "about:config" page.
4. Search for any pref that is used by the experiment (eg: browser.urlbar.update1.interventions).
5. Set the pref to "false/true" depending on the enrolled branch.
6. Navigate to "about:telemetry#events-tab" page and observe the events registered.

[Expected results]:

• Only the "unenroll" event is correctly registered.

[Actual results]:

• An "unenrollFailed" event is sent with the "already-unenrolled" reason.

[Notes]:

• We haven't managed to reproduce this issue with any other prefs. Probably it's related to the experiment's prefs that are used.
• Attached a screen recording with the issue.

Comment 1

[Michael Cooper [:mythmon]]

I believe I have found the issue. Looking at Normandy stage recipe 963 I see several prefs changing. Notably, there is both browser.urlbar.update1 and browser.urlbar.update1.interventions (and other prefs that start with browser.urlbar.update1.. To be specific, one pref is a prefix of another in the experiment. This is a pattern I haven't seen before, and is causing this issue.

For some background, to watch for preference changes, Normandy uses Services.prefs.addObserver. This method takes not a preference, but a preference prefix to watch. To quote the IDL docs (emphasis mine)

The preference on which to listen for changes. This can be the name of an entire branch to observe. e.g. Holding the "root" prefbranch and calling addObserver("foo.bar.", ...) will observe changes to foo.bar.baz and foo.bar.bzip

So when Normandy is registering preference observers for this experiment, it registers one observer for each pref under browser.urlbar.update1.*, and another observer that listens to everything in that list a second time. This results in the one preference change being observed twice, and Normandy being triggered to stop the experiment twice.

This is a weird case specific to this experiment, and shouldn't actually cause any trouble for the experiment. I don't think this needs to block any experiment. I also don't think this is a high priority to fix (nor am I sure quite how we would detect this issue...)

Comment 2

[Firefox Bug Husbandry Bot]

Updating Severity from minor to P4 per May 4th changes to Severity field and triage process.

Comment 3

[Salmana]

Discovering the perfect Christmas gifts at Christmasgifts.co.uk For Women is a delightful journey filled with thoughtful surprises and cherished moments. From sleek gadgets to cozy essentials, there's a festive gift waiting to bring joy and warmth to the holiday season. Cheers to spreading love and happiness with thoughtful presents this Christmas.

Christmasgifts.co.uk For Women at: https://christmasgifts.co.uk/gift-category/gifts-for-women/