Closed Bug 1556391 Opened 6 months ago Closed 6 months ago

Preference experiments that are targeted for a specific channel no longer expire on channel switch

Categories

(Firefox :: Normandy Client, defect, major)

defect
Not set
major

Tracking

()

VERIFIED FIXED
Firefox 69
Tracking Status
firefox-esr60 --- unaffected
firefox67 --- unaffected
firefox68 --- verified
firefox69 --- verified

People

(Reporter: cmuresan, Assigned: glasserc)

Details

Attachments

(1 file)

[Affected versions]:

  • Firefox Beta 68.0b6 [Build ID 20190529145824]
  • Firefox Nightly 69.0b1 [Build ID 20190602221035]

[Affected Platforms]:

  • All Windows
  • All Mac
  • All Linux

[Prerequisites]:

  • Access to NORMANDY Delivery Console https://delivery-console.stage.mozaws.net/ is needed.
  • Set the app.normandy.dev_mode preference to true to run recipes immediately on startup.
  • Set the app.normandy.logging.level preference to 0 to enable more logging.
  • Set the security.content.signature.root_hash preference to DB:74:CE:58:E4:F9:D0:9E:E0:42:36:BE:6C:C5:C4:F6:6A:E7:74:7D:C0:21:42:7A:03:BC:2F:57:0C:8B:9B:90.
  • Set the preference value for app.normandy.api_url to https://stage.normandy.nonprod.cloudops.mozgcp.net/api/v1.
  • Set the services.settings.server preference to https://settings.stage.mozaws.net/v1.
  • An preference experiment recipe targeting the Release channel was already created and is live. ( eg #716 )
  • Have the browser console opened.

[Steps to reproduce]:

  1. Open the Release version of the browser with the profile targeted in the prerequisites.
  2. Wait for the recipes to be executed on the profile.
  3. Close the browser and open a Beta version with the same profile.
  4. Navigate to about:studies and observe the page.

[Expected result]:

  • The recipe from prerequisites is displayed as completed.

[Actual result]:

  • The recipe from prerequisites is still active.

[Notes]:

  • This issue is not reproducible on builds before 2019-05-16 which leads me to believe that either bug 1547034 or bug 1553198 has caused this.
  • No errors are displayed in the browser console.
  • The issue is not reproducible with add-on studies.
  • I used the filter expression: 'andreea'|preferenceExists and the following recipe: #716

@Ethan, since you worked on both bugs that I mentioned, could you please take a look at this?

Severity: normal → major
Flags: needinfo?(eglassercamp)

I think this means my fix from https://bugzilla.mozilla.org/show_bug.cgi?id=1553198 didn't actually work. The code refers to action instead of actionName. A fix should be straightforward. Let me work on it now.

Flags: needinfo?(eglassercamp)
Pushed by eglassercamp@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d5750bca9f67
Fix experiments that never expire r=mythmon

@Ciprian, when you get a chance, could you tell me if you can verify that this patch fixes the bug? If so, I'll request uplift.

Flags: needinfo?(cmuresan)

@Ethan, I just verified that the patch fixes the bug on Windows 10, Arch Linux 4.14.3, and macOS 10.14. I think we're good to go.

Flags: needinfo?(cmuresan)

Comment on attachment 9069448 [details]
Bug 1556391: Fix experiments that never expire

Beta/Release Uplift Approval Request

  • User impact if declined: Normandy cannot unenroll users from Preference Experiments.
  • Is this code covered by automated tests?: Yes
  • Has the fix been verified in Nightly?: Yes
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): This is a minor correction of https://bugzilla.mozilla.org/show_bug.cgi?id=1553198 and includes a test that increases confidence that these components are working correctly.
  • String changes made/needed:
Attachment #9069448 - Flags: approval-mozilla-beta?
Status: NEW → RESOLVED
Closed: 6 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 69
Assignee: nobody → eglassercamp

Comment on attachment 9069448 [details]
Bug 1556391: Fix experiments that never expire

scary. approved for 68.0b8.

Attachment #9069448 - Flags: approval-mozilla-beta? → approval-mozilla-beta+

I have verified that the issue is no longer reproducible on the latest Nightly 69.0a1 and on the latest Firefox Beta 68.0b8 (Build ID 20190606101422) on Windows 10, macOS 10.14, and Arch Linux 14.4.3.

Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.