1564153 - [Enterprise roots holdback study] Only Windows 10 users will be targeted if the OS filters from the previous experiment will be used

Status: VERIFIED FIXED

(Shield :: Shield Study, defect)

Description

[Carmen Fat [:cfat] - Ecosystem QA]

[Notes]:

• The testing recipe was created using the same filters from the previous recipe for "Pref-Flip: Retention/Engagement impact of enabling the Enterprise roots feature in the presence of an AV"
• We believe the previous pref experiment didn't manage to target the Windows 8 and 8.1 population as it seems that Normandy only acknowledges the Windows Release Version number, but not the actual Windows name. For example, Windows 8.1 is NT 6.3 and Windows 10 is NT 10.0.
• Please see the following link with the Release Version numbers corresponding to the Windows versions: https://en.wikipedia.org/wiki/List_of_Microsoft_Windows_versions

[Affected versions]:

• Release Candidate 68 - build 3

[Affected platforms]:

• Windows 8.1 x64
• Windows 8 x64

[Prerequisites]:

• Access to NORMANDY Delivery Console https://delivery-console.stage.mozaws.net/ is needed.
• Set the app.normandy.dev_mode preference to true to run recipes immediately on startup.
• Set the app.normandy.logging.level preference to 0 to enable more logging.
• Set the security.content.signature.root_hash preference to DB:74:CE:58:E4:F9:D0:9E:E0:42:36:BE:6C:C5:C4:F6:6A:E7:74:7D:C0:21:42:7A:03:BC:2F:57:0C:8B:9B:90.
• Set the preference value for app.normandy.api_url set to https://stage.normandy.nonprod.cloudops.mozgcp.net/api/v1.
• A preference experiment recipe was already created and is live (https://delivery-console.stage.mozaws.net/recipe/780/).

[Steps to reproduce]:

1. Open a browser profile that can be targeted by the previously set recipe.
2. Open the Browser Console (Ctrl+Shift+J).
3. Observe the output related to the recipe in the test.

[Expected result]:

• The following output is displayed:
  "1562589570955 app.normandy.recipe-runner INFO Executing recipe "Pref flip: Enterprise roots feature holdback study" (action=preference-experiment)
  1562589570958 app.normandy.preference-experiments DEBUG PreferenceExperiments.has(Enterprise_roots)
  1562589570959 app.normandy.preference-experiments DEBUG PreferenceExperiments.start(Enterprise_roots, control)
  1562589570959 app.normandy.preference-experiments DEBUG PreferenceExperiments.startObserver(Enterprise_roots)"

[Actual result]:

• The following output is displayed:
  "No recipes to execute".

Comment 1

[Carmen Fat [:cfat] - Ecosystem QA]

Considering that this issue was addressed in the new recipe and the experiment is live, this report can be closed.