1447025 - Preference Experiment is run when preference is userset while recipe expects default

Status: RESOLVED INVALID

(Firefox :: Normandy Client, defect)

Description

[roxleitan]

[Affected versions]: 
Firefox Beta 60.0b3, Firefox Beta 59.0b11

[Affected platforms]:
Windows 7 x64, Ubuntu 16.04 x64

[Prerequisites]:
You need access to New Admin interface (https://normandy-admin.stage.mozaws.net/control-new)
1. Set the app.normandy.dev_mode preference to true to run recipes immediately on startup.
2. Set the app.normandy.logging.level preference to 0 to enable more logging.
3. Set the security.content.signature.root_hash preference to DB:74:CE:58:E4:F9:D0:9E:E0:42:36:BE:6C:C5:C4:F6:6A:E7:74:7D:C0:21:42:7A:03:BC:2F:57:0C:8B:9B:90.
4. Set the preference value for app.normandy.api_url set to https://normandy.stage.mozaws.net/api/v1

[Steps to reproduce]:
1.Open Firefox, about:config and pick a preference you will want to change using normandy preference experiment
2.Open Admin Interface, create a new preference experiment using "user" preference branch type
3.Add two branches with distinct values 
4.Save, Approve and Publish the recipe
5.Restart Firefox
6.Open about:config and search for the experiment preference value

[Expected result]:
The recipe should not run since the client does not match the Preference Branch Type

[Actual result]:
The recipe is run, the preference value is unmodified.

[Note1]:
This reproduces also for the "default" preference branch type:
{"ro9":{"name":"ro9","branch":"branch9","expired":false,"lastSeen":"2018-03-19T15:33:16.148Z","preferenceName":"accessibility.support.url","preferenceValue":"999","preferenceType":"string","previousPreferenceValue":"https://support.mozilla.org/%LOCALE%/kb/accessibility-services","preferenceBranchType":"default","experimentType":"exp"}}

[Note2]:
Reproducible also on:
Shield Recipe Client	80	[e-mail]
Version 	59.0b11 Build ID 	20180219114835

Comment 1

[Michael Cooper [:mythmon]]

> [Expected result]:
> The recipe should not run since the client does not match the Preference Branch Type

This is not correct. The "preference branch type" is not a filter that controls whether or not a recipe executes. It is control over how the preference is changed. Specifically, with the preference branch type set to default, it affects the default branch of preferences, where browser defaults are stored. User values on the preference would override the study values.