Closed
Bug 1502078
Opened 8 years ago
Closed 8 years ago
Expired preference experiment do not set the initial state in case of non-existent preferences
Categories
(Firefox :: Normandy Client, defect)
Firefox
Normandy Client
Tracking
()
RESOLVED
DUPLICATE
of bug 1502410
People
(Reporter: danibodea, Unassigned)
References
Details
(Keywords: regression)
[Description]:
- Expired preference experiment do not set the initial state in case of non-existent preferences.
[Affected versions]:
-(not affected) Nightly v62.0a1 (2018-05-31) build ID: 20180531220139
-(affected) Nightly v65.0a1 (2018-10-25)
-(affected) Release v63.0
[Affected platforms]:
- most likely all, tested on Windows 10
[Pre-conditions:]
user_pref("security.content.signature.root_hash", "DB:74:CE:58:E4:F9:D0:9E:E0:42:36:BE:6C:C5:C4:F6:6A:E7:74:7D:C0:21:42:7A:03:BC:2F:57:0C:8B:9B:90");
user_pref("app.normandy.api_url", "https://normandy.stage.mozaws.net/api/v1");
user_pref("app.normandy.dev_mode", true);
user_pref("app.normandy.logging.level", 0);
[Steps to reproduce]:
1. Create/Publish a pref-experiment that sets a preference that doesn’t exist.
2. Run a firefox client with the pre-conditions.
3. Disable the pref-experiment
4. Restart the client.
[Expected result]:
2. Preference is created and the experiment value is set.
4. Experiment expires -> not seen, Preference and its value are removed.
[Actual result]:
- the pref. Experiment is shown as not-seen / expired.
- the preference still exists.
- the preference value is still the one set by the experiment.
[Additional Information:]
for the non-existent preference experiment:
-preference not removed, value not reset
-normandy start-up preference not removed: app.normandy.startupExperimentPrefs.SVpref
-Value for shield-preference-experiments.json:
{"experidani":{"name":"experidani","branch":"default","expired":true,"lastSeen":"2018-10-25T12:19:38.013Z","preferenceName":"SVpref","preferenceValue":100,"preferenceType":"integer","previousPreferenceValue":null,"preferenceBranchType":"default","experimentType":"exp"},
for the preference experiment that affects existent preferences:
-preference value reset to initial value
-normandy start-up preference not removed: app.normandy.startupExperimentPrefs.accessibility.delay_plugin_time
-Value for shield-preference-experiments.json:
"experidani2":{name":"experidani2","branch":"var1","expired":true,"lastSeen":"2018-10-25T12:35:44.086Z","preferenceName":"accessibility.delay_plugin_time","preferenceValue":500,"preferenceType":"integer","previousPreferenceValue":10000,"preferenceBranchType":"default","experimentType":"exp"}}
[Regression range]:
Not available.
Comment 1•8 years ago
|
||
The revision in question is this one: https://normandy.stage.mozaws.net/api/v3/recipe_revision/973/
The arguments causing this are
{
"preferenceBranchType": "default",
"slug": "experidani",
"experimentDocumentUrl": "https://www.google.com/",
"isHighVolume": false,
"preferenceName": "SVpref",
"preferenceType": "integer",
"isEnrollmentPaused": false,
"branches": [
{
"slug": "default",
"value": 100,
"ratio": 1
},
{
"slug": "variant1",
"value": 11,
"ratio": 1
},
{
"slug": "variant2",
"value": 22,
"ratio": 1
},
{
"slug": "variant3",
"value": 33,
"ratio": 1
}
]
}
Comment 2•8 years ago
|
||
I was able to reproduce this following the steps in comment 0. While doing so I noticed the traceback for bug 1502182, so possible these two bugs are related.
I tried to write a test for this, but haven't been able to reproduce in a mochitest. I'm going to switch gears to 1502182, and see if I can get reproduce that in tests and fix it. Perhaps it will also fix this bug.
Comment 3•8 years ago
|
||
Investigating the scenario for the existent preferences for which a pref-experiment is conducted, bug 1502410 was logged. The same regression range for bug 1502410 is confirmed for this bug as well:
last good build: 63.0a1 (2018-07-14)
first bad build: 63.0a1 (2018-07-15)
https://hg.mozilla.org/mozilla-central/pushloghtml?startdate=105+days+ago&enddate=103+days+ago
Updated•8 years ago
|
status-firefox-esr60:
affected → ---
Tracked as a blocking issue for 63 until we fully understand the impact.
tracking-firefox63:
--- → blocking
Comment 5•8 years ago
|
||
Adrian, Michael, does it affect both Desktop and Fennec or only Desktop?
Flags: needinfo?(mcooper)
Flags: needinfo?(adrian.florinescu)
Comment 6•8 years ago
|
||
Normandy is not available for mobile at the current time, therefore it just affects Desktop.
Flags: needinfo?(mcooper)
Flags: needinfo?(adrian.florinescu)
Comment 7•8 years ago
|
||
I'm now convinced that this bug and bug 1502410 are duplicates of eachother. There is more progress made in the other bug, so I'm duping this one to it. I'll transfer over the tracking metadata as well.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → DUPLICATE
| Reporter | ||
Comment 8•8 years ago
|
||
These are my steps on the try build from bug 1502410:
1. Create and Publish a pref-experiment that sets a preference that doesn’t exist.
2. Run a firefox client with the pre-conditions.
3. Disable the pref-experiment
4. Restart the client.
[Actual result]:
- the pref-experiment is shown as not-seen / expired.
- the preference still exists.
- the preference value is still the one set by the experiment.
5. Restart the client again.
- the pref. is ONLY NOW removed
| Reporter | ||
Comment 9•8 years ago
|
||
Furthermore, I have tested the situation where the recipe is disabled, the bug is reproduced on the affected build, then the fixed build is opened to see if the issue is fixed for the users that reproduced it with older recipes.
STR:
1. Create/Publish a pref-experiment that sets a preference that doesn’t exist.
2. Run the affected build (latest nightly v64.0a1 from 2018-10-30) with the pre-conditions.
Notice that: Preference is created and the experiment value is set.
3. Disable the pref-experiment
4. Restart the affected build 2 times to be sure that the issue reproduced.
Notice that: The current issue reproduces (the created pref is not removed).
5. Take the try build from bug 1502410 and open it using the same profile.
Notice that the pref-experiment pref is not yet removed.
6. Restart the client (fixed try build).
Notice that: Only now the pref-experiment created pref gets removed.
As a conclusion, having the fix applied for the users that were enrolled in pref-experiments that were disabled and the issue occurred, the issue is fixed after 2 restarts. (The incorrectly remaining pref is removed.)
Updated•7 years ago
|
tracking-firefox63:
blocking → ---
Updated•7 years ago
|
You need to log in
before you can comment on or make changes to this bug.
Description
•