Document what made it possible for the User Journey's 2 branch confirmation experiment control branch unenrollment to happen
Categories
(Shield :: Shield Study, task)
Tracking
(Not tracked)
People
(Reporter: cmuresan, Unassigned)
Details
In the interest of other people not repeating what happened with the User Journey's 2 branch confirmation experiment (users updating from 67 to 68 got kicked out of the control branch) it would be nice if the limitations that were encountered there could be documented in the Experiments Mana page.
@mythmon, I think that you might be the best person to reach out to in order to document this as you know the intricacies of the Normandy feature and what can and can't happen, as well as why (technically) this happened. Do you think you can find some time for this?
|
||
Comment 1•6 years ago
|
||
I added some notes about this to the Normandy Recipe Best Practices document.
In short, the combination of a user-branch study, and a default branch that changes is not well supported by Normandy, and should be avoided. If it can't be avoided, the control branch needs to be modified to not equal the built-in default. If neither of these are possible, the experiment is difficult-to-impossible to run successfully.
|
Reporter | |
Updated•6 years ago
|
Description
•