Closed Bug 598653 Opened 14 years ago Closed 13 years ago

Search Study: Investigate why some users switched experiment group mid-study

Categories

(Mozilla Labs Graveyard :: Test Pilot Studies, defect)

Product:
Mozilla Labs Graveyard
Component:
Test Pilot Studies
Platform:
x86
All
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: hulmer, Unassigned)

Details

(Whiteboard: Search UI) 

We've found that around 10% of users in a 35,000 user sample switch experimental groups.  This is not coincident with the study version change - they often occur before or after.  It does not appear that there is any particular experimental group that changes more often - seems to be a general bug.
Whiteboard: Search UI
The code that looked at whether you had already been assigned an experiment group or not was incorrectly interpreting "0" as "false", meaning that if this code ran when you were in experiment group 0, it would assign you a new group.

This was fixed by http://hg.mozilla.org/labs/testpilotweb/rev/68c1542aa15b

but that would only explain changes from group 0 to other groups.  Were there any group ID changes in the data set that started in a non-zero group?  If so, there must still be another bug.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Product: Mozilla Labs → Mozilla Labs Graveyard
You need to log in before you can comment on or make changes to this bug.