Closed Bug 1388678 Opened 7 years ago Closed 5 years ago

[Shield] Opt-out Study: Onboarding Tour

Categories

(Shield :: Shield Study, enhancement)

Product:
Shield
Component:
Shield Study
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: chsiang, Assigned: chsiang, NeedInfo)

References

(
URL
)

Details

User Story

Milestones:

Experiment period - 9/11-9/22 US time
Analysis of the tour and the heartbeat survey- 9/25-9/28
Survey link: http://qsurvey.mozilla.com/s3/onboarding-to-day6
We believe we could retain users better if we show the onboarding tour in the right order and timing. 

In this add-on study, we would like to change the order of 56 tour and the timing between each tour and see which combination retains users the best over the two week period. We would also run the same set of testings in different countries and see if they perform differently.
Summary: [Shield] Opt-in/Opt-out Study: Onboarding Tour → [Shield] Opt-out Study: Onboarding Tour
How are we expecting analysis, feedback, and pref changes before the Shield study ships?  Is that supposed to be 8/21 not 9/21?
Flags: needinfo?(chsiang)
User Story: (updated)
Flags: needinfo?(chsiang)
(In reply to Rebecca Weiss from comment #1)
> How are we expecting analysis, feedback, and pref changes before the Shield
> study ships?  Is that supposed to be 8/21 not 9/21?

yes i got that date wrong. just fixed it.
No longer blocks: 1390450
Depends on: 1390450
Requesting approval from shield study owner and data steward. 
The design of the study, the user recruitment methodology, and debrief survey are covered in the PHD and discussed with gregg, matt, cmore, saptarshi, jennifer, and kamyar.
Flags: needinfo?(sguha)
Flags: needinfo?(rweiss)
Flags: needinfo?(mgrimes)
Flags: needinfo?(kardekani)
Flags: needinfo?(glind)
User Story: (updated)
Will only test in US, not other countries now
r+

1) Is there documentation that describes the schema for the ultimate data set available publicly, complete and accurate? (see here, here, and here for examples)
Yes, see the PHD for this study.

2) Is there a control mechanism that allows the user to turn the data collection on and off? (Note, for data collection not needed for security purposes, Mozilla provides such a control mechanism)   
Yes, unenrollment via the Shield mechanism.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?
No permanent data collection.

4) Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?  
Category 1 and 2, with some self-reports.

5) Is the data collection default-on or default-off? 
This experiment will be run on release as an opt-out study.  For those enrolled participants, it is default-on.

6) Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)? 
No.

7) Is the data collection covered by the existing Firefox privacy notice? 
Yes

8) Does there need to be a check-in in the future to determine whether to renew the data? 
No
Flags: needinfo?(rweiss)
User Story: (updated)
User Story: (updated)
Ready to ship on 9/11!
Flags: needinfo?(sguha)
Flags: needinfo?(kardekani)
Flags: needinfo?(mgrimes)
56 is landing in Release on Thurs. I plan on ending this experiment today/tomorrow. Have the surveys gone out yet? Is :joy ready to begin analysis?
Flags: needinfo?(glind) → needinfo?(rrayborn)
Hey folks. Can we get a quick recap of the results and then have this bug closed? A couple sentences is fine.
Flags: needinfo?(chsiang)
Robert is likely to have more details on this :)
Flags: needinfo?(chsiang)

the tour has since changed. closing this out as this particular study is done.

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.