1811084 - (group-reach-sizing) launch experiment to understand group sizing for startup defaultBrowserCheck with users w/default # of bookmarks

Status: RESOLVED FIXED

(Firefox :: Messaging System, task, P1)

Description

[Dan Mosedale (:dmosedale, :dmose)]

This is for launching an experiment to help us better understand how many infrequent users would see messages with various combinations of FxMS groups when triggered by defaultBrowserCheck to help size the Import Bookmarks following experiments that this bug blocks.

From the "Action" section of https://docs.google.com/document/d/1_ISWCqKJjOEituU0Vt75rs71gT_F6hEj8mv6Ad0z8PM/edit# :

Run a 3 branch no-message research experiment to help us understand the size of the current sample pool available
Treatment A: messagegroup eco
Treatment B: messagegroup import-spotlights
Treatment C: messagegroup import-spotlights + messagegroup eco

Timing: Launch ASAP

Caveat: This measure would be a point in time, it is likely that the sample pool increases after these experiments as more users end their cool-down period.

More details shortly...

Comment 1

[Emily McMinn :emcminn]

Tentative experiment recipe:

https://experimenter.services.mozilla.com/nimbus/import-spotlight-and-eco-group-reach-experiment/summary

Comment 2

[Dan Mosedale (:dmosedale, :dmose)]

I've linked to an partial experiment draft. Writing a draft mostly was helpful in that it acted as a checklist. Although review on the doc would be nice (and I'm not against it), what I think is most likely to be helpful is answers to the questions in the right margin...

Comment 3

[Dan Mosedale (:dmosedale, :dmose)]

10% pre-experiment launched; waiting for QA feedback to replace with a 100% launch.

Comment 4

[Dan Mosedale (:dmosedale, :dmose)]

QA signed off green; we've ended the 10% and launched the real thing.