Closed Bug 1393628 Opened 7 years ago Closed 7 years ago

Staged roll-out of full TP preferences in release 57

Categories

(Firefox :: Protections UI, defect, P1)

Product:
Firefox
Component:
Protections UI
Type:
defect

Tracking

()

Status:
RESOLVED WONTFIX
Tracking Flags:
Tracking Status
firefox57 + wontfix

People

(Reporter: francois, Assigned: francois)

References

Details

This will consist of setting this pref:

  privacy.trackingprotection.ui.enabled = true

and will either be done in a system add-on (https://wiki.mozilla.org/Firefox/Go_Faster/System_Add-ons/Process) or via the upcoming Shield mechanism for staged roll-outs.
Decision on which mechanism to use will be made by 2017-09-11.
Depends on: 1394003
After talking to Kamyar, it's now clear that since this is a simple pref flip, the easiest mechanism would be a Shield experiment:

1. ship an experiment which turns the pref on (no control group)
2. gradually increase the percentage of users who are enrolled in the experiment
3. once we're ready to go to 100%, flip the pref in a hotfix or point release
4. stop the experiment
Let's plan for using Shield.  (I'll file the follow-up bug)
Let's plan for a 10% user deployment, monitor uptake of privacy.trackingprotection.ui.enabled = true for 3 weeks since profile creation date.  

Full deployment will be based on uptake in the period (for the experiment group) being no more than 4% higher relative to those not in the experiment/control.
[Tracking Requested - why for this release]: Not sure if you want to track this since it's only a pref flip to be done via Shield.
tracking-firefox57: --- → ?
Priority: -- → P1
Thanks François, probably worth keeping an eye here.
status-firefox57: --- → affected
tracking-firefox57: ? → +
(In reply to Peter Dolanjski [:pdol] from comment #4)
> Full deployment will be based on uptake in the period (for the experiment
> group) being no more than 4% higher relative to those not in the
> experiment/control.

Thinking this through further - because the number is so low to start with 4% relative difference doesn't make sense.  We'll use a 10% relative difference.
Depends on: 1400980
(In reply to Peter Dolanjski [:pdol] from comment #4)
> Let's plan for a 10% user deployment, monitor uptake of
> privacy.trackingprotection.ui.enabled = true for 3 weeks since profile
> creation date.  

The pref we need to monitor is this one:

  privacy.trackingprotection.enabled = true
(In reply to Peter Dolanjski [:pdol] from comment #7)
> (In reply to Peter Dolanjski [:pdol] from comment #4)
> > Full deployment will be based on uptake in the period (for the experiment
> > group) being no more than 4% higher relative to those not in the
> > experiment/control.
> 
> Thinking this through further - because the number is so low to start with
> 4% relative difference doesn't make sense.  We'll use a 10% relative
> difference.

Going through the beta and release numbers, we're revising this threshold to 30% since the % of session that use TP now is so low (1.5%).
Blocks: 1393627
Hi Francois, I am reviewing the tracked bugs for 57. When will this pref flip be done? Is it during the beta cycle or post go-live? Thanks!
Flags: needinfo?(francois)
(In reply to Ritu Kothari (:ritu) from comment #10)
> Hi Francois, I am reviewing the tracked bugs for 57. When will this pref
> flip be done? Is it during the beta cycle or post go-live? Thanks!

Post go-live, via Shield.
Flags: needinfo?(francois)
(In reply to François Marier [:francois] from comment #11)
> (In reply to Ritu Kothari (:ritu) from comment #10)
> > Hi Francois, I am reviewing the tracked bugs for 57. When will this pref
> > flip be done? Is it during the beta cycle or post go-live? Thanks!
> 
> Post go-live, via Shield.

Correction, it's already enabled for 10% of Beta users and tracked in bug 1393628.

This bug however is to track the post go-live release experiment using Shield.
Hi Francois, SHIELD cannot be used to rollout to 100% of a channel. Would you be able to share here/email what the staged rollout plan for this looks like? If you are planning to rollout gradually and eventually ramp up to all of release users then you will need to write your own system add-on instead of using SHIELD. If it is a system add-on option, I think we should consider implementing and testing it now.
Flags: needinfo?(francois)
(In reply to Ritu Kothari (:ritu) from comment #13)
> Hi Francois, SHIELD cannot be used to rollout to 100% of a channel. Would
> you be able to share here/email what the staged rollout plan for this looks
> like? If you are planning to rollout gradually and eventually ramp up to all
> of release users then you will need to write your own system add-on instead
> of using SHIELD. If it is a system add-on option, I think we should consider
> implementing and testing it now.

The plan was to roll-out via Shield as much as we can, and then when/if we're ready to go 100%, we'll do that via a point release.
Flags: needinfo?(francois)
How do we determine when/if we're ready to go 100%?
Assignee: nobody → francois
(In reply to Wennie from comment #15)
> How do we determine when/if we're ready to go 100%?

Using the threshold defined in comment 9.
What is the plan to get to 30% in release 57??
(In reply to Wennie from comment #17)
> What is the plan to get to 30% in release 57??

We're shipping the UI experiment to 10% of users. If the uptake of the feature is within Peter's threshold (max 30% increase over current level), then we ship the UI to everyone.
A staged roll-out is no longer needed: https://bugzilla.mozilla.org/show_bug.cgi?id=1393627#c5
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WONTFIX
Update tracking flag per comment 19.
status-firefox57: affected → wontfix
You need to log in before you can comment on or make changes to this bug.