1241591 - Switch onboarding to A version only

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: General, defect)

Description

[Chenxia Liu [:liuche]]

We concluded the testing for the Import/Welcome firstrun panels experiences, and since they were the same in terms of retention, we're going to turn this experiment off and go with onboarding A (which is simpler).

Comment 1

[Chenxia Liu [:liuche]]

Attachment: MozReview Request: Bug 1241591 - Switch onboarding to A version only. r=margaret

Review commit: https://reviewboard.mozilla.org/r/31813/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/31813/

Comment 2

[:Margaret Leibovic]

Comment on attachment 8710626 [details]
MozReview Request: Bug 1241591 - Switch onboarding to A version only. r=margaret

https://reviewboard.mozilla.org/r/31813/#review28547

::: gradle/wrapper/gradle-wrapper.properties:6
(Diff revision 1)
> -distributionUrl=https\://services.gradle.org/distributions/gradle-2.7-all.zip
> +distributionUrl=https\://services.gradle.org/distributions/gradle-2.7-bin.zip

This has happened to me, too, recently. You should remove these changes (and let's follow up with nalexander about why this is happening to us).

::: mobile/android/base/java/org/mozilla/gecko/firstrun/FirstrunPagerConfig.java
(Diff revision 1)
> -            }

This is a simple fix to uplift. I would say we should do more to remove the experiment, but we're going to be changing it in 46, so no need to do that.

Comment 3

[:Margaret Leibovic]

Let's uplift this.

Ideally we would have decided a winner sooner so that we could ship 44 with only onboarding A as well.

I think this could be a good candidate for a ride-along if we're making another 44 build. NI to ritu to get this on her radar.

Comment 4

[Pulsebot]

https://hg.mozilla.org/integration/fx-team/rev/94daabff223d

Comment 5

[Chenxia Liu [:liuche]]

Comment on attachment 8710626 [details]
MozReview Request: Bug 1241591 - Switch onboarding to A version only. r=margaret

Approval Request Comment
[Feature/regressing bug #]: Turning off Import, which has caused some regressions
[User impact if declined]: More regressions from Import being highlighted in firstrun
[Describe test coverage new/current, TreeHerder]: local testing
[Risks and why]: very low, choosing one item in a if statement
[String/UUID change made/needed]: none

Comment 6

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/94daabff223d

Comment 7

[Sylvestre Ledru [:Sylvestre]]

Comment on attachment 8710626 [details]
MozReview Request: Bug 1241591 - Switch onboarding to A version only. r=margaret

Too late for 44

Comment 8

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to Sylvestre Ledru [:sylvestre] from comment #7)
> Comment on attachment 8710626 [details]
> MozReview Request: Bug 1241591 - Switch onboarding to A version only.
> r=margaret
> 
> Too late for 44

Margaret - Can't we just change SwitchBoard to send onboarding A to everyone?

Comment 9

[:Margaret Leibovic]

(In reply to Mark Finkle (:mfinkle) from comment #8)
> (In reply to Sylvestre Ledru [:sylvestre] from comment #7)
> > Comment on attachment 8710626 [details]
> > MozReview Request: Bug 1241591 - Switch onboarding to A version only.
> > r=margaret
> > 
> > Too late for 44
> 
> Margaret - Can't we just change SwitchBoard to send onboarding A to everyone?

The logic for the on-boarding A/B experiments hard-coded directly into the product so that we can read it on first run. Even if we switched the experiment bucketing, we would need to change it in the product here:
http://hg.mozilla.org/mozilla-central/diff/94daabff223d/mobile/android/base/java/org/mozilla/gecko/firstrun/FirstrunPagerConfig.java#l1.12

To reduce confusion about the source of the experiment behavior, we should really removing these onboarding experiments from the Swtichboard server config. Although we should make sure that "local" experiments are also included in the telemetry ping data.

Comment 10

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to :Margaret Leibovic from comment #9)
> (In reply to Mark Finkle (:mfinkle) from comment #8)
> > (In reply to Sylvestre Ledru [:sylvestre] from comment #7)
> > > Comment on attachment 8710626 [details]
> > > MozReview Request: Bug 1241591 - Switch onboarding to A version only.
> > > r=margaret
> > > 
> > > Too late for 44
> > 
> > Margaret - Can't we just change SwitchBoard to send onboarding A to everyone?
> 
> The logic for the on-boarding A/B experiments hard-coded directly into the
> product so that we can read it on first run. Even if we switched the
> experiment bucketing, we would need to change it in the product here:
> http://hg.mozilla.org/mozilla-central/diff/94daabff223d/mobile/android/base/
> java/org/mozilla/gecko/firstrun/FirstrunPagerConfig.java#l1.12

I forgot about that. Carry on.

Comment 11

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

I've seen this one but so far there are no drivers for a 44.0 RC build3. Given that, I don't think this one will be in Fx44. As Sylvestre said, it's too late.

Comment 12

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/releases/mozilla-aurora/rev/fa3cde129174