1590504 - Modal variant with 'Start Here' and 'Meet the rest of Firefox' UI

Status: VERIFIED FIXED

(Firefox :: Messaging System, enhancement, P1)

Description

[Punam Dahiya [:pdahiya]]

Based on data science experiment design feedback, add a new modal variant with 'Start Here' and 'Meet the rest of Firefox' language to better compare full page modal less and modal variant of welcome page

https://bugzilla.mozilla.org/show_bug.cgi?id=1583262#c7

modal_variant_f is same as modal_control with only difference being it uses 'Start Here' in signup form instead of 'Join Firefox' string

Comment 1

[Punam Dahiya [:pdahiya]]

Attachment: Bug 1590504 - About welcome modal variant with Start Here and Meet the rest of Firefox UI

Comment 3

[Ed Lee :Mardak]

If you want to manually test the experiment branches, here's some string values for app.normandy.user_id that trigger various variants:

10 modal_control
 2 modal_variant_a
15 modal_variant_b
 5 modal_variant_c
 8 modal_variant_f
 1 full_page_d
 3 full_page_e

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c142790318c4
About welcome modal variant with Start Here and Meet the rest of Firefox UI r=Mardak

Comment 5

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/c142790318c4

Comment 6

[Punam Dahiya [:pdahiya]]

Steps for QA to verify modal_variant_f experiment branch

1. Create a new profile
2. Set trailhead.firstrun.branches to modal_variant_f-supercharge
3. Restart and open about:welcome

UI should match as noted in modal_varaint_f experiment branch in tech plan here https://docs.google.com/document/d/1MtWXnenZ8poPr1XSeR9ztDgjlyNcOUvPiW0_Y8dVibg/edit#heading=h.bexw5m85abfi

To test experiment branch and respective prefs are getting set correctly, we can use app.normandy.user_id as described in comment #3

1. Create a new profile
2. Verify trailhead.firstrun.branches is empty
3. Set app.normandy.user_id to 8
4. Restart and open about:welcome
5. Verify pref trailhead.firstrun.didSeeAboutWelcome is set to true and trailhead.firstrun.interruptsExperiment value is set to 'modal_variant_f'
6. about:welcome UI should match modal_variant_f as noted in tech plan link above

Comment 7

[Punam Dahiya [:pdahiya]]

[Tracking Requested - why for this release]:
This fix is needed for 71 first run about:welcome experiments. As per data science feedback, patch implements a new experiment branch 'modal_variant_f' with messaging change to better compare modal control variant with modal less full page about:welcome.

Comment 8

[Pascal Chevrel:pascalc (PTO until April 26)]

Punam, please prepare an uplift request if the patch has to be in 71. Thanks

Comment 9

[Robert Martin [:romartin], Ecosystem QA]

I have verified that the modal-f is properly displayed using the latest Firefox Nightly (72.0a1 Build ID - 20191028094851) installed, on Windows 10 x64, Arch Linux and Mac 10.15.

Comment 10

[Punam Dahiya [:pdahiya]]

Comment on attachment 9103369 [details]
Bug 1590504 - About welcome modal variant with Start Here and Meet the rest of Firefox UI

Beta/Release Uplift Approval Request

• User impact if declined: This patch is needed to run about:welcome experiment logic in 71 by emptying pref 'trailhead.firstrun.branches'. Patch implements a new experiment branch 'modal_variant_f' to better compare modal variant with modal less full page about:welcome
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: https://bugzilla.mozilla.org/show_bug.cgi?id=1590504#c6
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Small patch introducing a new experiment branch very similar to existing default about:welcome, with only difference being text as 'Start Here' in signup form instead of 'Join Firefox'
• String changes made/needed: None

Comment 11

[Pascal Chevrel:pascalc (PTO until April 26)]

Comment on attachment 9103369 [details]
Bug 1590504 - About welcome modal variant with Start Here and Meet the rest of Firefox UI

P1, has tests and verified on nightly by QA, uplift approved for 71 beta 6, thanks.

Comment 12

[Dorel Luca [:dluca]]

https://hg.mozilla.org/releases/mozilla-beta/rev/1c04ad803c62

Comment 13

[Robert Martin [:romartin], Ecosystem QA]

I have verified that the modal-f is properly displayed using the latest Firefox Beta (71.0b6 Build ID - 20191031113544) installed, on Windows 10 x64, Arch Linux and Mac 10.15.