1691126 - Save to Pocket button sign up buttons test

Status: VERIFIED FIXED

(Firefox :: Pocket, enhancement, P1)

Description

[Scott [:thecount] Downe]

We want to test a single signup button test.

We'll want a variant attached to this pref "extensions.pocket.loggedOutVariant"

For 1 button, and for a control.

We want to send the variant names to the utm params.

Comment 1

[Scott [:thecount] Downe]

[Tracking Requested - why for this release]:

Comment 2

[Scott [:thecount] Downe]

Attachment: Bug 1691126 - Save to Pocket button test.

Comment 3

[Scott [:thecount] Downe]

To test:

1. Log out of Pocket.
2. Set extensions.pocket.loggedOutVariant to button_control
3. Click the save to Pocket button on a page.
4. Click Sign up with Firefox or Sign up with email.

expected: should see in the url these params for both buttons "utm_source=button_control" and "utm_campaign=logged_out_button_test"

1. Change extensions.pocket.loggedOutVariant to button_variant
2. Click the save to Pocket button on a page.
3. Click Sign up

Expected: Should only have 1 button option, Sign up.
Expected: the url params should be "utm_source=button_variant" and "utm_campaign=logged_out_button_test"
Expected: Height of the panel should reasonably wrap the content.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f37df218fef6
Save to Pocket button test. r=gvn

Comment 5

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/f37df218fef6

Comment 6

[Scott [:thecount] Downe]

Comment on attachment 9201520 [details]
Bug 1691126 - Save to Pocket button test.

Beta/Release Uplift Approval Request

• User impact if declined: We want to run an experiment with the save to Pocket buttons, 1 button vs 2.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce:

1. Log out of Pocket.
2. Set extensions.pocket.loggedOutVariant to button_control
3. Click the save to Pocket button on a page.
4. Click Sign up with Firefox or Sign up with email.

expected: should see in the url these params for both buttons "utm_source=button_control" and "utm_campaign=logged_out_button_test"

1. Change extensions.pocket.loggedOutVariant to button_variant
2. Click the save to Pocket button on a page.
3. Click Sign up

Expected: Should only have 1 button option, Sign up.
Expected: the url params should be "utm_source=button_variant" and "utm_campaign=logged_out_button_test"
Expected: Height of the panel should reasonably wrap the content.

• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): It's a simple change and behind a pref
• String changes made/needed: None

Comment 7

[Marius Coman [:mcoman], Ecosystem QA]

I have verified this enhancement and I can confirm the following :

• The set extensions.pocket.loggedOutVariant pref set to button_control:

  • The "utm_source=button_control" and "utm_campaign=logged_out_button_test" params are successfully displayed inside the URL.

• The set extensions.pocket.loggedOutVariant pref set to button_variant:

  • Only one button is displayed in the "Saved to Pocket" doorhanger, more exactly the "Sign up" one.
  • The "utm_source=button_variant" and "utm_campaign=logged_out_button_test" params are successfully displayed inside the URL.
  • All the elements are correctly rendered inside the "Saved to Pocket" doorhanger.

Verified using the latest Firefox Nightly (87.0a1 Build ID - 20210208092848) installed on Windows 10 x64, Mac 10.15.7, and Linux Mint 20 x64.

Comment 8

[Pascal Chevrel:pascalc]

Comment on attachment 9201520 [details]
Bug 1691126 - Save to Pocket button test.

Approved for 86 beta 8, thanks.

Comment 9

[Julien Cristau [:jcristau]]

https://hg.mozilla.org/releases/mozilla-beta/rev/50d2731ec6a4

Comment 10

[Marius Coman [:mcoman], Ecosystem QA]

I have verified this enhancement and I can confirm the following :

• The set extensions.pocket.loggedOutVariant pref set to button_control:

  • The "utm_source=button_control" and "utm_campaign=logged_out_button_test" params are successfully displayed inside the URL.

• The set extensions.pocket.loggedOutVariant pref set to button_variant:

  • Only one button is displayed in the "Saved to Pocket" doorhanger, more exactly the "Sign up" one.
  • The "utm_source=button_variant" and "utm_campaign=logged_out_button_test" params are successfully displayed inside the URL.
  • All the elements are correctly rendered inside the "Saved to Pocket" doorhanger.

Verified using the latest Firefox Beta (86.0b8 Build ID - 20210209185733) installed on Windows 10 x64, Mac 10.15.7, and Linux Mint 20 x64.