1592107 - Update utm term utm_campaign value by appending channel name for first run about:welcome experiments

Status: VERIFIED FIXED

(Firefox :: Messaging System, enhancement, P1)

Description

[Punam Dahiya [:pdahiya]]

about:welcome experiments in 71 are controlled from firefox not normandy, and the normal build process makes the very late beta/rcs appear like a regular release build.

To indicate to amplitude requests coming from first run experiments are from specific channel beta vs release — append channel in a utm term utm_campaign e.g. utm_campaign=firstrun-beta

As of now utm_campaign value is set to 'firstrun' and this change will update utm_campaign value to 'firstrun-<channel>' where channel is value such as nightly, beta or release picked from pref 'app.update.channel'

Comment 1

[Ed Lee :Mardak]

Is there a good way to get / pass the release channel to asrouter content? If it doesn't already exist, I suppose app.update.channel could be exposed as a redux Pref value ?

Comment 2

[Andrei Oprea [:andreio]]

That sounds like the easiest way to go.

Comment 3

[Punam Dahiya [:pdahiya]]

Attachment: Bug 1592107 - Append channel name to utm_campaign parameter for about:welcome experiments

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f3e7b3551c54
Append channel name to utm_campaign parameter for about:welcome experiments r=Mardak

Comment 5

[Punam Dahiya [:pdahiya]]

Steps for QA:
1. Create a new profile
2. Verify trailhead.firstrun.branches is empty
3. Set app.normandy.user_id to one of the values https://bugzilla.mozilla.org/show_bug.cgi?id=1590504#c3 to trigger respective about:welcome variant.
4. Restart and open about:welcome
5. Click on links on about:welcome page and verify utm_campaign parameter value sent in url opened is set to 'firstrun-<channel>' where channel is value from pref 'app.update.channel'
6. Enter email address and trigger Signin or Signup flow and verify utm_campaign value in the url is set correctly. Validate user is able to successfully complete signin and signup
7. Verify click on triplets cards on new tab and full page sends utm_campaign value as 'firstrun-<channel>'

Trigger above steps and verify for both modal and full page variant of about:welcome

Comment 6

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/f3e7b3551c54

Comment 7

[Punam Dahiya [:pdahiya]]

[Tracking Requested - why for this release]:
To support 71 first run about:welcome experiments and differentiate users by channel in FxA data analysis in Amplitude.

Comment 8

[Andreea Cupsa [:acupsa], Experiments QA]

I have verified this issue with the steps provided in Comment 5 on Latest Firefox Nightly 72.0a1 (Build ID: 20191103213857) on Windows 10 x64, Mac 10.14.6 and Arch Linux.
The utm_campaign value is correctly set to the value from the app.update.channel preference when triggering the Signup flow, and when clicking on the triplet cards both from the modals and the full page variants.

Comment 9

[Punam Dahiya [:pdahiya]]

Attachment: Bug 1592107 - Append channel name to utm_campaign for about:welcome experiments

Comment 10

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/f3e7b3551c54

Comment 11

[Ed Lee :Mardak]

Is attachment 9106382 [details] the same as the one that landed in m-c? If so, there doesn't need to be a separate patch for mozilla-beta if it applies cleanly to mozilla-beta already.

Comment 12

[Punam Dahiya [:pdahiya]]

(In reply to Ed Lee :Mardak from comment #11)

Is attachment 9106382 [details] the same as the one that landed in m-c? If so, there doesn't need to be a separate patch for mozilla-beta if it applies cleanly to mozilla-beta already.

Wasn't sure as patch includes bundle file, just verified https://phabricator.services.mozilla.com/D50968 applies cleanly to beta. Will abandon 9106382 and submit beta approval request for D50968.

Comment 13

[Punam Dahiya [:pdahiya]]

Comment on attachment 9104963 [details]
Bug 1592107 - Append channel name to utm_campaign parameter for about:welcome experiments

Beta/Release Uplift Approval Request

• User impact if declined: Impacts data analysis in FxA, patch helps to differentiate late beta users from release users
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: https://bugzilla.mozilla.org/show_bug.cgi?id=1592107#c5
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Specific to data analysis and changes existing utm parameter value by appending channel name from pref 'app.update.channel'
• String changes made/needed: None

Comment 14

[Pascal Chevrel (relman team) -> :pascalc]

Comment on attachment 9104963 [details]
Bug 1592107 - Append channel name to utm_campaign parameter for about:welcome experiments

Low risk, covered by tests, uplift approved for 71 beta 8, thanks.

Comment 15

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/releases/mozilla-beta/rev/49d7ce4e016a

Comment 16

[Andreea Cupsa [:acupsa], Experiments QA]

I have verified this issue with the steps provided in Comment 5 on Firefox Beta 71.0b8 (Build ID: 20191107101713) on Windows 10 x64, Mac 10.14.6 and Arch Linux.
The utm_campaign value is correctly set to the value from the app.update.channel preference when triggering the Signup flow, and when clicking on the triplet cards both from the modals and the full page variants.