1650104 - Enable reach ping for Moments Page experiments

Status: VERIFIED FIXED

(Firefox :: Messaging System, enhancement, P1)

Comment 1

[Nan Jiang [:nanj]]

Attachment: Bug 1650104 - Enable reach event for Moments Page experiments r=andreio

Comment 2

[Nan Jiang [:nanj]]

QA steps:

• Install RemoteSettings devtools and switch to the stage environment, clear the local data for the messaging-experiment collection and force it to sync data with the backend
• There is a test Moments Page experiment called XMAN-MOMENTS-PAGE-80-NIGHTLY, make sure you are testing it in Nightly 80 with the "en-US" locale (as it's targeting this specific group)
• Restart the browser to let the Experiment Manager do enrollment
• Verify the experiment enrollment either through about:studies or checking the enrollment telemetry on about:telemetry#events-tab
• If enrolled in the treatment branch, verify that restarting the browser again should open the Moments Pages "https://www.mozilla.org/en-US/firefox/welcome/1/"
• If enrolled in the control branch, verify that the "reach" event gets recorded. You should be able to find it on "about:telemetr#events-tab" with category as "messaging_experiments", method as "reach", object as "moments_page", Value as "xman-moments-page-80-nightly", and extra as "{"branches": "treatment"}"

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c0040da2e779
Enable reach event for Moments Page experiments r=andreio

Comment 4

[Nan Jiang [:nanj]]

[Tracking Requested - why for this release]: this allows us to collect the reach telemetry for the Moments Page experiments.

Comment 5

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/c0040da2e779

Comment 6

[Nan Jiang [:nanj]]

Comment on attachment 9162244 [details]
Bug 1650104 - Enable reach event for Moments Page experiments r=andreio

Beta/Release Uplift Approval Request

• User impact if declined: We won't be able to collect the "reach" telemetry for the Moments Page experiments.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This is not a user-facing feature, but an addition to an existing telemetry for the Messaging System experiment platform.
• String changes made/needed: None

Comment 7

[Ryan VanderMeulen [:RyanVM]]

(In reply to Nan Jiang [:nanj] from comment #6)

• Needs manual test from QE?: No

I assume this was a typo given comment 2. Anyway, we should definitely try to verify that the Telemetry data looks good prior to shipping 79.

Comment 8

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9162244 [details]
Bug 1650104 - Enable reach event for Moments Page experiments r=andreio

Approved for 79.0b7.

Comment 9

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/fbe8c3891498

Comment 10

[Marius Coman [:mcoman], Ecosystem QA]

I have verified this enhancement using the steps from comment 2 and I can confirm the following:
- If enrolled in the treatment branch, the "https://www.mozilla.org/en-US/firefox/welcome/1/" page is displayed after a browser restart.
- If enrolled in the control branch, the 'reach' event is displayed on the "about:telemetry#events-tab" page. The event contains the category as "messaging_experiments", method as "reach", object as "moments_page", value as "xman-moments-page-80-nightly", and extra as "{"branches": "treatment"}"

Verified using the latest Firefox Beta (79.0b7 Build ID - 20200710175519) and the latest Firefox Nightly (80.0a1 Build ID - 20200712214458) installed on Windows 10 x64, Mac 10.15, and Ubuntu 18.04 x64.