Closed Bug 1650104 Opened 1 month ago Closed 1 month ago

Enable reach ping for Moments Page experiments

Categories

(Firefox :: Messaging System, enhancement, P1)

80 Branch
enhancement

Tracking

()

VERIFIED FIXED
Firefox 80
Iteration:
80.1 - June 29 - July 12
Tracking Status
firefox79 + verified
firefox80 --- verified

People

(Reporter: nanj, Assigned: nanj)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

No description provided.
Assignee: nobody → najiang
Iteration: --- → 80.1 - June 29 - July 12
Priority: -- → P1

QA steps:

  • Install RemoteSettings devtools and switch to the stage environment, clear the local data for the messaging-experiment collection and force it to sync data with the backend
  • There is a test Moments Page experiment called XMAN-MOMENTS-PAGE-80-NIGHTLY, make sure you are testing it in Nightly 80 with the "en-US" locale (as it's targeting this specific group)
  • Restart the browser to let the Experiment Manager do enrollment
  • Verify the experiment enrollment either through about:studies or checking the enrollment telemetry on about:telemetry#events-tab
  • If enrolled in the treatment branch, verify that restarting the browser again should open the Moments Pages "https://www.mozilla.org/en-US/firefox/welcome/1/"
  • If enrolled in the control branch, verify that the "reach" event gets recorded. You should be able to find it on "about:telemetr#events-tab" with category as "messaging_experiments", method as "reach", object as "moments_page", Value as "xman-moments-page-80-nightly", and extra as "{"branches": "treatment"}"
Pushed by najiang@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/c0040da2e779
Enable reach event for Moments Page experiments r=andreio

[Tracking Requested - why for this release]: this allows us to collect the reach telemetry for the Moments Page experiments.

Status: NEW → RESOLVED
Closed: 1 month ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 80

Comment on attachment 9162244 [details]
Bug 1650104 - Enable reach event for Moments Page experiments r=andreio

Beta/Release Uplift Approval Request

  • User impact if declined: We won't be able to collect the "reach" telemetry for the Moments Page experiments.
  • Is this code covered by automated tests?: Yes
  • Has the fix been verified in Nightly?: Yes
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): This is not a user-facing feature, but an addition to an existing telemetry for the Messaging System experiment platform.
  • String changes made/needed: None
Attachment #9162244 - Flags: approval-mozilla-beta?

(In reply to Nan Jiang [:nanj] from comment #6)

  • Needs manual test from QE?: No

I assume this was a typo given comment 2. Anyway, we should definitely try to verify that the Telemetry data looks good prior to shipping 79.

Flags: qe-verify+

Comment on attachment 9162244 [details]
Bug 1650104 - Enable reach event for Moments Page experiments r=andreio

Approved for 79.0b7.

Attachment #9162244 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
QA Whiteboard: [qa-triaged]

I have verified this enhancement using the steps from comment 2 and I can confirm the following:
- If enrolled in the treatment branch, the "https://www.mozilla.org/en-US/firefox/welcome/1/" page is displayed after a browser restart.
- If enrolled in the control branch, the 'reach' event is displayed on the "about:telemetry#events-tab" page. The event contains the category as "messaging_experiments", method as "reach", object as "moments_page", value as "xman-moments-page-80-nightly", and extra as "{"branches": "treatment"}"

Verified using the latest Firefox Beta (79.0b7 Build ID - 20200710175519) and the latest Firefox Nightly (80.0a1 Build ID - 20200712214458) installed on Windows 10 x64, Mac 10.15, and Ubuntu 18.04 x64.

Status: RESOLVED → VERIFIED
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.