Remove expiring Push telemetry probes; extend others

RESOLVED FIXED in Firefox 55

Status

()

Core
DOM: Push Notifications
RESOLVED FIXED
a year ago
a year ago

People

(Reporter: kitcambridge, Assigned: kitcambridge)

Tracking

unspecified
mozilla55
Points:
---

Firefox Tracking Flags

(firefox55 fixed)

Details

MozReview Requests

()

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(2 attachments)

Many of our probes are expiring in 55, and it's a good time to clean house and remove ones that aren't valuable. From the push@ thread, I think these can be axed:

* PUSH_API_NOTIFICATION_RECEIVED: Number of delivered notifications, including those that weren't dispatched to a service worker. We can already glean this from server metrics, with the advantage that they're per-user (not per-session), and include counts for all channels.
* PUSH_API_NOTIFICATION_RECEIVED_BUT_DID_NOT_NOTIFY, PUSH_API_NOTIFY_REGISTRATION_LOST: Captured by server "ack" metrics.
* PUSH_API_PERMISSION_DENIED, PUSH_API_PERMISSION_GRANTED, PUSH_API_PERMISSION_REQUESTED: We merged the notifications and push permissions some time ago, and `WEB_NOTIFICATION_PERMISSIONS` is an opt-out probe that covers these.
* PUSH_API_QUOTA_EXPIRATION_TIME, PUSH_API_QUOTA_RESET_TO: These aren't actionable, we haven't created dashboards for them, and the plan is to replace our quota mechanism with the Budget API.
* PUSH_API_SUBSCRIBE_ATTEMPT, PUSH_API_SUBSCRIBE_FAILED, PUSH_API_SUBSCRIBE_HTTP2_TIME, PUSH_API_SUBSCRIBE_SUCCEEDED, PUSH_API_SUBSCRIBE_WS_TIME, PUSH_API_UNSUBSCRIBE_ATTEMPT, PUSH_API_UNSUBSCRIBE_FAILED, PUSH_API_UNSUBSCRIBE_SUCCEEDED: Subscribe and unsubscribe failures are likely caused by network issues (which aren't actionable), or server errors (for which we already have metrics). The server also records latencies.
* PUSH_API_USED: We have better ways to determine the value of Push: server metrics, plus our opt-out probes.
* WEB_NOTIFICATION_REQUEST_PERMISSION_CALLBACK: Theoretically useful for bug 1241543, but I don't think anyone is actively monitoring this. We can always add it back later, unless there's a strong case for keeping it now.

These are valuable, and we should keep them:

* PUSH_API_NOTIFY: The number of times we fire the `push` event. Feeds into re:dash.
* WEB_NOTIFICATION_CLICKED: See below.
* WEB_NOTIFICATION_PERMISSIONS: MattN has a dashboard for this at  https://sql.telemetry.mozilla.org/queries/2121#3885.

Quoting Bryan, the product owner for push:

> Both of these are useful to understand the value of push.  The former
> is supposed to answer the question "Are messages useful?" though it
> requires WEB_NOTIFICATION_SHOWN in order to have a click vs.
> impression style metric.  The latter is supposed to answer the
> question around the rate of deny to accept and how we can streamline
> the permission interface to account for this.  As we make changes we
> need to see this rate alter.
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Chenxia, flagging you for both data-r? and r?, if you have cycles. Please take a look at your convenience, or let me know if I should redirect to another data peer. Thank you!
Comment on attachment 8844310 [details]
Bug 1345001 - Remove expiring Push telemetry probes.

https://reviewboard.mozilla.org/r/117804/#review121230

Sorry for the delay! Thanks for the very detailed documentation of this removal of probes, especially listing what probes have replaced them, or the reasons why they are no longer needed. (The oranges on try look intermittent and unrelated.)
Attachment #8844310 - Flags: review?(liuche) → review+
Comment on attachment 8844311 [details]
Bug 1345001 - Bump expiration for valuable Push telemetry probes.

https://reviewboard.mozilla.org/r/117806/#review121232

r+ and data-review+, bumping existing probes and included detailed reasons for why.

Thanks for being patient with me, sorry this took so long!
Attachment #8844311 - Flags: review?(liuche) → review+
No problem; this wasn't urgent. Thanks for the review!

Comment 7

a year ago
We're sorry, Autoland could not rebase your commits for you automatically. Please manually rebase your commits and try again.

hg error in cmd: hg rebase -s 9082e04afbe7 -d bb03dd134cdc: rebasing 381524:9082e04afbe7 "Bug 1345001 - Remove expiring Push telemetry probes. r=liuche"
merging toolkit/components/telemetry/Histograms.json
merging toolkit/components/telemetry/histogram-whitelists.json
warning: conflicts while merging toolkit/components/telemetry/Histograms.json! (edit, then use 'hg resolve --mark')
unresolved conflicts (see hg resolve, then hg rebase --continue)

Comment 8

a year ago
Pushed by kcambridge@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/6bb2a2950b27
Remove expiring Push telemetry probes. r=liuche
https://hg.mozilla.org/integration/mozilla-inbound/rev/86c6e784ed25
Bump expiration for valuable Push telemetry probes. r=liuche

Comment 9

a year ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/6bb2a2950b27
https://hg.mozilla.org/mozilla-central/rev/86c6e784ed25
Status: ASSIGNED → RESOLVED
Last Resolved: a year ago
status-firefox55: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
You need to log in before you can comment on or make changes to this bug.