1514812 - Re-implement push telemetry for subscriptions, declines, and subscription requests

Status: NEW

(Core :: DOM: Push Subscriptions, enhancement, P3)

Description

[Leif Oines [:loines]]

According to the probe dictionary, we used to have several telemetry measurements in beta and nightly that would be extremely valuable to re-implement from a product perspective:

- PUSH_API_PERMISSION_DENIED: User explicitly denied Push Notifications permission
- PUSH_API_PERMISSION_GRANTED: User explicitly granted Push Notifications permission
- PUSH_API_PERMISSION_REQUESTED: Count of number of times the PermissionManager explicitly prompted user for Push Notifications permission

w/r/t push, we currently only record:
- PUSH_API_NOTIFY: Number of push messages that were successfully decrypted and delivered to a ServiceWorker.

I'd like to:

1. get an estimate of the work required to re-implement the dead probes listed above 
2. clarify that counts in `PUSH_API_NOTIFY` correspond to the number of notifications actually seen by the user, and if not, what the cases are where counts are added to the probe but the notifications are not displayed

We currently have no idea about the value Firefox users see in push notifications - how often they consent/don't consent to receive them, how often they are asked, how often they interact with the doorhanger, etc.  

Possibly related to https://bugzilla.mozilla.org/show_bug.cgi?id=1203275

Comment 1

[Lina Butler [:lina]]

The PUSH_API_PERMISSION_{DENIED, GRANTED, REQUESTED} probes were removed because we merged the push and desktop notifications permissions, so they're covered by the WEB_NOTIFICATION_PERMISSIONS probe. This probe was also removed recently, in bug 1429016, but easy to add back.

PUSH_API_NOTIFY roughly corresponds to the number of notifications seen by the user. There are some corner cases (like silent pushes and if the user has "do not disturb" enabled) where this will under- or over-count pushes, but I don't think it's worth worrying about them. We'll only allow a limited number of silent pushes (bug 1375683, comment 17) before we remove the site's push subscription, and, for system-specific "do not disturb", like on Windows and macOS, we can't know if it's enabled or not.

A related probe also removed in bug 1429016 is WEB_NOTIFICATION_SHOWN, which is the total number of notifications we show, push and local. It's interesting to compare with PUSH_API_NOTIFY, but also counts sites that show local notifications without going through our push server at all.

Both of these fed into re:dash.

TL;DR: We can add back WEB_NOTIFICATION_PERMISSIONS and WEB_NOTIFICATION_SHOWN without much work, if there's value in having them.

Comment 3

[Johann Hofmann [:johannh]]

User interaction with the notification prompts (including notification permissions) is measured in this central probe: https://mzl.la/2GKbzcC

Just as an FYI, we're planning to run experiments on restricting notification permission prompts based on this probe to get acceptance rate up, probably starting in Nightly 68.

If you were looking at this, too, we should probably sync up.

We currently have no idea about the value Firefox users see in push notifications - how often they consent/don't consent to receive them, how often they are asked, how often they interact with the doorhanger, etc.

Based on the above we've had a good idea for quite some time and the harsh reality is that not even 3% of users allow notification permission requests.

Comment 4

[Leif Oines [:loines]]

Hey thanks for resurrecting this bug :lina, and for your comments :johannh.

Regarding POPUP_NOTIFICATION_STATS, I had no idea about this probe, so thanks for that info. A couple of questions:

(1) I agree that 3% seems quite low. since this probe is prerelease only, I wonder if there's a chance its slightly, but potentially significantly, higher for release? It doesn't seem that we'd have any way of knowing for sure, but given the general differences between these populations...

(2) What other types of notifications are covered by this probe? Website notifications, location permissions, password saving/updating, forms, etc? If its a pretty wide mixture then I think I'd prefer to re-instrument some of what :lina mentions, if we are interested in analyzing this data further.

FYI I think this got de-prioritized a bit from the product end. :adavis, can you read Lina's comment above and weigh in on whether it would be worth her/someone elses's time to re-implement WEB_NOTIFICATION_PERMISSIONS and WEB_NOTIFICATION_SHOWN?

Comment 5

[Johann Hofmann [:johannh]]

(In reply to Leif Oines [:loines] from comment #4)

Hey thanks for resurrecting this bug :lina, and for your comments :johannh.

Regarding POPUP_NOTIFICATION_STATS, I had no idea about this probe, so thanks for that info. A couple of questions:

(1) I agree that 3% seems quite low. since this probe is prerelease only, I wonder if there's a chance its slightly, but potentially significantly, higher for release? It doesn't seem that we'd have any way of knowing for sure, but given the general differences between these populations...

Certainly, though from anecdotal experience I'd say people don't like these prompts. It's important to note that because notification ALLOW permissions are permanent while DENY permissions are temporary this data is heavily under-representing useful sites like Facebook, Slack, Telegram etc. where the user allows once and never gets a permission request again.

(2) What other types of notifications are covered by this probe? Website notifications, location permissions, password saving/updating, forms, etc? If its a pretty wide mixture then I think I'd prefer to re-instrument some of what :lina mentions, if we are interested in analyzing this data further.

This probe measures all permission prompts separately, you can view stats for each permission type separately (e.g. for notifications or geolocation), so I don't see the point in having another probe.

Comment 6

[Alex Davis [:adavis]]

Leif, we will need to re-enable this in the medium term but with Fenix around the corner, let's hold off for now from distracting the team.