Closed Bug 1270482 Opened 8 years ago Closed 8 years ago

Add Push metrics to main_summary dataset.

Categories

(Cloud Services Graveyard :: Metrics: Pipeline, defect, P2)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: rvitillo, Assigned: mreid)

References

Details

Attachments

(1 file)

Per Bug 1269466, we need to add POPUP_NOTIFICATION_STATS, PUSH_API_NOTIFICATION_RECEIVED & WEB_NOTIFICATION_SHOWN to compute DAU/MAU & ER for Push.
Flags: needinfo?(mreid)
Blocks: 1269466
Assignee: nobody → mreid
Points: --- → 1
Priority: -- → P2
Depends on: 1272334
I will take this as part of the next round of changes to MainSummary
Flags: needinfo?(mreid)
Histograms are defined as follows:
  "POPUP_NOTIFICATION_STATS": {
    "releaseChannelCollection": "opt-out",
    "alert_emails": ["firefox-dev@mozilla.org"],
    "bug_numbers": [1207089],
    "expires_in_version": "50",
    "kind": "enumerated",
    "keyed": true,
    "n_values": 40,
    "description": "(Bug 1207089) Usage of popup notifications, keyed by ID (0 = Offered, 1..4 = Action, 5 = Click outside, 6 = Leave page, 7 = Use 'X', 8 = Not now, 10 = Open submenu, 11 = Learn more. Add 20 if happened after reopen.)"
  },
  "PUSH_API_NOTIFICATION_RECEIVED": {
    "alert_emails": ["push@mozilla.com"],
    "expires_in_version": "55",
    "kind": "count",
    "description": "Push notification was received from server."
  },
  "WEB_NOTIFICATION_SHOWN": {
    "releaseChannelCollection": "opt-out",
    "alert_emails": ["firefox-dev@mozilla.org"],
    "bug_numbers": [1225336],
    "expires_in_version": "50",
    "kind": "count",
    "description": "Count of times a Notification was rendered (accounting for XUL DND). A system backend may put the notification directly into the tray if its own DND is on."
  },

The two "count" type histograms should be stored as numeric values, but what about the enumerated one? How would you like that represented in the MainSummary records?  Previously (in bug 1261829) we added a separate field for each enum value - do you want to do the same here?
Flags: needinfo?(rvitillo)
Also, PUSH_API_NOTIFICATION_RECEIVED doesn't appear to be opt-out, so we won't be able to use it to compute MAU/DAU on the release channel.
(In reply to Mark Reid [:mreid] from comment #2)
> How would you like [POPUP_NOTIFICATION_STATS] represented in the
> MainSummary records?  Previously (in bug 1261829) we added a separate field
> for each enum value - do you want to do the same here?

It's also keyed, so we'll have to do something different - it appears there's a key for "(all)", we could just use that key or we could store it as a map.
(In reply to Mark Reid [:mreid] from comment #5)
> Brian, do you think it would suffice to include the "(all)" key for
> computing MAU/DAU based on the POPUP_NOTIFICATION_STATS histogram? Or do you
> need more fine-grained info for each possible key (other values I see
> locally include "geolocation", "password" and "web-notifications")?

I believe the "web-notifications" is the key we can use for push.  It would be useful to understand the baseline "permission requested" so we could compare that against the positive "1" and negative "2" as well as the neutral [x] or clicked away.  Is that is better represented as a funnel?
Flags: needinfo?(clarkbw)
Brian, do you think it would suffice to include the "(all)" key for computing MAU/DAU based on the POPUP_NOTIFICATION_STATS histogram? Or do you need more fine-grained info for each possible key (other values I see locally include "geolocation", "password" and "web-notifications")?
Flags: needinfo?(clarkbw)
(In reply to Bryan Clark (DevTools PM) [@clarkbw] from comment #6)
> (In reply to Mark Reid [:mreid] from comment #5)
> > Brian, do you think it would suffice to include the "(all)" key for
> > computing MAU/DAU based on the POPUP_NOTIFICATION_STATS histogram? Or do you
> > need more fine-grained info for each possible key (other values I see
> > locally include "geolocation", "password" and "web-notifications")?
> 
> I believe the "web-notifications" is the key we can use for push.  It would
> be useful to understand the baseline "permission requested" so we could
> compare that against the positive "1" and negative "2" as well as the
> neutral [x] or clicked away.  Is that is better represented as a funnel?

That could be done without a funnel. 

Mark, can we use the same layout that the longitudinal dataset is using? It might make sense to store all keys since the histogram seems to be generally useful. You might want to convert each histogram in POPUP_NOTIFICATION_STATS to a map though. 

We should probably use the same strategy for all enumerated histograms in the main_summary dataset as it can quickly become confusing if a histogram is treated one way and another of the same type in a completely different one.
Flags: needinfo?(rvitillo) → needinfo?(mreid)
Blocks: 1275889
It does seem like a map will be needed here, as I can definitely imagine the "(all)" key being of interst as well as the "web-notifications" one. I propose storing this data like:

{ histogram_key -> Row(offered, action_1, action_2, ..., reopen_offered, reopen_action_1, reopen_action_2, ..., reopen_learn_more) }

So we end up with a map of structs containing named fields for each enum value. Each field would be a numeric field, similar to how we've stored other count-type histograms.

For a non-keyed enum histogram, we should do the same, but with a single field containing a similar struct (and refactor the existing loop_activity_* fields).

Sound good?
Flags: needinfo?(mreid) → needinfo?(rvitillo)
Sounds good to me.
Flags: needinfo?(rvitillo)
After chatting with rweiss I wanted to clarify a little bit on how we're hoping to compare engagement of a push user vs. a non-push user.  This attachment shows an example outcome were we are expecting to see the engagement ratio of a push user compared to firefox users excluding those we defined as push users.  And hopefully our engagement ratio is higher, of course that may only be correlation for now.
Added in https://github.com/mozilla/telemetry-batch-view/pull/79
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Product: Cloud Services → Cloud Services Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: