837934 - Moar social telemetry

Status: RESOLVED DUPLICATE of bug 862623

(Firefox Graveyard :: SocialAPI, defect)

Description

[Mark Hammond [:markh] [:mhammond]]

In the social meeting there was a discussion about the need for more telemetry, particularly to measure engagement.  Some ideas:

* How often are the jewel buttons clicked on?
* How many chat windows are open?
* How often does a chat window go unnoticed by the user?

Not sure if we should have one bug per entry, but capturing the general requirement here before it gets thrown out of the short-term-memory cache...

Comment 1

[John Jensen]

...ideally these are on a per-provider, or per-whitelisted-provider, basis.

Comment 2

[Mark Hammond [:markh] [:mhammond]]

(In reply to John Jensen from comment #1)
> ...ideally these are on a per-provider, or per-whitelisted-provider, basis.

Are there any privacy issues with this?  Or should we CC the privacy team for them to decide?

Comment 3

[John Jensen]

> more telemetry

It seems like FHR would be a better place for this, no?

> Are there any privacy issues with this?  Or should we CC the privacy team
> for them to decide?

We will definitely need the privacy team to get involved. I don't have a particular solution in mind, but am just articulating the view that the product marketing team would like some kind of indication of usage level of the SocialAPI on a per-whitelisted-provider basis. Aggregated would be fine. The team would like this in order to be better informed when negotiating agreements with whitelisted providers.

Comment 4

[Mark Hammond [:markh] [:mhammond]]

Gregory,
  It would be great if you can offer some high-level feedback on this.  I was thinking that I could add a new 'SocialProvider' object to http://mxr.mozilla.org/mozilla-central/source/services/healthreport/providers.jsm, and this could start with a few methods specific to the above requirements such as "recordToolbarClick(provider, toolbarname)", "recordNewChat(provider, userInitiated)", "recordChatInteraction(...)" etc.  The social code would then obviously be instrumented to call these methods.

Does that sound like a sane and reasonable approach?

Comment 5

[Gregory Szorc [:gps]]

(In reply to Mark Hammond (:markh) from comment #4)
> Gregory,
>   It would be great if you can offer some high-level feedback on this.  I
> was thinking that I could add a new 'SocialProvider' object to
> http://mxr.mozilla.org/mozilla-central/source/services/healthreport/
> providers.jsm, and this could start with a few methods specific to the above
> requirements such as "recordToolbarClick(provider, toolbarname)",
> "recordNewChat(provider, userInitiated)", "recordChatInteraction(...)" etc. 
> The social code would then obviously be instrumented to call these methods.
> 
> Does that sound like a sane and reasonable approach?

That is roughly how things would work on a technical level, yes. One slight change is the end goal is for the different FHR providers to live outside /services/healthreport and instead live next to the code they instrument and under a Module's purview. (Providers are registered via Category Manager entries so they can live in any Cu.import'able file.) We haven't done this yet because it hasn't been a priority.

But even before you start thinking about code, you need sign-off on the data being collected. This process is still being ironed out. mconnor can provide more info. Essentially, you are going to need to propose all the data you want collected, how it will be used, and then you'll need sign-off from Privacy, etc.

Comment 6

[Mike Connor [:mconnor]]

I was sick most of last week, so the framework I'm going to draft is still all in my head, but the tl;dr version is that anything we collect in FHR on Beta and Release channels must be tied back to the user-facing health report in a way that benefits the user.  I'm not seeing that here, and that's a problem we need to sort out.

Comment 7

[Mark Hammond [:markh] [:mhammond]]

(In reply to Mike Connor [:mconnor] from comment #6)
> anything we collect in FHR on
> Beta and Release channels must be tied back to the user-facing health report
> in a way that benefits the user.  I'm not seeing that here, and that's a
> problem we need to sort out.

I don't quite see what the distinction is between the stuff we want to collect versus, say, the search provider metrics.

Do you have any advice for how we should collect such stats?  Telemetry sounds like it remains an option, but I'm not that clear on how it would help us collect per-provider stats (in the same way the FHR search provider stats are kept per-provider).

Comment 8

[Shane Caraveo (:mixedpuppy)]

Addition of social addons to the FHR addon provider is done in bug 851653.  Implementation of an FHR provider that is more specific to social services is in bug 862623.  Additional telemetry hooks are in bug 860549.