1595954 - Record event as the user explicitly connects or disconnects sync

Status: RESOLVED FIXED

(Firefox :: Sync, enhancement)

Description

[Mark Hammond [:markh] [:mhammond]]

We'd like to record an event when the user explicitly connects or disconnects sync, and possibly even explicitly connects or disconnects FxA. This would prevent us trying to infer such actions from the FXA_CONFIGURED and WEAVE_CONFIGURED probes, and would help us track "accidental" disconnections (ie, those probes moving to false without an event would imply something going wrong).

Leif, any guidance/preference here? Would this be better done in the main ping or the fxa/sync ping?

Comment 1

[Leif Oines [:loines]]

Hmm. So for now we have three possible outcomes for disconnect (?)

1. disconnect FxA only, which would imply that the user did not have sync connected
2. disconnect FxA which then also disconnects sync,
3. disconnect sync only

One way to do do this would be to record a single "disconnect" event and then use the extra keys in the event to denote which services were disconnected, following above. e.g. { fxa: true, sync: true } for when the user disconnects both fxa and sync.

We could also emit one event per service disconnected, but that seems more cumbersome from analysis perspective.

We would then have a parallel event for account connection, but in that case we would never have the fxa: false, sync: true outcome, like we would for disconnection (but iiuc we would still have the true/false case, though hopefully rarely).

As for which ping to send these in, I think we at least we need to send it in the main ping. Many others besides the immediate team here will find it useful.

The trickier issue is whether we can also send it in the sync/fxa ping. In theory, that's where we would be able to do a better analysis of the "accidental vs intentional" disconnection issue. As per our discussion in slack, unless bug 1584356 is resolved then we won't have any user/device ids associated with pings containing disconnection events (because the user had... disconnected), which would make the event almost useless were it contained in the sync ping. If we CAN resolve that, then the question becomes - can we still send a sync ping after the user disconnects their account completely, indicating that they disconnected? My guess is that this would be an engineering headache, though maybe I'm wrong.

If we can resolve bug 1584356 then maybe the best we can do is send a disconnect event in the sync ping iff the user disconnects sync and not fxa.

To summarize:

1. Let's try to implement parallel disconnect/connect events, with two boolean "extra keys", fxa and sync, that indicate what service was affected (in the future we may also need to add more keys for other services, such as monitor)
2. Let's send those events at least in the main ping,
3. Let's explore the feasibility of also including it in the sync ping, perhaps we can only capture the case of sync disconnection and not account disconnection generally. This is likely only worth doing if we can get bug 1584356.

Comment 2

[Mark Hammond [:markh] [:mhammond]]

Attachment: tempdata-review.txt

Data review request, patch forthcoming. Try at https://treeherder.mozilla.org/#/jobs?repo=try&revision=627bfdfd58b7ba1d3d73f77d20d495c043bac5c2

Comment 3

[Mark Hammond [:markh] [:mhammond]]

Attachment: Bug 1595954 - record event as the user explicitly connects or disconnects sync or fxa. r?eoger,chutten

Comment 4

[Mark Hammond [:markh] [:mhammond]]

(In reply to Leif Oines [:loines] from comment #1)

3. Let's explore the feasibility of also including it in the sync ping, perhaps we can only capture the case of sync disconnection and not account disconnection generally. This is likely only worth doing if we can get bug 1584356.

FTR, this patch doesn't do (3), but it will be trivial to do so once we get bug 1584356.

Comment 5

[Chris H-C :chutten]

Comment on attachment 9110140 [details]
tempdata-review.txt

Load-balancing to Nicole.

Comment 6

[Nicole Shadowen]

Hi all, could you please indicate proposed list of measurements and category of data collection for each measurement, using descriptions found on Mozilla wiki here: https://wiki.mozilla.org/Firefox/Data_Collection?
Also, is there (or will there be) documentation on this data set described in public somewhere? And where would that be?

Comment 7

[Mark Hammond [:markh] [:mhammond]]

(In reply to Nicole Shadowen from comment #6)

Hi all, could you please indicate proposed list of measurements and category of data collection for each measurement, using descriptions found on Mozilla wiki here: https://wiki.mozilla.org/Firefox/Data_Collection?

This should be considered Category 2 “Interaction data”

Also, is there (or will there be) documentation on this data set described in public somewhere? And where would that be?

No. This is adding a single event to the exiting 'event' ping. No documentation will be done beyond what's already added to Events.yaml in the patch.

Comment 8

[Nicole Shadowen]

(In reply to Mark Hammond [:markh] from comment #7)
Thanks Mark. If you were to deploy this new connect/disconnect event, by what means do you intend to have it reported? Will you use the method Leif suggested with extra keys indicating which service caused the event?

Comment 9

[Mark Hammond [:markh] [:mhammond]]

(In reply to Nicole Shadowen from comment #8)

(In reply to Mark Hammond [:markh] from comment #7)
Thanks Mark. If you were to deploy this new connect/disconnect event, by what means do you intend to have it reported? Will you use the method Leif suggested with extra keys indicating which service caused the event?

Yes (although note the extra keys don't indicate which service caused the event, they indicate which services were impacted. For example, an extra key of "sync" means that sync was connected or disconnected, not that sync caused the connection or disconnection.

I documented the extra keys in Events.yaml - please let me know if you think that documentation is insufficient.

Comment 10

[Nicole Shadowen]

Comment on attachment 9110140 [details]
tempdata-review.txt

Thanks for your responses and clarification.

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way? 

     Yes. This collection is Telemetry so it will be documented in its definitions file [Events.yaml](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/Events.yaml) and the [Probe Dictionary](https://telemetry.mozilla.org/probe-dictionary/).


Is there a control mechanism that allows the user to turn the data collection on and off?

     Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.


If the request is for permanent data collection, is there someone who will monitor the data over time?

     Yes, :markh is responsible.


Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

     Category 2, Interaction. One event every time the user connects or disconnects from either FxA or Sync. 


Is the data collection request for default-on or default-off?

     Default on for all channels.


Does the instrumentation include the addition of any new identifiers?

     No. 


Is the data collection covered by the existing Firefox privacy notice? 

     Yes.


Does there need to be a check-in in the future to determine whether to renew the data? 

     No, this collection is permanent.


Does the data collection use a third-party collection tool? 

     No, telemetry only.

___
Result: datareview+

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7a14658ae9a4
record event as the user explicitly connects or disconnects sync or fxa. r=eoger

Comment 12

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/7a14658ae9a4