Bug 1571584 Comment 6 Edit History

Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.

Late to the party, but here's my 2c. 

### I'll start with main telemetry. *ideally*:
1. We should ensure the current histogram FXA_CONFIGURED continues to reflect whether a user is authenticated with FxA in the browser, regardless if they have any services connected. 
2. We should consider adding a couple probes to main telemetry. The first should be something like a keyed histogram of services a user is currently authenticated with e.g. FXA_SERVICES_AUTHENTICATED: { sync: TRUE, monitor: FALSE, ... }. I get that this might not be possible, see below.
3. The second should be an event that fires when a user disconnects or connects a service. E.g. event action = connect/disconnect, event_object = {service_name}. 
4. If we can't have 2 or 3 we should fix SYNC_CONFIGURED so it accurately reports TRUE or FALSE for all users. The fill rate (rate that it is non-null) on that probe makes it next to useless currently. Maybe we should just fix this anyway - I know Mark tried at one point but I think he was blocked by the telemetry folks for some arcane reason. 

Why would we want 2 & 3? Because there have been a lot of questions, *from the desktop side* about what types of users sign into these services. E.g. are users who save a lot of passwords in the browser more likely to sign into monitor? This is *part* of what ecosystem telemetry is supposed to get us, but if we had these probes in main telemetry (without logging the FxA uid anywhere) then answering these questions starts to become possible (and from an analysis perspective, perhaps even easier). People will progressively begin to ask more and more questions of the nature "is measure X associated with more users signing into Y service?", I am quite sure of this. Finally, I suggest (3) because events have timestamps that allow us to know, relative to other events, when a user signs in (e.g. did they sign in before or after a doorhanger promoting monitor?).

However, based on what Ryan says above, is it true that, even after decoupling, if a user signs into send then nothing will change in the browser? If that's true, then maybe we can't do 2 or 3, and we will have to wait until ecosystem telemetry to answer the previous questions. But to the extent we can do 2 and/or 3, i think we should.

### On FxA telemetry:
We want to know for each user, separately, if they are connected to the browser and/or sync. We do this currently with the service property, which for all non-sync services is a mapping from oauth client id. In the ideal world I think we would continue to have `sync` as a value for service, reflecting only users who have sync configured. Perhaps then we add an additional `firefox` service. All users who have `sync` as a service should also have `firefox`, but not vice-versa. I'm hazy on the technical hurdles here, but this would be the ideal state.

Finally, we should continue to track service for all other reliers via FxA server-side measurements in the same way we do today to avoid any data discontinuities.  

### On sync telemetry:
I would be ok with sending the sync ping for all FxA users, though if we instrument the main telemetry probes above then the only real reason to do this would be to track send tab telemetry for non-sync users (as noted above), which I believe we definitely want to do in any case. 

### On ecosystem telemetry
We have an ambitious goal for lockwise mobile & desktop to be the first consumer of this by EOY. So it is very "real", but the first iterations of it will not allow us to answer questions that involve correlating services used to desktop measurements (similar to those posed above). Matt is the one taking the charge on that now. For the immediate future, my gut is telling me that if have the additional desktop measurements, then we don't have to worry too much about this ATM. Note that having those measurements will allow us to answer some, but not ALL of the questions that ecosystem telemetry should in theory allow us to answer. It won't allow us to answer things about cross-device usage, for example. 

I typed all that out very quickly - feel free to ask for clarification.
Late to the party, but here's my 2c. 

### I'll start with main telemetry. *ideally*:
1. We should ensure the current histogram FXA_CONFIGURED continues to reflect whether a user is authenticated with FxA in the browser, regardless if they have any services connected. 
2. We should consider adding a couple probes to main telemetry. The first should be something like a keyed histogram of services a user is currently authenticated with e.g. FXA_SERVICES_AUTHENTICATED: { sync: TRUE, monitor: FALSE, ... }. I get that this might not be possible, see below.
3. The second should be an event that fires when a user disconnects or connects a service. E.g. event action = connect/disconnect, event_object = {service_name}. 
4. If we can't have 2 or 3 we should fix SYNC_CONFIGURED so it accurately reports TRUE or FALSE for all users. The fill rate (rate that it is non-null) on that probe makes it next to useless currently. Maybe we should just fix this anyway - I know Mark tried at one point but I think he was blocked by the telemetry folks for some arcane reason. 

Why would we want 2 & 3? Because there have been a lot of questions, *from the desktop side* about what types of users sign into these services. E.g. are users who save a lot of passwords in the browser more likely to sign into monitor? This is *part* of what ecosystem telemetry is supposed to get us, but if we had these probes in main telemetry (without logging the FxA uid anywhere) then answering these questions starts to become possible (and from an analysis perspective, perhaps even easier). People will progressively begin to ask more and more questions of the nature "is measure X associated with more users signing into Y service?", I am quite sure of this. Finally, I suggest (3) because events have timestamps that allow us to know, relative to other events, when a user signs in (e.g. did they sign in before or after a doorhanger promoting monitor?).

However, based on what Ryan says above, is it true that, even after decoupling, if a user signs into send then nothing will change in the browser? If that's true, then maybe we can't do 2 or 3, and we will have to wait until ecosystem telemetry to answer the previous questions. But to the extent we can do 2 and/or 3, i think we should.

### On FxA telemetry:
We want to know for each user, separately, if they are connected to the browser and/or sync. We do this currently with the service property, which for all non-sync services is a mapping from oauth client id. In the ideal world I think we would continue to have `sync` as a value for service, reflecting only users who have sync configured. Perhaps then we add an additional `firefox` service. All users who have `sync` as a service should also have `firefox`, but not vice-versa. I'm hazy on the technical hurdles here, but this would be the ideal state.

Finally, we should continue to track service for all other reliers via FxA server-side measurements in the same way we do today to avoid any data discontinuities.  

### On sync telemetry:
I would be ok with sending the sync ping for all FxA users, though if we instrument the main telemetry probes above then the only real reason to do this would be to track send tab telemetry for non-sync users (as noted above), which I believe we definitely want to do in any case. Edit: elaborating on this point a bit more, we want accurate telemetry on send tab success and failure rates, including:

1. a tab sent event with the device_id of initiating device, timestamp, random id identifying the sending event, list of target device_ids, and number of tabs sent
2. a tab received event logged by each receiving device including its device_id, the same random id from 1, number of tabs received, and if possible the device id of the sending device.

We in theory have that information in today's send tab telemetry (except maybe number of tabs sent?). If it makes sense to continue using the sync ping to log these events, even for users who are now send-tab only (no formal sync), then we should continue to do that. However, depending on the technicals, we could also move this telemetry to "main" event telemetry, and somehow obfuscate the device_ids with a client-side secret so that they cannot be joined on FxA telemetry data. 

### On ecosystem telemetry
We have an ambitious goal for lockwise mobile & desktop to be the first consumer of this by EOY. So it is very "real", but the first iterations of it will not allow us to answer questions that involve correlating services used to desktop measurements (similar to those posed above). Matt is the one taking the charge on that now. For the immediate future, my gut is telling me that if have the additional desktop measurements, then we don't have to worry too much about this ATM. Note that having those measurements will allow us to answer some, but not ALL of the questions that ecosystem telemetry should in theory allow us to answer. It won't allow us to answer things about cross-device usage, for example. 

I typed all that out very quickly - feel free to ask for clarification.

Back to Bug 1571584 Comment 6