Closed Bug 1347266 Opened 7 years ago Closed 6 years ago

Add telemetry to Focus for understanding user actions

Categories

(Firefox for Android Graveyard :: Metrics, enhancement, P1)

All
Android
enhancement

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: frank, Unassigned)

References

Details

User Story

Purpose: Understand how users are interacting with and using Focus.

We need data on:
 - Settings
 - UI Event Telemetry

This data is combined with some environmental data - os, version, app, etc.

DATA REVIEW: https://docs.google.com/document/d/1QcOfkYGi06KAq-gmWe5mG43Ajwr_GmGu4bIIHkUG-jY/edit#heading=h.musg4vsr2wsk

PING OVERVIEW: https://docs.google.com/document/d/165jaEz45XVaiTWSjzaGr_coEEq5NVslH6VjxIWuW6Jw/edit
      No description provided.
Hi Rebecca and Benjamin, we have an urgent need for data-review. Android Focus is slated for an April 6 release.
Severity: normal → blocker
User Story: (updated)
Flags: needinfo?(rweiss)
Flags: needinfo?(benjamin)
Priority: P2 → P1
I will do a *preliminary* data review now. The final data review happens on the patch which checks in final data documentation. I guess for this project we'd check the data docs into the Focus repo (on github?). I'm looking primarily at https://docs.google.com/document/d/165jaEz45XVaiTWSjzaGr_coEEq5NVslH6VjxIWuW6Jw/edit

Basic docs: this doesn't indicate *when* this ping is sent. That's important.

clientId: The doc says "this will use clientID", but I wouldn't have figured that Firefox Focus *had* a clientID until now. Is this a random ID generated and stored on first install? Does this get cleared and recreated if the user uninstalls/reinstalls Firefox? What is the risk that other apps on the system would be able to read this ID?

tz: why is this a string, and what values do you expect? If we have an ISO timestamp that includes the time zone automatically.

What UI exists for users to turn off this data collection? That should be an option in app preferences.

(minor worry) Is it a technical requirement settings are recorded as strings? From the perspective of validating privacy characteristics, allowing as few strings as possible means there are fewer fields that could have identifying data, and most of these are booleans.

For search engine setting (and change event), is this a pre-shipped set of search engines or can users install their own arbitrarily? Does this record both builtin and user-installed values, or are user-installed values recorded as "other" like we do on desktop?

Event questions:
What is the resolution of the timestamps here? Precise timestamps are in general a privacy risk because they can allow easy correlation and identification. We may need to explore ways to reduce the granularity, such as *ordering* the events but not timestamping them.

I don't understand the rest of the documentation. What does "Category: action" mean in practice? What is "type_url" or "type_query"? what does "Object" mean?

Start/open app: is this opening another app from within Focus? Under what conditions does that happen? If this is arbitrary apps opened as a result of web browsing, I have browsing and identification concerns with this.

browse/search: I can't tell what this is collecting, so it's really hard to understand the privacy risks.
Flags: needinfo?(benjamin)
Benjamin, thanks so much. I'll address what I can here.

> I will do a *preliminary* data review now. The final data review happens on
> the patch which checks in final data documentation. I guess for this project
> we'd check the data docs into the Focus repo (on github?). I'm looking
> primarily at
> https://docs.google.com/document/d/
> 165jaEz45XVaiTWSjzaGr_coEEq5NVslH6VjxIWuW6Jw/edit
> 
> Basic docs: this doesn't indicate *when* this ping is sent. That's important.

We have yet *another* doc that outlined some of our thoughts about when, but it's more of a notes than a specification. Basically, when a user leaves the app - whether that's closing, exiting, popping out, sharing, sending to background, etc.

I will talk to Sebastian and we will clear *exactly* when it will be sent.

> 
> clientId: The doc says "this will use clientID", but I wouldn't have figured
> that Firefox Focus *had* a clientID until now. Is this a random ID generated
> and stored on first install? Does this get cleared and recreated if the user
> uninstalls/reinstalls Firefox? What is the risk that other apps on the
> system would be able to read this ID?

I'm not sure about this. We need clientid so that we can follow usage, it will presumably be a random ID from first install. It would get recreated in that case. I'm not sure about the risk from other apps reading it.

> 
> tz: why is this a string, and what values do you expect? If we have an ISO
> timestamp that includes the time zone automatically.

We mirrored the core ping for this.

> 
> What UI exists for users to turn off this data collection? That should be an
> option in app preferences.

It is.

> (minor worry) Is it a technical requirement settings are recorded as
> strings? From the perspective of validating privacy characteristics,
> allowing as few strings as possible means there are fewer fields that could
> have identifying data, and most of these are booleans.

The only one that is not is search provider. We could certainly send the rest as bools.

> 
> For search engine setting (and change event), is this a pre-shipped set of
> search engines or can users install their own arbitrarily? Does this record
> both builtin and user-installed values, or are user-installed values
> recorded as "other" like we do on desktop?

There is a pre-set list of search providers.

> 
> Event questions:
> What is the resolution of the timestamps here? Precise timestamps are in
> general a privacy risk because they can allow easy correlation and
> identification. We may need to explore ways to reduce the granularity, such
> as *ordering* the events but not timestamping them.

Resolution is to second - we're following the event telemetry guidelines laid out already. Rweiss said this is a discussion you would be having in the policy meeting on Monday. While we would prefer to-second precision so we can best understand how users use the app, less granularity would be fine. (e.g. how long to search? browse?)

> 
> I don't understand the rest of the documentation. What does "Category:
> action" mean in practice? What is "type_url" or "type_query"? what does
> "Object" mean?

These are from the event telemetry specs I linked to above (in the doc). Basically there is [Timestamp, Category, Action, Object]. Values and Extras are optional.

> 
> Start/open app: is this opening another app from within Focus? Under what
> conditions does that happen? If this is arbitrary apps opened as a result of
> web browsing, I have browsing and identification concerns with this.

Sorry, I should have been more clear - this is opening Focus itself.

> 
> browse/search: I can't tell what this is collecting, so it's really hard to
> understand the privacy risks.

We aren't collecting any info about what the browsing/searches are, only that they happened, and in what part of the user journey.
I don't think this timestamp introduces additional risk. That would be the case if that data is tied to certain other data that is sensitive to our users, but from what I can seen it doesn't meaningfully change the identifiability/sensitivity of the data if it is only associated with the data in the ping overview document.  

> 
> Resolution is to second - we're following the event telemetry guidelines
> laid out already. Rweiss said this is a discussion you would be having in
> the policy meeting on Monday. While we would prefer to-second precision so
> we can best understand how users use the app, less granularity would be
> fine. (e.g. how long to search? browse?)
> 
> > 
> > I don't understand the rest of the documentation. What does "Category:
> > action" mean in practice? What is "type_url" or "type_query"? what does
> > "Object" mean?
>
r+, data review details are here: https://docs.google.com/document/d/1jeAYzvC5T2pF6KlYp0jYFqFgWoPodbAThmqUfrKAM6M/edit#.
Flags: needinfo?(rweiss)
See Also: → 1420949
Frank, can this be closed?
Flags: needinfo?(fbertsch)
Yes, definitely!
Status: NEW → RESOLVED
Closed: 6 years ago
Flags: needinfo?(fbertsch)
Resolution: --- → FIXED
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.