1355564 - Perform a data steward audit of the pings we're sending for Activity Stream

Status: RESOLVED FIXED

(Firefox for iOS :: Telemetry, enhancement, P1)

Description

[Stephan Leroux [:sleroux]]

Some of the data we're looking to send might contain PII data. Considering that we might need to run our A/B testing with real, production users, we should hold our data reporting to the same standards as our other release products.

Comment 1

[Stephan Leroux [:sleroux]]

Hey Barbara,

After some discussion on the work incorporating telemetry for the performance of Activity Stream [1], it became clear that we should be running the telemetry data we're sending past the data stewards before we launch Activity Stream to the general population. Desktop has been able to work around this by falling under TestPilot but for iOS we:

1. Don't ship AS as a TestPilot experiment so we have no additional privacy conditions.
2. There is a good chance we'll have to A/B test with release users so we'll want to make sure our data is clean before we ship.

[1] https://github.com/mozilla-mobile/firefox-ios/pull/2599

Comment 2

[Stephan Leroux [:sleroux]]

Sorry I forgot to actually asking my question in the previous comment:

What's the best way to get a start on this?

Comment 3

[Barbara Bermes [:barbara] - NI please!]

:) no problem, the way it works for Fennec is, the person who writes the telemetry code needs to do 2 things

- write telemetry documentation, you should check if we want to do this is the readthedocs code or in Github, I think for Focus we've just decided to do it in readthedocs (see here as an example, https://hg.mozilla.org/mozilla-central/diff/044cc30c767a/toolkit/components/telemetry/docs/core-ping.rst)
- get the data steward to review the docs and code (via a bug)


Now the question is who is the data steward for iOS, for Android it's Chenxia, but I'm sure bschmedberg would know (NI).

Let me know if you have any other questions.

Comment 4

[Benjamin Smedberg]

Technically I'm the data steward for all the Firefox browsers (with Chenxia as peer). Is this bug about Firefox for iOS, or a separate activity stream app for iOS?

If it's the former, I or Chenxia can review. If it's the latter, we should talk with ckarlof and marshall and decide who an appropriate data steward would be.

Comment 5

[Stephan Leroux [:sleroux]]

> Is this bug about Firefox for iOS, or a separate activity stream app for iOS?

This is for the New Tab Activity Stream experience inside Firefox for iOS. We're using the same system the desktop new tab screen is using for reporting data (Ping Centre) and using similar events with some sensitive information removed.

I'll setup some documentation for you or Chenxia to go over.

Comment 6

[Stephan Leroux [:sleroux]]

Attachment: Link to documentation

I've whipped up some documentation detailing the data we're sending in the first pass of the new tab panel.

Comment 7

[Chenxia Liu [:liuche]]

Took a look at these, thanks for the documentation, Steph!

A few notes of concern:

- I see that you're tracking opening of private browsing tabs. I don't think this is something we historically are allowed to track. Android masks private tab usage [1] and in the past, I've seen discussion that this violates "local privacy" and "stealth" in the Firefox privacy guidelines [2]
That being said, I think we're talking about pointing people towards Focus if they are using private tabs frequently, which is all right as long as we don't collect that information or send it as telemetry. The person I've heard talk about this most is :ehsan, though I don't know if there is a more specific place. I've heard discussion on emails cc-ing the fxprivacyandsecurity mailing list, but it's a closed list (probably a team list).

- "page" - Assuming this is one of the default home panels (and that iOS doesn't allow custom home panels), this is fine! But if (like on Android) there are custom panels, they should be hashed because these could contain PI.

- There are a few places where's it's ambiguous what is being collected (as a String, source, etc). If this is general information (e.g. "share button") that is fine, but no urls can be collected (e.g., bookmark url being deleted, or page that the user is on during an action).

Clearing the review flag, please flag me again, or feel free to ask any additional questions!

[1] https://dxr.mozilla.org/mozilla-central/source/mobile/android/base/java/org/mozilla/gecko/BrowserApp.java#3585
[2] https://wiki.mozilla.org/Private_Browsing

Comment 8

[Stephan Leroux [:sleroux]]

Attachment: Link to Github pull-request: https://github.com/mozilla-mobile/firefox-ios/pull/2697

Thanks for the quick review Chenxia! I've attached a PR removing the new private tab event and answered some questions below
----

> - I see that you're tracking opening of private browsing tabs. I don't think this is something we historically are allowed to track. Android masks private tab usage [1] and in the past

I've gone ahead and removed that. Shouldn't be critical to our measurements in the AS panel.

> - "page" - Assuming this is one of the default home panels (and that iOS doesn't allow custom home panels), this is fine!

We don't support custom home panels and this is hard coded to always be 'NEW_TAB' for now but if we do start including additional panels or custom ones we can revise.

> - There are a few places where's it's ambiguous what is being collected (as a String, source, etc)

We only send a predefined set of values here that. For example, 'source' is is one of 3 options: TOP_SITES, HIGHLIGHTS, HIGHLIGHT_INTRO.

Comment 9

[Chenxia Liu [:liuche]]

Comment on attachment 8864543 [details] [review]
Link to Github pull-request: https://github.com/mozilla-mobile/firefox-ios/pull/2697

r+ for this patch (on github) and data-review+ on the documentation (with the removal of the NEW_PRIVATE_TAB field name)

Comment 10

[Stephan Leroux [:sleroux]]

Thanks Chenxia!

master d70ca89725c83868fbc2fdcd8b150af0cce47159