1570250 - telemetry data-review for secure-proxy

Status: RESOLVED FIXED

(Core :: General, task, P1)

Description

[Andrea Marchesini [:baku]]

Secure-proxy, also known as Firefox Private Network (FPN), is a new mozilla project in collaboration with cloudflare. The project is not public yet.

We want to collect telemetry data about user interaction, proxy error handling and so on. This bug is about the data-review process.

Comment 1

[Andrea Marchesini [:baku]]

Attachment: data-review form

Comment 2

[Chris H-C :chutten|PTO (back July 13)]

Comment on attachment 9081902 [details]
data-review form

We had a chat about this on Slack. To summarize, the Data Review needs to be visible/accessible to everyone who will be affected by this collection. So this moco-conf bug can't have a Data Review for the whole population.

:baku did say we'll be starting with an internal (Mozilla) trial, and a moco-conf bug can certainly be used for that. We'll need a Data Collection Review that highlights we're only collecting from that population, and then we can proceed here.

For the future population expansions I think :baku is penning an email to Trust (:agray, specifically) about how we might handle the timing and whatnot.

(:baku to factcheck me in case I have any of this wrong)

Comment 3

[Liz Henry (:lizzard)]

Does this need to happen before we ship Skyline in 70?

Comment 4

[Andrea Marchesini [:baku]]

Yes, but we don't want to start the process yet because secure-proxy is not public yet.
How much time do we have? Note that this review process doesn't require code in central.

Comment 5

[Liz Henry (:lizzard)]

Before Aug 30 would be good (i.e. just before 70 moves to beta)

Comment 6

[Liz Henry (:lizzard)]

Changing to corp confidential temporarily (till after Sept 10) on Tony and elan's request.

Comment 7

[Nhi Nguyen (:nhi)]

moving the bug out of networking

Comment 8

[Chris H-C :chutten|PTO (back July 13)]

Comment on attachment 9081902 [details]
data-review form

Clearing data-review? until the documentation is prepared. (work tracked on [this issue](https://github.com/mozilla/secure-proxy/issues/517))

As :baku and I discussed on Slack: Dynamic events as implemented in the addon will require their own documentation (since they're not in Events.yaml, they can't generate their own). A good example to follow is the Lockwise addon's [`metrics.md`](https://github.com/mozilla-lockwise/lockwise-addon/blob/master/docs/metrics.md): it is public, it explains why and what and how data is collected, and is specific.

Comment 9

[Tony Cinotto [:tcinotto] (UTC-8)]

(In reply to Chris H-C :chutten from comment #8)

Comment on attachment 9081902 [details]
data-review form

Clearing data-review? until the documentation is prepared. (work tracked on
this issue)

As :baku and I discussed on Slack: Dynamic events as implemented in the
addon will require their own documentation (since they're not in
Events.yaml, they can't generate their own). A good example to follow is the
Lockwise addon's
[metrics.md](https://github.com/mozilla-lockwise/lockwise-addon/blob/
master/docs/metrics.md): it is public, it explains why and what and how data
is collected, and is specific.

This is implemented as to spec now right? I would like to confirm all the data/telemetry reviews we need are done and we are clear for Beta.

Comment 10

[Andrea Marchesini [:baku]]

Yes, all is done. We need to write metrics.md before Sep 10th.

Comment 11

[Tony Cinotto [:tcinotto] (UTC-8)]

Hi Chris, can we mark this bug resolved?

Comment 12

[Chris H-C :chutten|PTO (back July 13)]

I don't think so. The data review hasn't yet received data-review+, and to receive that it needs complete and public (at the time data begins to be collected) documentation (the metrics.md that needs to be written).

As far as I can tell that's the only missing piece before Data Review can be successfully completed. (Though it needs to mention that the opt-out mechanism (Question 8) is the Firefox Data Collection preference, and it should probably note that analysis (Question 9) will be using sql.telemetry.mozilla.org)

Comment 13

[Liz Henry (:lizzard)]

Who should be doing the data review here? Chris is that you, or are you waiting on baku to write the documentation?

Comment 14

[Tony Cinotto [:tcinotto] (UTC-8)]

We have the metrics MD here: https://github.com/mozilla/secure-proxy/blob/master/docs/metrics.md.

I was told that this was completed and the bug will be opened up tomorrow during the launch. Baku/Chutten can you please confirm?

Comment 15

[Andrea Marchesini [:baku]]

The data review has been done. We are planning to make this bug public today.

Comment 16

[Chris H-C :chutten|PTO (back July 13)]

Comment on attachment 9081902 [details]
data-review form

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is documented in secure-proxy's [metrics.md file](https://github.com/mozilla/secure-proxy/blob/master/docs/metrics.md).

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, :baku is responsible.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

    Is the data collection request for default-on or default-off?

Default on for all channels.

    Does the instrumentation include the addition of any new identifiers?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---
Result: datareview+

Comment 17

[Liz Henry (:lizzard)]

baku, is it ok to close this bug as "fixed" now? It looks like it was a placeholder bug and the work is landing in github.

Comment 18

[Tony Cinotto [:tcinotto] (UTC-8)]

Hi Chris, can we close the bug as fixed now? Thanks!

Comment 19

[Chris H-C :chutten|PTO (back July 13)]

Work's done, so it seems fair.