1667391 - Should Activity Stream data reporting respect the 30min Data Reporting Policy?

Status: RESOLVED INVALID

(Firefox :: Messaging System, task, P2)

Description

[Chris H-C :chutten]

Telemetry doesn't report data until 30min after the first session starts (or at that session's shutdown). I believe this was to ensure the user had a chance to view the Privacy Notice that we load on first run and make an informed choice to opt out of data collection if they so chose.

:decoder was surprised on a new profile to see "telemetry" being sent within that 30min window. According to his logs, they're all being sent to the activity-stream structured ingestion endpoint. (so it doesn't point to a bug in TelemetryReportingPolicy.jsm or TelemetrySend.jsm)

I don't know if Activity Stream should respect the 30min "read the policy" window, but it seems to have (at least for :decoder) violated the "No Surprises" data collection doctrine.

Comment 1

[Tim Spurway [:tspurway]]

Nan, could you weigh in on this one? I recall covering this with the legal team some time ago.

Comment 2

[Nan Jiang [:nanj]]

:chutten, thanks for bringing this up!

This is a known "concern" of PingCentre which serves as the telemetry client for Activity Stream and various related projects. In fact, this concern had been raised on several occasions by our data stewards, and we had to consult with the legal and trust team in a case by case manner to get the signoff (such as this) and go ahead without having the 30min data collection delay.

Admittedly, this is not ideal. We've been working on migrating existing AS telemetry to use the standard Telemetry.* API (such as 1, 2) so that the data reporting policy could be properly respected. Though certain metrics, e.g. those that can't be sent with client_id, still rely on the current client to do the work.

:chutten, would like to hear your thoughts about the best solution here. Shall we wait for the new Telemetry.* API, like Glean, so that we can completely switch over to the standard API? Or we should consider implementing the policy in PingCentre if the new API still needs some time to be ready on desktop? My main concern with the latter is that implementing the whole policy again sounds quite wasteful as we'll be phasing out PingCentre eventually unless we can reuse some existing parts from the Teleemtry.* module.

Comment 3

[Chris H-C :chutten]

I've only just reached out to Trust about the Reporting Policy specifics when it comes to Project FOG (Firefox on Glean) which will bring the Glean SDK to FIrefox Desktop, so I don't know what things will look like there. It might be that the 30min policy might stay as just a Firefox Telemetry thing, or it might be that FOG will adopt the existing TelemetryReportingPolicy, or it might be that FOG develops its own internal timer, or something else entirely. I will let you know how that goes.

In the meantime it sounds as though signoff has been granted so the answer to the originating question is "No. Activity Stream uses PingCentre which does not need to respect the 30min Policy."

:decoder, does this satisfy your original question?

Comment 4

[Christian Holler (:decoder)]

(In reply to Chris H-C :chutten from comment #3)

I've only just reached out to Trust about the Reporting Policy specifics when it comes to Project FOG (Firefox on Glean) which will bring the Glean SDK to FIrefox Desktop, so I don't know what things will look like there. It might be that the 30min policy might stay as just a Firefox Telemetry thing, or it might be that FOG will adopt the existing TelemetryReportingPolicy, or it might be that FOG develops its own internal timer, or something else entirely. I will let you know how that goes.

In either case it sounds like there will be a consistent policy and I think that is important.

In the meantime it sounds as though signoff has been granted so the answer to the originating question is "No. Activity Stream uses PingCentre which does not need to respect the 30min Policy."

I am not very positive about this. From a users perspective, there is no such distinction between PingCentre and Telemetry, if you take a look at the data reporting, it looks virtually identical (same data endpoint). The risk here is that the whole Telemetry 30min policy is rendered ineffective.

Comment 5

[Chris H-C :chutten]

Alrighty, let's bring in Trust to see what the intent is. Is it that Telemetry is doing something unnecessary and we just let it keep doing its own thing, or is it that the 30min "policy" is aspirational and we should strive for uniformity? Or something else?

Comment 6

[Alicia Gray]

Trust feedback: Here’s what I’d like us to do.

1. We should treat data collection timing for Firefox telemetry consistently across desktop, mobile and Activity Stream.
2. Maintain the current policy of 30 minute delay.
3. Apply the 30 minute delay to desktop, mobile and Activity Stream.

If Activity Stream has a special case as to why the pings should be received before the 30 minute delay, let's review again. I cannot locate the reason why this timing difference was necessary or referenced in https://bugzilla.mozilla.org/show_bug.cgi?id=1550065#c6

Comment 7

[Nan Jiang [:nanj]]

Thanks for the feedback, Alicia!

We've come up with a plan to implement the 30min delay policy for PingCentre.

:chutten, is there any doc available for the policy so that we can reference with?

/cc Kenny and Kirill for this change.

Comment 8

[Chris H-C :chutten]

(In reply to Nan Jiang [:nanj] from comment #7)

:chutten, is there any doc available for the policy so that we can reference with?

None that I know of. It predates my involvement in the data client. Let's do some sleuthing...

Huh. Well this changes things quite a bit. It appears as though there is no 30min delay policy after all? Come with me down this rabbit hole.

• bug 1576948 approaches early telemetry from a "huh, this isn't desireable" but without any specific threshold. It pushes out "environment-change"-reason "main" pings and "modules" pings that happened a bit early (10 and 11 min).
• bug 1351393 is where, after a discussion with Legal, we decided 30min was a good delay for the "new-profile" ping only

And that's it. There's no timing code or firstrun detection in TelemetrySend to make sure we don't send anything. There's no timing code in TelemetryReportingPolicy itself to ensure canUpload returns false for a set period of time... there's no bug or code or test or doc suggesting that this should have ever applied to anything besides the "new-profile" ping.

It seems as though we've kinda accidentally stumbled along with the "new-profile" ping's specific needs informing a generic data collection policy all this time? TIL.

ni?alicia again to make sure we're not blindly stepping up a generic policy without considering the full picture. It seems to me as though there's demand for a data-free window for a user to peruse the Privacy Notice and find the opt out. Do we want to create a policy? Should it be 30min? (maybe 10min is better?) Where should it be documented? Who owns the policy and is responsible for updating it as circumstances change around it?

Comment 9

[Alicia Gray]

:chutten I'll set a followup meeting with you to ask some questions around the different pings mentioned in Comment 8 and try to get some more background information.

I agree, we don't want to set up a generic policy based on partial information and there's enough historical context lacking here to be able to fully evaluate for a decision point. It sounds like Legal holds the main historical reference/knowledge so this may not be able to be resolved until next month due to an OOO status.

Comment 10

[Jessilyn Davis]

howdy folks! Can you be sure to include me, Kenny & Kirill in any convos around this?

Alicia, I'll add it this as an agenda item to our next reg. occurring mtg (next week). Thanks!

Comment 11

[Alicia Gray]

(In reply to Jessilyn Davis from comment #10)

howdy folks! Can you be sure to include me, Kenny & Kirill in any convos around this?

Alicia, I'll add it this as an agenda item to our next reg. occurring mtg (next week). Thanks!

HI Jess,

Let's keep off the agenda for now. The person that holds the remaining historical context that I need to be able to better consider all pieces here is out until November. Once I have that we'll be positioned to fully evaluate. I'll update the bug after that happens, but for now we'll pend.

Comment 12

[Chris H-C :chutten]

Activity Stream will no longer be special after bug 1820548's removal of PingCentre. It'll adhere to the global behaviour then.