1855257 - Expose an API to disable glean-managed pings

Status: RESOLVED FIXED

(Data Platform and Tools :: Glean: SDK, enhancement, P1)

Description

[Alessio Placitelli [:Dexter]]

From the this ticket.

Glean currently provides the ability to send custom pings with no client_id. This allows collecting population wide telemetry without tracking a user across time.

For short lived applications (Firefox background tasks), this conflicts with baseline pings being sent automatically, as the two occur in sufficiently close proximity that it's probable as a vector to deanonymize users.

In order to accommodate this use case, Glean would need to expose a way to prevent baseline pings from being sent.

Comment 1

[Nick Alexander :nalexander [he/him]]

I didn't realize this was happening, even with this ticket existing. We're currently getting baseline pings for the WDBA and we probably shouldn't send them; we're collecting more than the single-daily-WDBA-ping-per-Windows-OS-user that we intended to. (See https://sql.telemetry.mozilla.org/queries/97124/source?p_channel=release&p_date=d_last_60_days#239796 for volumes.)

We definitely don't want to report Glean client_id in these baseline pings. What needs to happen to stop this collection?

Comment 2

[Jan-Erik Rediger [:janerik]]

Yeah, good question. Let me figure something out.

Comment 3

[Jan-Erik Rediger [:janerik]]

Right now the baseline ping is sent because FOG sets up the user activity observer.
We need that for normal usage and wanted that for other background tasks too, so we need to add a way to pass through a config to disable it. That's doable.

Talking with chutten made me aware that the WDBA storage is completely separate from the main profile, so the client_id sent from those WDBA baseline pings is not the same as the one from the main profile, so correlation with that is not directly possible.
I assume the WDBA mirrors the upload preference, which means it also sends deletion-request pings correctly.

:nalexander, I assume we can safely delete those baseline pings from WDBA, because noone should or does look at them anyway, correct?

Comment 4

[Nick Alexander :nalexander [he/him]]

(In reply to Jan-Erik Rediger [:janerik] from comment #3)

Right now the baseline ping is sent because FOG sets up the user activity observer.
We need that for normal usage and wanted that for other background tasks too, so we need to add a way to pass through a config to disable it. That's doable.

That would work -- perhaps initFOG takes a list of disabled pings or something like that.

Talking with chutten made me aware that the WDBA storage is completely separate from the main profile, so the client_id sent from those WDBA baseline pings is not the same as the one from the main profile, so correlation with that is not directly possible.
I assume the WDBA mirrors the upload preference, which means it also sends deletion-request pings correctly.

The first part is correct: there's no (direct) connection to browsing profiles. The second part is not: the WDBA is purposefully independent of the browsing profile, so the upload preferences will never be altered and deletion-request pings should never be sent. (The WDBA has its own data policy; it's possible to opt-out of it but it's not the same as the rest of Firefox's data collection.) The point here is that there should be no longitudinal identifier in any of the WDBA data, and right now there is.

:nalexander, I assume we can safely delete those baseline pings from WDBA, because noone should or does look at them anyway, correct?

Correct: we're not using this data and I don't think we event want access to this historical baseline ping data.

Comment 5

[Jan-Erik Rediger [:janerik]]

Attachment: Bug 1855257 - Add a flag to disable internal pings r?chutten!

Comment 6

[Jan-Erik Rediger [:janerik]]

Attachment: Bug 1855257 - Disable internal Glean pings for WDBA r?chutten!

Depends on D201078

Comment 7

[Jan-Erik Rediger [:janerik]]

The patches are a first try at adding a new flag letting us disable the pings.
Not totally happy that this is yet another flag, maybe we should switch to a configuration object instead or something.

I have yet to actually test that this works as expected though (and I don't have a Windows dev machine around, so I either need chutten's help or find another way to realistically test it out)

Comment 8

[Jan-Erik Rediger [:janerik]]

:chutten, when you get a chance can you take a look at the patches and give me a general vibe check if this is the approach we should take for now?

Comment 9

[Chris H-C :chutten]

This ain't half bad, but it's a bit of an architectural mishmash. It requires knowing that the Glean SDK only cares about handle_client_(in)active for the purposes of sending "events" or "baseline" pings. It requires assuming that it'll always be that case. It doesn't cover "events" pings with reasons max_capacity or startup (and probably shouldn't, unless we want monotonically-increasing events storage). It reduces our ability to report on metrics errors via the Glean Error Reporting Mechanism (sent on the "metrics" ping).

I'm not happy, but I could be convinced that this is the most expedient course.

If we're going to do this, I think I'd prefer an SDK-led solution so that there's some API or configuration that communicated unambiguously to the SDK to not send builtin pings. Instead of just "not calling handle_client_(in)active".

Comment 10

[BMO Github Automation]

Attachment: [mozilla/glean] Bug 1855257 - Add a configuration option to disable internal pings (#2786)

Comment 11

[Jan-Erik Rediger [:janerik]]

badboy merged PR [mozilla/glean]: Bug 1855257 - Add a configuration option to disable internal pings (#2786) in 4cb9b36.

The Rust parts landed and we should have that out soon.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d10fffe2c2aa
Add a flag to disable internal pings r=chutten
https://hg.mozilla.org/integration/autoland/rev/2c94c9a3e996
Disable internal Glean pings for WDBA r=chutten,application-update-reviewers,nalexander

Comment 13

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/d10fffe2c2aa
https://hg.mozilla.org/mozilla-central/rev/2c94c9a3e996