1604854 - Clear metrics with 'lifetime: user' if they expire

Status: RESOLVED FIXED

(Data Platform and Tools :: Glean: SDK, task, P1)

Description

[Alessio Placitelli [:Dexter]]

We currently don't provide any way to clear 'lifetime: user' metrics, even though they expire.

We should fix that so that any expired metric with the user lifetime gets purged from the user profile.

Comment 1

[Alessio Placitelli [:Dexter]]

Here's the current set of options we have: https://docs.google.com/document/d/1gWkYRJu2Elk_eQlmbSRHtrZLH-h_2BZ0632OXSRPa0E/edit#

What do you think? Please do leave feedback on the doc!

Comment 2

[Travis Long [:travis_]]

It's definitely a complex problem, and I don't have a clear preference for any of the options as I think we will have some difficulties with components/libraries that consume Glean. I see the need to have an API in Glean to register the components' allow-lists/tables and I see this as being a potential source of synchronization issues depending on when components are initialized and can interact with such an API.

Comment 3

[Michael Droettboom [:mdroettboom]]

Clearing my needinfo after a first read through the doc.

Comment 4

[Alessio Placitelli [:Dexter]]

This problem is effectively an hard nut to crack :-) We discussed this internally and decided on a reasonable strategy for moving forward.

1. We will de-emphasize the docs for ‘user’ lifetime, discouraging their use.
2. We will create a built-in list of currently known 'user' lifetime metrics in the SDK, to deal with their expiration.
3. (optional) we could create the built-in list from probe-scraper, in the future, if we see the number of such metrics grow.

This bug will deal with (1) and (2). Bug 1625123 will handle (3), if needed.

Comment 5

[Alessio Placitelli [:Dexter]]

Attachment: Update 'user' lifetime docs

Comment 6

[Alessio Placitelli [:Dexter]]

Attachment: Expiration mechanism in the SDK

Comment 7

[Alessio Placitelli [:Dexter]]

We will tackle this bug after the 1st of February, 2021. That's the date when all of the user lifetime metrics we're aware of will expire (most of them being in m-c).

The decision was due to the following reasons:

• hardcoding the list of know expirations in the code right now doesn't buy us anything compared to waiting;
• any expiration hardcoded in the SDK will work on "run time" date versus the "build time" expiration date we're using for all the metrics;
• we need to disable adding expirations to 'user' lifetimes anyway (see bug 1626516);

We can partially re-use what's in this PR for removing the data from the expired metrics.

Comment 8

[Michael Droettboom [:mdroettboom]]

Moving this to incoming as a reminder for us to triage it at our next meeting, as decided 10 months ago.

Comment 9

[Alessio Placitelli [:Dexter]]

There's two metrics that have expiration in 2022. Metrics with user lifetime should never expire, so we should fix these up.

Comment 10

[Alessio Placitelli [:Dexter]]

Hey,

the GFX team fixed comment 9 :-) Anything else we need to do here?

Comment 11

[Jan-Erik Rediger [:janerik]]

With nothing left to expire there's nothing to cleanup. I'm glad we didn't invest time to roll out a code change for this back in the day. :)