1525045 - Implement the "first_run_date" (formerly known as "profile_age") in ping_info

Status: RESOLVED FIXED

(Toolkit :: Telemetry, enhancement, P1)

Description

[Alessio Placitelli [:Dexter]]

The baseline ping needs to contain information about how old is the profile sending telemetry.

The old telemetry library, handled it like that, by saving the date it was first run to a file.

We can probably do the same :)

Comment 1

[Alessio Placitelli [:Dexter]]

Jan-Erik suggests we could use firstInstallTime, available since API level 9. Sebastian, do you know how reliable that is compared to the way telemetry-library was doing (see comment 0)?

Comment 2

[Alessio Placitelli [:Dexter]]

(In reply to Alessio Placitelli [:Dexter] from comment #1)

Jan-Erik suggests we could use firstInstallTime, available since API level 9. Sebastian, do you know how reliable that is compared to the way telemetry-library was doing (see comment 0)?

Oh, nevermind, this is "install time", profile age is "age to first run time".

Comment 3

[Alessio Placitelli [:Dexter]]

I'm planning on using the Datetime metric type to power this, so that we're not reinventing a new storage system (since this needs to be a "user" lifetime metric).

*** Question 1 ***

Instead of reporting the profile_age, should we report the first_run_date? There was a discussion on IRC about this, and the consensus seemed to be that, since there's no concept of "profile" on mobile, doesn't make much sense to cargo-cult that name.

Moreover, we could use the metric types we support rather than have some other custom made mechanism just for this bit of information. We could use Datetime with a time_unit=day resolution and lifetime=user and be done with it :)

@All + @Chris, @Jan-Erik, did I capture the tone of the discussion correctly? Thoughts?

*** Question 2 (if we're cool with using our metric types) ***

There's one unclear aspect about the spec, though. We're allowing to specify the time_unit, as we do for Timespan. In this latter case, when assembling the ping, we're always sending out both the time_unit and the value of the timespan. That makes sense, as we're just reporting a raw number that doesn't make any sense without the accompanying time unit.

In the Datetime case though, we're always reporting an ISO8601 compliant date string, which will always make sense. The time_unit specified in the registry files will only affect the rounding of the date, e.g. time_unit=day will ensure the report string won't contain anything more granular than days, "2018-12-19T00:00-06:00". So I don't think we gain anything by sending the value of time_unit in the ping, along with the date string. This additional metadata will be available in the probe dictionary, sometime in the future.

@Mike, @Georg, @Frank, @Travis - what do you think?

Comment 4

[Jan-Erik Rediger [:janerik]]

👍 on first_run_date, that's what it is. Make sure to properly document that this is sourced from the user's device time, which tends to be unreliable as we learned (but should be good enough for most of our analysis)

Comment 5

[Michael Droettboom [:mdroettboom]]

I'm :+1: on not sending the time unit with datetimes, just truncating the datetime as specified. The current schema does include the time_unit, so we'll have to take it out, but that shouldn't be a big problem.

Comment 6

[Travis Long [:travis_]]

Question 1: first_run_date is more descriptive and accurate so +1 on that. I'm also all for using our built in metric types for this and DateTime seems like an obvious choice.

Question 2: I don't think a separate time-unit/resolution is meaningful to a DateTime. I very much like the idea of truncating on the desired granularity, such as midnight for day, the top of the hour for hour, etc.

Comment 7

[Chris H-C :chutten]

<Insert basically Jan-Erik's comment, but translated into chuttenglish>

Comment 8

[Frank Bertsch [:frank]]

Can we get first_run_date from the edge server? That would seriously benefit user retention analysis since we won't have to be concerned about client time.

Also, now that you've said install_date is available, I think we should include it. I hate to expand requirements but that could be very useful.

re: Question 1: first_run_date is fine. client_date could be added but that's just schematic sugar.
re: Question 2: No need to send time_unit, it will be in the probe-info-service.

Comment 9

[Alessio Placitelli [:Dexter]]

(In reply to Frank Bertsch [:frank] from comment #8)

Can we get first_run_date from the edge server? That would seriously benefit user retention analysis since we won't have to be concerned about client time.

Are you suggesting we don't add anything client side to the ping? How would you check on the edge if the client id was seen for the first time? Or do you mean we should still add this client side and then you would add the same metadata on the edge? (I suspect one could check the submission date/receiving date for pings with sequence == 0).

Also, now that you've said install_date is available, I think we should include it. I hate to expand requirements but that could be very useful.

Would you mind filing a bug about this here?

re: Question 1: first_run_date is fine. client_date could be added but that's just schematic sugar.
re: Question 2: No need to send time_unit, it will be in the probe-info-service.

Cool, thanks!

Comment 10

[Frank Bertsch [:frank]]

(In reply to Alessio Placitelli [:Dexter] from comment #9)

(In reply to Frank Bertsch [:frank] from comment #8)

Are you suggesting we don't add anything client side to the ping? How would you check on the edge if the client id was seen for the first time? Or do you mean we should still add this client side and then you would add the same metadata on the edge? (I suspect one could check the submission date/receiving date for pings with sequence == 0).

What I would truly love is the client to have the first_run date but from the server. It's notoriously difficult to track this information in the pipeline-side, but much simpler on the client (ask, answer, store). We had a previous discussion about using server-time.

One particular reason to use it in this case is if the client-clock is not only skewed, but drifts, N days into the future we can't use the existing skew to tell the previous skew (because it will have drifted even further every day).

Comment 11

[Chris H-C :chutten]

I have feelings about this.

1. I think we should at least conduct some preliminary skew analysis of mobile clocks to show that client skew's bad enough that we expect it to matter at the day resolution we're proposing for first_run_date.
2. Making that request is data leakage. I don't know what the process will be for glean-enabled applications, but on Desktop we give the user half an hour to review the Privacy Notice and decide whether or not to disable data upload. We don't talk to the Telemetry servers in the meantime.

Might be me making a mountain of a molehill, though.

Comment 12

[Michael Droettboom [:mdroettboom]]

Attachment: GitHub Pull Request

Comment 13

[Michael Droettboom [:mdroettboom]]

Attachment: GitHub Pull Request

Comment 14

[Michael Droettboom [:mdroettboom]]

Attachment: Data review for `firstRunDate`

Comment 15

[Chenxia Liu [:liuche]]

Is this going to replace profileAge in https://github.com/mozilla-mobile/android-components/blob/master/components/service/glean/docs/pings.md ? If so, can you also open an issue/PR to update that in the documentation? I just need to see the documentation for this change - once you link that, I can fill out the form to finish data review.

Comment 16

[Michael Droettboom [:mdroettboom]]

Attachment: GitHub Pull Request

Comment 17

[Michael Droettboom [:mdroettboom]]

:liuche There was some confusion around where this metric will go: ping_info (which is in absolutely every ping Glean sends) vs. "the baseline ping" (which is a ping containing metrics that every glean-using product sends). After clarifying, it looks like the plan is to put this on ping_info, and replace profileAge as you say. I will update the PR to reflect this and update the docs you linked to as part of that.

Comment 18

[Chenxia Liu [:liuche]]

Comment on attachment 9043618 [details]
Data review for `firstRunDate`

One nit, include this bug number in the Github PR.

data-review+
1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes - replaces `profile_age` probe in ping_info with `first_run_date`

2) Is there a control mechanism that allows the user to turn the data collection on and off? 
Yes, all products using Glean are required to include data controls and this will be controlled at the product level.


3) If the request is for permanent data collection, is there someone who will monitor the data over time?**
Yes, Glean team specifically :mdroettboom

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?  **
Type 1, app first start

5) Is the data collection request for default-on or default-off?
Default on

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?
No, timestamp only

7) Is the data collection covered by the existing Firefox privacy notice?
Yes

8) Does there need to be a check-in in the future to determine whether to renew the data? (Yes/No)
No, will be part of `ping_info` as baseline telemetry.

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3f414e712692
[components] 1525045: Implement "first_run_date" and dependencies