1680783 - Increase the resolution of first_run_date

Status: RESOLVED FIXED

(Data Platform and Tools :: Glean: SDK, defect, P2)

Description

[Michael Droettboom [:mdroettboom]]

In :gkabbz 's investigation, it may be helpful to have higher resolution for first_run_date to better group it into UTC days.

As a solution, this may be as simple as changing the unit on first_run_date, but that should go through data review as it is "collecting more data (at a higher resolution)".

Comment 1

[Chris H-C :chutten]

I recommend instead adding a datetime metric with a higher resolution to see if it helps, rather than jumping straight to decreasing the (deliberate) coarseness of first_run_date

(( Also, then wouldn't we have to rename it to first_run_hour ? : D ))

Comment 2

[Frank Bertsch [:frank]]

(In reply to Chris H-C :chutten from comment #1)

I recommend instead adding a datetime metric with a higher resolution to see if it helps, rather than jumping straight to decreasing the (deliberate) coarseness of first_run_date

(( Also, then wouldn't we have to rename it to first_run_hour ? : D ))

I agree with chutten here. I was going to recommend first_run_timestamp but first_run_hour is good as well.

Comment 3

[Michael Droettboom [:mdroettboom]]

That seems fine, as long as it's in both metrics, events and baseline pings, since I think all will be useful for this initial investigation.

Comment 4

[Michael Droettboom [:mdroettboom]]

Attachment: Data review request for `first_run_hour`

Comment 5

[Chris H-C :chutten]

Comment on attachment 9192679 [details]
Data review request for first_run_hour

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

No. This collection will expire on 2020-03-01.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does there need to be a check-in in the future to determine whether to renew the data?

Yes. :mdboom is responsible for renewing or removing the collection before it expires on 2020-03-01.

---

Result: datareview+

Comment 6

[Michael Droettboom [:mdroettboom]]

The implementation of this is getting hairy, as it's a special case that we need to bring across telemetry disabling/re-enabling (just as we do for first_run_date).

:chutten: How strongly do you feel about making this a test metric to start rather than just increasing the resolution of first_run_date now? I think it's pretty clear that this is the source of the timezone problems in the "first ping latency" analysis. The Glean pipeline (as it's ISO 8601 all the way down) makes it pretty easy to change the resolution of these values without requiring schema changes or otherwise.

Comment 7

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/glean/pull/1403

Comment 8

[Chris H-C :chutten]

The name'll be out of sync with its contents, which offends my sense of order in the universe, but that I'd get over : )

I'm still leaning towards implementing a version of first_run_hour as a metric to verify. Persisting it across telemetry opt-outs isn't necessary for the validation as we can use it to figure things out across the population that hasn't yet opted out. I'm a little confused as to why first_run_date, which has a timezone in it, contains insufficient data on its own to ascertain whether days starting at different hours are the reasons for the observed differences, so maybe I'm underestimating the need.

Comment 9

[Frank Bertsch [:frank]]

It occurred to me that there are half-hour timezones which could still be affected by this issue even with hourly resolution, though it will be a very small number (since it will only occur for those timezones, and only during the hour where they switch to UTC).

As someone who has previously dealt with the "this column has different values" problem from legacy mobile telemetry, I strongly recommend creating a new metric.

Comment 10

[Michael Droettboom [:mdroettboom]]

Ok -- so it sounds like consensus is to add a new metric. The problem that causes is a lot of test breakage because now we have a user-lifetime metric that appears in every ping (up until now, these were all special-case metrics in ping_info/client_info that don't trigger the sending of a ping in all cases). That test breakage can probably be fixed through some clever controlling of which pings to send in the tests themselves, but it definitely takes this bug out of the "quick fix" category.

:frank, good point about half hour timezones -- "most of India" is one of those timezones, so not a trivial number of people ;)

Comment 11

[Michael Droettboom [:mdroettboom]]

:chutten: The reason for this change is - a day with a timezone is insufficient to project into a UTC day, since one needs to know what time it is locally to know what day it is in UTC. All of this is to align with aggregating by submission date in UTC, which feels like the least bad of a bunch of bad options as the standard aggregation.

Another option would be to store/send first_run_date in UTC (which would do the timezone conversion on the client first) -- that wouldn't send data in any higher resolution than we do now, and wouldn't be any less accurate (wrt to incorrect clocks and timezones) than we have now.

Comment 12

[Chris H-C :chutten]

And we could keep the name. And all the analyses. And we wouldn't have this watershed of "Behaviour A before version X, Behaviour B afterwards" unless you sincerely care about time zones.