1529012 - Verification Analysis: Firefox Desktop's profile.firstUseDate

Status: RESOLVED MOVED

(Data Science :: Investigation, task)

Description

[Chris H-C :chutten]

Brief description of the request:

In bug 1495792 (Firefox 64) we introduced environment.profile.firstUseDate to be similar to what profile.creationDate is, but better in nearly every way.

We never verified that it was working as expected and actually was better in all the ways we hoped it would be.

I'd like to see an analysis that verifies that we're receiving firstUseDate as expected:

• from all clients new to us in Fx64 and none from before
• doesn't change over time (for clients with firstUseDate it should be present in all of their Environments and should maintain a stable value)
• has much more sensible values than PCD
  • has a much shorter distance to the first appearance date than PCD
  • has a much lower incidence rate of obviously incorrect values

Link to any assets:

Is there a specific data scientist you would like or someone who has helped to triage this request:

:shong helped out with this back in the day and suggested I file this bug.

Comment 1

[Chris H-C :chutten]

Oh, a note on timelines: whenever you get to it.

Though if we decide we want to use this exclusive of PCD for Ecosystem Telemetry (bug 1522664) that may change.

Comment 2

[Jess Mokrzecki [:jmok]]

Is this bug still valid or should we close?

Comment 3

[Felix Lawrence]

Yeah, this bug is languishing on my dashboard. I did some prelim work on this months ago but it got beaten out by higher priority and more urgent tasks.

Here are a few directions we could go from here:

1. I up-prioritise this and actually complete it
2. We send the bug back to triage and decide (whether) to reassign it to someone else
3. We close it as WONTFIX

As "dream up an exhaustive list of ways this might be broken and then check it's not broken in any of those ways" is a relatively exhausting exercise for me, I would prefer direction 2 or 3 - but if necessary I can actually complete it.

Comment 4

[Jess Mokrzecki [:jmok]]

Work for the DS team is now tracked in Jira. You can search with the Data Science Jira project for the corresponding ticket.

Comment 5

[Michael Droettboom [:mdroettboom]]

Attachment: JIRA bug replacing this bug

Comment 6

[Michael Droettboom [:mdroettboom]]

https://jira.mozilla.com/browse/DS-107

Comment 7

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

(cross-post to JIRA)

I briefly investigated the correctness of first_use_date. One of my main findings is that a users' min(first_use_date) is equal to their min(creation_date) [1] for ~98.5% of users who submit those values over a recent two week period. Since I was not able to fully understand what the bugs were in creation_date (bug 1449739), I'm not sure what conclusion to draw from this: on recent data, it seems likely that they either both have the bugs or neither of them do. first_use_date otherwise looked fairly sensible.

I wonder if the bugged creation_date values came primarily from the method to backfill a creation date based on the files in the profile and that we have proportionally fewer of these clients on recent data such that first_use_date and creation_date look very similar.

Disclaimer: I am not a data scientist, this analysis was not reviewed, and it was brief.