Add histogram to track file dates in profile directory

RESOLVED FIXED in Firefox 62

Status

()

enhancement
P1
normal
RESOLVED FIXED
Last year
Last year

People

(Reporter: janerik, Assigned: janerik)

Tracking

unspecified
mozilla62
Points:
---

Firefox Tracking Flags

(firefox62 fixed)

Details

Attachments

(2 attachments)

In bug 1449739 we suspect that certain profiles get their "Profile Creation Date" from timestamps of files in the profile directory.

We want to track timestamps of all files in the profile directory to more clearly know if these are unusual outliers.
The easiest might be to add a histogram to ProfileAge.jsm.

Unresolved questions:
* ProfileAge.jsm might not scan the directory at all, because the timestamp is written in times.json already. Do we need to account for that?
The only way a filesystem outlier can be an issue for a new profile is if we scan the filesystem on the new profile, so I think that it ought to be sufficient to record it only when the scan occurs.
Assignee: nobody → jrediger
Priority: -- → P2
Attachment #8979732 - Flags: review?(chutten)
Comment on attachment 8979731 [details]
Bug 1458574 - Add histogram to track file age in profile directory.

https://reviewboard.mozilla.org/r/245884/#review252216

Only small things. I am very curious to see the shape of this data.

::: commit-message-b75ac:8
(Diff revision 1)
> +During every scan of the profile directory, we now record the age relative to the current date of each file seen.
> +In theory we should rarely need to scan the profile directory, as on invocation it also stores the found time in a `times.json` file.
> +We therefore don't need precise tracking above 4 years (~ 1500 days).
> +
> +One thing to note:
> +ProfileAge might be invoked multiple times in parallel by different modules in the browser, each one then starting a profile directory scan,

ProfileAge can only ever be run on the main thread. Do you mean to highlight that getOldestProfileTimestamp could be re-entered?

::: toolkit/components/telemetry/Histograms.json:13730
(Diff revision 1)
> +    "bug_numbers": [1458574],
> +    "expires_in_version": "65",
> +    "high": 1500,
> +    "n_buckets": 20,
> +    "kind": "exponential",
> +    "description": "Age of files in profile directory in days"

Should probably have a terminal period.

Also, maybe it should mention when and where this is recorded so data spelunkers don't assume this is sent all the time and kept up-to-date.

::: toolkit/modules/ProfileAge.jsm:138
(Diff revision 1)
>     * Traverse the contents of the profile directory, finding the oldest file
>     * and returning its creation timestamp.
>     */
>    getOldestProfileTimestamp() {
>      let self = this;
> -    let oldest = Date.now() + 1000;
> +    let nowDate = Date.now();

bikeshed nit: it isn't really a date. Could call it "start" or something.
Attachment #8979731 - Flags: review?(chutten) → review+
Comment on attachment 8979732 [details]
Data Review 1458574.md

Redirecting to :francois for impartiality (I was closely involved in the development)
Attachment #8979732 - Flags: review?(chutten) → review?(francois)
Comment on attachment 8979731 [details]
Bug 1458574 - Add histogram to track file age in profile directory.

https://reviewboard.mozilla.org/r/245884/#review252382

::: toolkit/components/telemetry/Histograms.json:13724
(Diff revision 1)
>      "keyed": true,
>      "description": "Permission requests (showing a permission prompt) by whether they were requested from code handling a user input event."
>    },
> +  "PROFILE_DIRECTORY_FILE_AGE": {
> +    "record_in_processes": ["main"],
> +    "alert_emails": ["telemetry-client-dev@mozilla.com"],

Can you please add a person's email address to the list too?

(In order to help track potentially unused probes, we now ask that all new probes are tied to a specific individual.)

::: toolkit/components/telemetry/Scalars.yaml:1235
(Diff revision 1)
> +    description: >
> +      The number of times a profile directory scan was initiated in ProfileAge.jsm
> +    expires: "65"
> +    kind: uint
> +    notification_emails:
> +      - telemetry-client-dev@mozilla.com

ditto
Comment on attachment 8979732 [details]
Data Review 1458574.md

1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes, in Histograms.json.

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, telemetry setting.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?**

Not permanent.

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?  **

Category 1 or 2.

5) Is the data collection request for default-on or default-off?

Default ON in Nightly only.

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?

No.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes.

8) Does there need to be a check-in in the future to determine whether to renew the data?

No, telemetry alerts are fine.
Attachment #8979732 - Flags: review?(francois) → review+
BTW, next time, if you rename the data review form to a .txt, you can work-around bug 1421032 and have the form displayed inline in Bugzilla.
Priority: P2 → P1
Keywords: checkin-needed
Keywords: checkin-needed
Pushed by jrediger@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/64da6d6cc6d4
Add histogram to track file age in profile directory. r=chutten
https://hg.mozilla.org/mozilla-central/rev/64da6d6cc6d4
Status: NEW → RESOLVED
Closed: Last year
Resolution: --- → FIXED
Target Milestone: --- → mozilla62
You need to log in before you can comment on or make changes to this bug.