1426270 - Add telemetry for prefs loading

Status: RESOLVED FIXED

(Core :: Preferences: Backend, enhancement)

Description

[Nicholas Nethercote [inactive]]

Prefs loading is likely to get faster soon, due to bug 1423840 and bug 1426269. It would be good to have telemetry probes in place to determine the effects of these changes on release users.

We can also get some info about how big user prefs files are from these probes, and how common user.js prefs files are.

Comment 1

[Nicholas Nethercote [inactive]]

Attachment: Bug 1426270 - Add telemetry for prefs loading. .

This patch introduces three keyed histograms:

- PREFERENCES_FILE_LOAD_SIZE_B
- PREFERENCES_FILE_LOAD_NUM_PREFS
- PREFERENCES_FILE_LOAD_TIME_US

They are all keyed on the prefs file's name; in my local Linux64 build there
are 13 such files.

Because prefs start up earlier than telemetry, we have to save the measurements
and then pass them to telemetry later.

Review commit: https://reviewboard.mozilla.org/r/208944/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/208944/

Comment 2

[Nicholas Nethercote [inactive]]

> What questions will you answer with this data?

First: How much time does prefs file parsing take for real users (across all kinds of hardware, configurations, etc.)? Are there outliers where it is very slow?

Later: How much of a speedup do proposed optimizations of prefs file reading (bug 1423840, bug 1426269) give us?

Also: how big are the prefs.js and user.js files that hold preferences in user profiles? It's possible that some of them are unexpectedly large, in which case we would try to work out why and how to shrink them.

> Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

Relates to speed of Firefox, and disk space taken up by user profiles.

> What alternative methods did you consider to answer these questions? Why were they not sufficient?

We can measure developer's profiles on developers machines, but they aren't representative of the general population: developer machines tend to be much higher-end. Also, the variety of profile data for developers would be much lower.

> Can current instrumentation answer these questions?

No.

> List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories on the found on the Mozilla wiki.

The load times (PREFERENCES_FILE_LOAD_TIME_US) and file sizes (PREFERENCES_FILE_LOAD_SIZE_B) are category 1 "technical data".

The number of prefs (PREFERENCES_FILE_LOAD_NUM_PREFS) is either category 1 or category 2 "interaction data", I'm not sure.

These are accumulated on a per-file basis. The filenames are all known in advance, and don't provide any additional data about the user or their actions, other than some users have a user.js file and some do not.

> How long will this data be collected? Choose one of the following:

I have set the telemetry probes to expire in Firefox 62.

> What populations will you measure?

All. I want the widest population to maximize detection of outliers.

> Please provide a general description of how you will analyze this data.

For the times, seeing if there are outliers where loading is unexpectedly slow, and seeing how much bug 1423840 and bug 1426269 reduce them.

For the file sizes and number of prefs, seeing what fraction of users have user.js, and seeing if there are outliers with unexpectedly large prefs.js and user.js files.

> Where do you intend to share the results of your analysis?

Most likely in subsequent bugs, such as bug 1423840 and bug 1426269.

Comment 3

[Nicholas Nethercote [inactive]]

Comment on attachment 8938198 [details]
Bug 1426270 - Add telemetry for prefs loading. .

François, comment 2 has the data review details.

Comment 4

[François Marier [:francois]]

Comment on attachment 8938198 [details]
Bug 1426270 - Add telemetry for prefs loading. .

https://reviewboard.mozilla.org/r/208944/#review214656

1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes, histograms.json.

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, telemetry setting.

If the answer to either of the first two questions is no, reviewers give an r-. **Incremental changes to measurements or systems that have previously gone through analysis review may not require additional review.**

3) If the request is for permanent data collection, is there someone who will monitor the data over time?**

Not permanent.

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?  **

Category 1/2.

5) Is the data collection request for default-on or default-off?

Default on everywhere.

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?

No, the filenames are fixed and known ahead of time.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes.

8) Does there need to be a check-in in the future to determine whether to renew the data?

No, telemetry alerts will be sent.

Comment 5

[Mike Hommey [:glandium]]

Comment on attachment 8938198 [details]
Bug 1426270 - Add telemetry for prefs loading. .

https://reviewboard.mozilla.org/r/208944/#review214700

::: modules/libpref/Preferences.cpp:1556
(Diff revision 1)
> +  uint32_t loadTime_us = (TimeStamp::Now() - aStartTime).ToMicroseconds();
> +
> +  // Most prefs files are read before telemetry initializes, so we have to save
> +  // these measurements now and send them to telemetry later.
> +  TelemetryLoadData loadData = { uint32_t(aBufLen), numPrefs, loadTime_us };
> +  gTelemetryLoadData->Put(aName, loadData);

¢2 question: what if the same aName is used as a previous measure? Shouldn't the earliest one (likely the slowest one) be kept instead of overwritten?

Comment 6

[Nicholas Nethercote [inactive]]

> ¢2 question: what if the same aName is used as a previous measure? Shouldn't
> the earliest one (likely the slowest one) be kept instead of overwritten?

In normal Firefox execution there won't be any overwriting. I actually had an assertion to this effect, but xpcshell tests violated it so I removed it...

Comment 7

[Nicholas Nethercote [inactive]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/bfac566813209b4c66ee49e3c782ae950fc6482f
Bug 1426270 - Add telemetry for prefs loading. r=glandium.

Comment 8

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/bfac56681320