Report dummy prefs via telemetry to verify prefs are being set correctly

RESOLVED FIXED in Firefox 64

Status

()

enhancement
RESOLVED FIXED
9 months ago
8 months ago

People

(Reporter: cpearce, Assigned: cpearce)

Tracking

(Blocks 1 bug)

unspecified
Firefox 64
Points:
---
Dependency tree / graph
Bug Flags:
qe-verify -

Firefox Tracking Flags

(firefox64 fixed)

Details

Attachments

(2 attachments)

The telemetry collected by Normandy for the WebRender Pref experiment, and also the telemetry added in bug 1485156, shows that about 25% of people enrolled in the experiment via Normandy are reporting or acting as if the pref that's supposed to be set by Normandy is set (see bug 1495152).

We'd like to verify that preferences set by Normandy Preference Experiments are set and are reported via telemetry correctly.

So we'd like to land a dummy pref and report that via the telemetry reporting mechanism used to report the WebRender pref and run a dummy Normandy Preference Experiment to verify that Normandy Preference Experiments work as expected.

This bug covers landing the pref and reporting it via TelemetryEnvironment.jsm.
The Webrender Pref Experiment is reporting its pref via telemetry and that
is reporting a different value than what the Normandy experiments telemetry
indicates should be being seen.

So add reporting for two dummy prefs, one with a default value, and one
without. We intend to push out Normandy rules to report these prefs to
double-check that Normandy is working as expected.
Requesting data collection review.

We are seeing telemetry which indicates that clients which are enrolled in a pref flip experiment are not having their preferences flipped. So we'd like to run a dummy experiment to verify that prefs get flipped.
Attachment #9018080 - Flags: review?(francois)
Comment on attachment 9018080 [details]
1499552-data-request.md

1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes, in this bug.

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, disabling Shield.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?**

Not permanent.

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?  **

Category 1.

5) Is the data collection request for default-on or default-off?

Default ON.

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?

No.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes.

8) Does there need to be a check-in in the future to determine whether to renew the data?

No, it's a time-limited experiment.
Attachment #9018080 - Flags: review?(francois) → review+
Attachment #9017714 - Attachment description: Bug 1499552 - Add dummy pref and report via telemetry to verify telemetry pref reporting. r?mythmon → Bug 1499552 - Add dummy pref and report via telemetry to verify telemetry pref reporting. r=mythmon data-review=francois
Pushed by cpearce@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/1c74b82884a8
Add dummy pref and report via telemetry to verify telemetry pref reporting. r=mythmon data-review=francois
https://hg.mozilla.org/mozilla-central/rev/1c74b82884a8
Status: NEW → RESOLVED
Closed: 9 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 64
Blocks: 1500230
Flags: qe-verify-
Our study has concluded, and the Normandy rules are now disabled. ni? myself to remember to back out this data collection.
Flags: needinfo?(cpearce)
Depends on: 1502779
Since our study has concluded, I backed out this telemetry in bug 1502779.
Flags: needinfo?(cpearce)
You need to log in before you can comment on or make changes to this bug.