Closed Bug 1499552 Opened 2 years ago Closed 2 years ago
Report dummy prefs via telemetry to verify prefs are being set correctly
The telemetry collected by Normandy for the WebRender Pref experiment, and also the telemetry added in bug 1485156, shows that about 25% of people enrolled in the experiment via Normandy are reporting or acting as if the pref that's supposed to be set by Normandy is set (see bug 1495152). We'd like to verify that preferences set by Normandy Preference Experiments are set and are reported via telemetry correctly. So we'd like to land a dummy pref and report that via the telemetry reporting mechanism used to report the WebRender pref and run a dummy Normandy Preference Experiment to verify that Normandy Preference Experiments work as expected. This bug covers landing the pref and reporting it via TelemetryEnvironment.jsm.
The Webrender Pref Experiment is reporting its pref via telemetry and that is reporting a different value than what the Normandy experiments telemetry indicates should be being seen. So add reporting for two dummy prefs, one with a default value, and one without. We intend to push out Normandy rules to report these prefs to double-check that Normandy is working as expected.
Requesting data collection review. We are seeing telemetry which indicates that clients which are enrolled in a pref flip experiment are not having their preferences flipped. So we'd like to run a dummy experiment to verify that prefs get flipped.
Attachment #9018080 - Flags: review?(francois)
Comment on attachment 9018080 [details] 1499552-data-request.md 1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate? Yes, in this bug. 2) Is there a control mechanism that allows the user to turn the data collection on and off? Yes, disabling Shield. 3) If the request is for permanent data collection, is there someone who will monitor the data over time?** Not permanent. 4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under? ** Category 1. 5) Is the data collection request for default-on or default-off? Default ON. 6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)? No. 7) Is the data collection covered by the existing Firefox privacy notice? Yes. 8) Does there need to be a check-in in the future to determine whether to renew the data? No, it's a time-limited experiment.
Attachment #9018080 - Flags: review?(francois) → review+
Attachment #9017714 - Attachment description: Bug 1499552 - Add dummy pref and report via telemetry to verify telemetry pref reporting. r?mythmon → Bug 1499552 - Add dummy pref and report via telemetry to verify telemetry pref reporting. r=mythmon data-review=francois
Pushed by firstname.lastname@example.org: https://hg.mozilla.org/integration/autoland/rev/1c74b82884a8 Add dummy pref and report via telemetry to verify telemetry pref reporting. r=mythmon data-review=francois
Our study has concluded, and the Normandy rules are now disabled. ni? myself to remember to back out this data collection.
Since our study has concluded, I backed out this telemetry in bug 1502779.
You need to log in before you can comment on or make changes to this bug.