Closed Bug 1500251 Opened 3 years ago Closed 3 years ago

Report true as well as false dummy prefs via telemetry to verify prefs are being set correctly

Categories

(Firefox :: Normandy Client, enhancement)

enhancement
Not set
normal

Tracking

()

RESOLVED FIXED
Firefox 64
Tracking Status
firefox64 --- fixed

People

(Reporter: cpearce, Assigned: cpearce)

References

(Blocks 1 open bug)

Details

Attachments

(2 files)

We suspect Normandy Pref Experiments aren't correctly setting or reporting that they've set prefs. In bug 1499552 I added a default pref with value false for us to set via a Pref Experiments, but I think we should also have a pref with default true to set via the dummy Pref Experiment.

We're seeing about a 25% failure rate in the Pref Experiment, so maybe one of these four cases isn't working as expected?

So we should add another default pref with a default value of true so we can set that via a dummy Pref Experiment.
The Webrender Pref Experiment is reporting its pref via telemetry and that is
reporting a different value than what the Normandy experiments telemetry
indicates should be being seen.

In Bug 1499552 I added a pref and telemetry reporting in preparation for running
a dummy Pref Experiment to verify that prefs set are reported as set, but I
realised I missed a case; where there's an existing default pref set,
we're only covering the case where that default pref has value false. We're seeing
a ~25% failure rate in the pref reporting. So we should cover the other three cases
in the dummy Pref Experiment, just in case it's one of these four cases that
is failing.

So here I add another dummy pref with a default value of true, and I report that
via telemetry. I rename the pref I added yesterday to make things consistent.
I realised the data collection I requested yesterday in bug 1499552 missed a case, so requesting a second pref to be collected.
Attachment #9018406 - Flags: review?(francois)
Comment on attachment 9018406 [details]
Data collection request

The changes in this bug do not change any of the answers in the original data review: https://bugzilla.mozilla.org/show_bug.cgi?id=1499552#c3
Attachment #9018406 - Flags: review?(francois) → review+
Pushed by cpearce@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/38c1fc84fcaf
Add dummy pref with default value true, report via telemetry. r=mythmon
https://hg.mozilla.org/mozilla-central/rev/38c1fc84fcaf
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 64
Since our study has concluded, I backed out this telemetry in bug 1502779.
You need to log in before you can comment on or make changes to this bug.