Maybe stop setting Services.telemetry.canRecordExtended in tests
Categories
(Firefox :: Address Bar, task, P3)
Tracking
()
People
(Reporter: adw, Unassigned)
References
Details
A number of urlbar tests (and other front-end tests) set Services.telemetry.canRecordExtended = true. This prevented one of our tests from properly reporting failures for some telemetry that was incorrectly set to release opt-in instead of release opt-out. If the test had not set canRecordExtended, it would have failed once it was run against an RC in automation, and we would have realized the problem sooner. See bug 1780188 comment 9 and later for context.
I haven't thought about what we should do about this but the situation in bug 1780188 isn't good and we should try to prevent it from happening again.
|
Reporter | |
Comment 1•3 years ago
|
||
I looked into why the urlbar tests do this. It looks like canRecordExtended was first set to true in urlbar tests in the file that's now named browser_urlbar_telemetry.js. Before that it was browser/modules/test/browser_UsageTelemetry_urlbar.js. The context for that is in bug 1308420 comment 12:
(In reply to Mark Banner (:standard8) from bug 1308420 comment #12)
That failed, so I did some more thinking & investigation. It appears the
nightly-asan mozconfigs don't have MOZ_TELEMETRY_REPORTING set, as a result
the telemetry part of the test fails.Additionally, building locally without MOZ_TELEMETRY_REPORTING in the
.mozconfig would also fail the test.I eventually discovered that we need to do
Services.telemetry.canRecordExtended = true;to get this to work without
the option set.
|
|
Reporter | |
Comment 2•3 years ago
|
||
I removed all the canRecordExtended assignments in urlbar tests and ran them locally using a simulated Release build [1], and browser_urlbar_telemetry_tabtosearch.js failed because some tab-to-search scalars are also release opt-in. I haven't looked into whether that was intentional but I can't imagine so.
[1] I used an artifact mozconfig that also included the following:
ac_add_options --enable-official-branding
export MOZILLA_OFFICIAL=1
|
|
Reporter | |
Comment 3•3 years ago
|
||
(In reply to Drew Willcoxon :adw from comment #2)
I haven't looked into whether that was intentional but I can't imagine so.
Yes, it was intentional, see bug 1686330 and this comment in the telemetry doc in the urlbar.tabtosearch.* subsection:
Note
Due to the potentially sensitive nature of these data, they are currently collected only on pre-release version of Firefox. See bug 1686330.
Description
•