Telemetry: Stop all monkey patching in devtools telemetry tests

RESOLVED FIXED in Firefox 61

Status

DevTools
Inspector
P3
normal
RESOLVED FIXED
4 years ago
a month ago

People

(Reporter: miker, Assigned: miker)

Tracking

(Blocks: 2 bugs)

unspecified
Firefox 61
Dependency tree / graph

Firefox Tracking Flags

(firefox61 fixed)

Details

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(1 attachment, 15 obsolete attachments)

59 bytes, text/x-review-board-request
yulia
: review+
Details | Review
Current telemetry tests monkey patch telemetry.js and gather the information from telemetry.log(). Unfortunately, this method fails for bug 1091796 because the instance of telemetry used in browser.js is not affected by any tests monkey patching.

I have found a way to avoid monkey patching whilst making telemetry tests far more reliable. I have also created a method that can evaluate changed telemetry histograms and generate a list of checks that can be pasted into the result checking method of a test... this makes it very simple to create reliable telemetry tests.
Assignee: nobody → mratcliffe
Created attachment 8522999 [details] [diff] [review]
fix-telemetry-tests-1098374.patch

Due to revising these tests I have discovered that the telemetry information for the following tools was reported twice. This means that the number of times these tools were opened (DEVTOOLS_*_OPENED_BOOLEAN) should be half of what was reported before this patch landed:

- Canvas Debugger
- Shader Editor
- Storage Inspector
- Webaudio Editor

Try:
https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=d58efefbf19b
https://tbpl.mozilla.org/?tree=Try&rev=d58efefbf19b
Attachment #8522999 - Flags: review?(fayearthur)
Jeff: Just pinging you to let you know that the amount of times some tools were opened was reported twice per tool opening... see comment 1.
Flags: needinfo?(jgriffiths)
(In reply to Michael Ratcliffe [:miker] [:mratcliffe] from comment #2)
> Jeff: Just pinging you to let you know that the amount of times some tools
> were opened was reported twice per tool opening... see comment 1.

Okay - nothing *really* critical, but good to know.
Flags: needinfo?(jgriffiths)
Comment on attachment 8522999 [details] [diff] [review]
fix-telemetry-tests-1098374.patch

Review of attachment 8522999 [details] [diff] [review]:
-----------------------------------------------------------------

There's a telemetry error hitting all the tests. Fix that and re-flag me.
Attachment #8522999 - Flags: review?(fayearthur)
Created attachment 8524525 [details] [diff] [review]
fix-telemetry-tests-1098374.patch

(In reply to Heather Arthur  [:harth] from comment #4)
> Comment on attachment 8522999 [details] [diff] [review]
> fix-telemetry-tests-1098374.patch
> 
> Review of attachment 8522999 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> There's a telemetry error hitting all the tests. Fix that and re-flag me.

Fixed. We need to use a try / catch in the histogram loop (no choice) but the cost of doing so in Spidermonkey is negligible.
Attachment #8522999 - Attachment is obsolete: true
Attachment #8524525 - Flags: review?(fayearthur)
Attachment #8524525 - Flags: review?(fayearthur)
Comment on attachment 8524525 [details] [diff] [review]
fix-telemetry-tests-1098374.patch

Lookin' green now... of course, I will convert any tests that land before this.
Attachment #8524525 - Flags: review?(fayearthur)
Created attachment 8526632 [details] [diff] [review]
fix-telemetry-tests-1098374.patch

Rebase
Attachment #8524525 - Attachment is obsolete: true
Attachment #8524525 - Flags: review?(fayearthur)
Attachment #8526632 - Flags: review?(fayearthur)
Comment on attachment 8526632 [details] [diff] [review]
fix-telemetry-tests-1098374.patch

Review of attachment 8526632 [details] [diff] [review]:
-----------------------------------------------------------------

This is an awesome cleanup, thank you. I just realized that all the toolbox tab tests also have a repetitive reportError() function. If you want to consolidate that too, that's nice, but if you don't no worries.
Attachment #8526632 - Flags: review?(fayearthur) → review+
Blocks: 990741
Created attachment 8530523 [details] [diff] [review]
fix-telemetry-tests-1098374.patch

Sorry this took time but I have spent the last few days in hospital (E-coli O157).

(In reply to Heather Arthur  [:harth] from comment #10)
> Comment on attachment 8526632 [details] [diff] [review]
> fix-telemetry-tests-1098374.patch
> 
> Review of attachment 8526632 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> This is an awesome cleanup, thank you. I just realized that all the toolbox
> tab tests also have a repetitive reportError() function. If you want to
> consolidate that too, that's nice, but if you don't no worries.

Nit addressed.
Attachment #8526632 - Attachment is obsolete: true
Attachment #8530523 - Flags: review+
Created attachment 8530536 [details] [diff] [review]
fix-telemetry-tests-1098374.patch

Renamed "hastimeentries" to "hasentries" as this makes more sense for some linear / exponential histograms.
Attachment #8530523 - Attachment is obsolete: true
Attachment #8530536 - Flags: review+
Created attachment 8565902 [details] [diff] [review]
fix-telemetry-tests-1098374.patch

Rebased.

E10S issue appears to have been fixed.

Try:
https://tbpl.mozilla.org/?tree=Try&rev=5f08fe65d48f
Attachment #8530536 - Attachment is obsolete: true
Attachment #8565902 - Flags: review+
Sorry had to back this out in https://treeherder.mozilla.org/#/jobs?repo=fx-team&revision=73f1285748cf since one of this 2 patches caused :

https://treeherder.mozilla.org/logviewer.html#?job_id=2018723&repo=fx-team

(that failure is also in the try run)
Flags: needinfo?(mratcliffe)
No longer blocks: 1091796
Comment on attachment 8565902 [details] [diff] [review]
fix-telemetry-tests-1098374.patch

I will need to investigate what is causing this error.
Flags: needinfo?(mratcliffe)
Attachment #8565902 - Flags: checkin+ → checkin-
Created attachment 8585554 [details] [diff] [review]
fix-telemetry-tests-1098374.patch

Refactored a little and now it seems that we avoid the ABORT in histogram.cc:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=5689ce1c5766

Anyhow, try is green but I would prefer another review to finally get this thing landed.

bgrins was asking about what was happening with the telemetry tests so I will have him re-review the code considering that harth is no longer around.
Attachment #8565902 - Attachment is obsolete: true
Attachment #8585554 - Flags: review?(bgrinstead)
Comment on attachment 8585554 [details] [diff] [review]
fix-telemetry-tests-1098374.patch

Review of attachment 8585554 [details] [diff] [review]:
-----------------------------------------------------------------

::: browser/devtools/shared/test/browser_telemetry_misc.js
@@ +2,5 @@
> +   http://creativecommons.org/publicdomain/zero/1.0/ */
> +
> +const TEST_URI = "data:text/html;charset=utf-8,<p>browser_telemetry_misc.js</p>";
> +
> +// Because we need to gather stats for the period of time that a tool has been

It doesn't look like checkResults is actually looking at the tool active times in this test.  Seems like we either don't need this delay + comment, or the assertions should be testing the relevant telemetry bits.

Also, it looks like this test does the same thing as browser_telemetry_toolboxtabs_inspector.js.

::: browser/devtools/webide/test/head.js
@@ +275,5 @@
> + *
> + * @param  {String} prefix
> + *         Optionally limits results to histogram ids starting with prefix.
> + */
> +function generateTelemetryTests(prefix="") {

I guess we don't really have another place to put this sort of thing.  Doesn't really hurt to have it here though.
Attachment #8585554 - Flags: review?(bgrinstead)
Created attachment 8585662 [details] [diff] [review]
fix-telemetry-tests-1098374.patch

(In reply to Brian Grinstead [:bgrins] from comment #20)
> Comment on attachment 8585554 [details] [diff] [review]
> fix-telemetry-tests-1098374.patch
> 
> Review of attachment 8585554 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: browser/devtools/shared/test/browser_telemetry_misc.js
> @@ +2,5 @@
> > +   http://creativecommons.org/publicdomain/zero/1.0/ */
> > +
> > +const TEST_URI = "data:text/html;charset=utf-8,<p>browser_telemetry_misc.js</p>";
> > +
> > +// Because we need to gather stats for the period of time that a tool has been
> 
> It doesn't look like checkResults is actually looking at the tool active
> times in this test.  Seems like we either don't need this delay + comment,
> or the assertions should be testing the relevant telemetry bits.
> 

Delay and comment removed

> Also, it looks like this test does the same thing as
> browser_telemetry_toolboxtabs_inspector.js.
> 

Yeah, we are checking different telemetry stats though and expect to a a bunch more to the misc test in the near future.

> ::: browser/devtools/webide/test/head.js
> @@ +275,5 @@
> > + *
> > + * @param  {String} prefix
> > + *         Optionally limits results to histogram ids starting with prefix.
> > + */
> > +function generateTelemetryTests(prefix="") {
> 
> I guess we don't really have another place to put this sort of thing. 
> Doesn't really hurt to have it here though.

True but this really helps when generating telemetry tests.
Attachment #8585554 - Attachment is obsolete: true
Attachment #8585662 - Flags: review?(bgrinstead)
Comment on attachment 8585662 [details] [diff] [review]
fix-telemetry-tests-1098374.patch

Review of attachment 8585662 [details] [diff] [review]:
-----------------------------------------------------------------

Taking into account harth's r+ and assuming the try is green -- ship it.  Please see the comment about removing duplicate code from webide/test/head.js

::: browser/devtools/shared/test/browser_telemetry_misc.js
@@ +8,5 @@
> +  yield promiseTab(TEST_URI);
> +
> +  startTelemetry();
> +
> +  yield openAndCloseToolbox(1, TOOL_DELAY, "inspector");

I wish the parameter ordering on this function was different and took toolID first and then had a default value for number of times and the delay.  Then this case could just call openAndCloseToolbox("inspector").  (No need to address this here)

::: browser/devtools/webide/test/head.js
@@ +275,5 @@
> + *
> + * @param  {String} prefix
> + *         Optionally limits results to histogram ids starting with prefix.
> + */
> +function generateTelemetryTests(prefix="") {

If this code is the same as the one in in shared/head.js then I think it shouldn't be duplicated here (since it's a one-off script that's never directly used in tests it is very likely to fall out of sync).  You could instead put instructions in the webide test to use generateTelemetryTests from the shared/test directory.
Attachment #8585662 - Flags: review?(bgrinstead) → review+
Keywords: checkin-needed
There's a ton of test failures in that Try push. Can you please post a run with clearly-OK results?
Keywords: checkin-needed
Created attachment 8593351 [details] [diff] [review]
fix-telemetry-tests-1098374.patch

My apologies, try:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=1ad68d9e0226
Attachment #8590842 - Attachment is obsolete: true
Attachment #8593351 - Flags: review+
Keywords: checkin-needed
Comment hidden (obsolete)
Comment hidden (obsolete)
Created attachment 8595849 [details] [diff] [review]
fix-telemetry-tests-1098374.patch

Discussed this with jwalker who said that test shouldn't pass on E10S builds as the Toolbar isn't remoted so this failure is no surprise (all toolbar tests should really be disabled on E10S).

He remotes the toolbar as part of bug 1128988 but for now the best option is to disable the test.

Try:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c444937ec8f4
Attachment #8593351 - Attachment is obsolete: true
Flags: needinfo?(mratcliffe)
Attachment #8595849 - Flags: review+
Comment hidden (obsolete)
Created attachment 8972368 [details] [diff] [review]
Telemetry: Stop all monkey patching in devtools telemetry tests
Attachment #8595849 - Attachment is obsolete: true
Attachment #8595849 - Flags: review+
Created attachment 8972665 [details] [diff] [review]
Telemetry: Stop all monkey patching in devtools telemetry tests
Attachment #8972368 - Attachment is obsolete: true
Created attachment 8972810 [details] [diff] [review]
Telemetry: Stop all monkey patching in devtools telemetry tests
Attachment #8972665 - Attachment is obsolete: true
Created attachment 8972931 [details] [diff] [review]
Telemetry: Stop all monkey patching in devtools telemetry tests
Attachment #8972810 - Attachment is obsolete: true
Comment hidden (mozreview-request)
Attachment #8972931 - Attachment is obsolete: true

Comment 40

3 months ago
mozreview-review
Comment on attachment 8972953 [details]
Bug 1098374 - Telemetry: Stop all monkey patching in devtools telemetry tests

https://reviewboard.mozilla.org/r/241522/#review247330


Code analysis found 91 defects in this patch (only the first 30 are reported here):
 - 91 defects found by mozlint

You can run this analysis locally with:
 - `./mach lint path/to/file` (JS/Python)


If you see a problem in this automated review, please report it here: http://bit.ly/2y9N9Vx


::: devtools/client/framework/test/browser_toolbox_hosts_telemetry.js:12
(Diff revision 1)
> -function getHostHistogram() {
> -  return Services.telemetry.getHistogramById("DEVTOOLS_TOOLBOX_HOST");
> -}
> -
>  add_task(async function() {
> -  // Reset it to make counting easier
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/framework/test/browser_toolbox_hosts_telemetry.js:38
(Diff revision 1)
>  function checkResults() {
> -  let counts = getHostHistogram().snapshot().counts;
> -  is(counts[0], 3, "Toolbox HostType bottom has 3 successful entries");
> -  is(counts[1], 2, "Toolbox HostType side has 2 successful entries");
> -  is(counts[2], 2, "Toolbox HostType window has 2 successful entries");
> +  // Check for:
> +  //   - 3 "bottom" entries.
> +  //   - 2 "side" entries.
> +  //   - 2 "window" entries.
> +  checkTelemetry("DEVTOOLS_TOOLBOX_HOST", "", [3, 2, 2, 0, 0, 0, 0, 0, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/inspector/grids/test/browser_grids_number-of-css-grids-telemetry.js:28
(Diff revision 1)
> -const CSS_GRID_COUNT_HISTOGRAM_ID = "DEVTOOLS_NUMBER_OF_CSS_GRIDS_IN_A_PAGE";
> -
>  add_task(async function() {
>    await addTab("data:text/html;charset=utf-8," + encodeURIComponent(TEST_URI1));
>  
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/inspector/grids/test/browser_grids_number-of-css-grids-telemetry.js:46
(Diff revision 1)
>  });
> +
> +function checkresults() {
> +  // Check for:
> +  //   - 1 CSS Grid Element
> +  checkTelemetry("DEVTOOLS_NUMBER_OF_CSS_GRIDS_IN_A_PAGE", "",

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-01.js:16
(Diff revision 1)
>  const { UI_ENABLE_MEMORY_PREF } = require("devtools/client/performance/test/helpers/prefs");
>  const { initPerformanceInNewTab, teardownToolboxAndRemoveTab } = require("devtools/client/performance/test/helpers/panel-utils");
>  const { startRecording, stopRecording } = require("devtools/client/performance/test/helpers/actions");
>  
>  add_task(async function() {
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-01.js:41
(Diff revision 1)
>  });
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_PERFTOOLS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_RECORDING_COUNT", "", [2, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-01.js:42
(Diff revision 1)
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_PERFTOOLS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_RECORDING_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_RECORDING_DURATION_MS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-01.js:43
(Diff revision 1)
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_PERFTOOLS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_RECORDING_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_RECORDING_DURATION_MS", "", null, "hasentries");
> +  checkTelemetry(

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-01.js:45
(Diff revision 1)
> +  // here.
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_RECORDING_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_RECORDING_DURATION_MS", "", null, "hasentries");
> +  checkTelemetry(
> +    "DEVTOOLS_PERFTOOLS_RECORDING_FEATURES_USED", "withMarkers", [0, 2, 0], "array");
> +  checkTelemetry(

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-01.js:47
(Diff revision 1)
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_RECORDING_DURATION_MS", "", null, "hasentries");
> +  checkTelemetry(
> +    "DEVTOOLS_PERFTOOLS_RECORDING_FEATURES_USED", "withMarkers", [0, 2, 0], "array");
> +  checkTelemetry(
> +    "DEVTOOLS_PERFTOOLS_RECORDING_FEATURES_USED", "withMemory", [1, 1, 0], "array");
> +  checkTelemetry(

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-01.js:49
(Diff revision 1)
> +    "DEVTOOLS_PERFTOOLS_RECORDING_FEATURES_USED", "withMarkers", [0, 2, 0], "array");
> +  checkTelemetry(
> +    "DEVTOOLS_PERFTOOLS_RECORDING_FEATURES_USED", "withMemory", [1, 1, 0], "array");
> +  checkTelemetry(
> +    "DEVTOOLS_PERFTOOLS_RECORDING_FEATURES_USED", "withAllocations", [2, 0, 0], "array");
> +  checkTelemetry(

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-02.js:16
(Diff revision 1)
>  const { initPerformanceInNewTab, teardownToolboxAndRemoveTab } = require("devtools/client/performance/test/helpers/panel-utils");
>  const { startRecording, stopRecording } = require("devtools/client/performance/test/helpers/actions");
>  const { once } = require("devtools/client/performance/test/helpers/event-utils");
>  
>  add_task(async function() {
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-02.js:47
(Diff revision 1)
>  });
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_PERFTOOLS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_RECORDING_IMPORT_FLAG", "", [0, 1, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-02.js:48
(Diff revision 1)
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_PERFTOOLS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_RECORDING_IMPORT_FLAG", "", [0, 1, 0], "array");
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_RECORDING_EXPORT_FLAG", "", [0, 1, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-03.js:16
(Diff revision 1)
>  const { initPerformanceInNewTab, teardownToolboxAndRemoveTab } = require("devtools/client/performance/test/helpers/panel-utils");
>  const { startRecording, stopRecording } = require("devtools/client/performance/test/helpers/actions");
>  const { once } = require("devtools/client/performance/test/helpers/event-utils");
>  
>  add_task(async function() {
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-03.js:51
(Diff revision 1)
>  });
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_PERFTOOLS_")
> +  // here.
> +  checkTelemetry(

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-03.js:53
(Diff revision 1)
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_PERFTOOLS_")
> +  // here.
> +  checkTelemetry(
> +    "DEVTOOLS_PERFTOOLS_SELECTED_VIEW_MS", "js-calltree", null, "hasentries");
> +  checkTelemetry(

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-03.js:55
(Diff revision 1)
> +  // here.
> +  checkTelemetry(
> +    "DEVTOOLS_PERFTOOLS_SELECTED_VIEW_MS", "js-calltree", null, "hasentries");
> +  checkTelemetry(
> +    "DEVTOOLS_PERFTOOLS_SELECTED_VIEW_MS", "js-flamegraph", null, "hasentries");
> +  checkTelemetry(

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-04.js:14
(Diff revision 1)
>  const { SIMPLE_URL } = require("devtools/client/performance/test/helpers/urls");
>  const { initPerformanceInTab, initConsoleInNewTab, teardownToolboxAndRemoveTab } = require("devtools/client/performance/test/helpers/panel-utils");
>  const { waitForRecordingStartedEvents, waitForRecordingStoppedEvents } = require("devtools/client/performance/test/helpers/actions");
>  
>  add_task(async function() {
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-04.js:44
(Diff revision 1)
>  });
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_PERFTOOLS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_CONSOLE_RECORDING_COUNT", "", [1, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-04.js:45
(Diff revision 1)
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_PERFTOOLS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_CONSOLE_RECORDING_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_RECORDING_DURATION_MS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-04.js:46
(Diff revision 1)
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_PERFTOOLS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_CONSOLE_RECORDING_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_RECORDING_DURATION_MS", "", null, "hasentries");
> +  checkTelemetry(

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-04.js:48
(Diff revision 1)
> +  // here.
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_CONSOLE_RECORDING_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_RECORDING_DURATION_MS", "", null, "hasentries");
> +  checkTelemetry(
> +    "DEVTOOLS_PERFTOOLS_RECORDING_FEATURES_USED", "withMarkers", [0, 1, 0], "array");
> +  checkTelemetry(

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-04.js:50
(Diff revision 1)
> +  checkTelemetry("DEVTOOLS_PERFTOOLS_RECORDING_DURATION_MS", "", null, "hasentries");
> +  checkTelemetry(
> +    "DEVTOOLS_PERFTOOLS_RECORDING_FEATURES_USED", "withMarkers", [0, 1, 0], "array");
> +  checkTelemetry(
> +    "DEVTOOLS_PERFTOOLS_RECORDING_FEATURES_USED", "withMemory", [1, 0, 0], "array");
> +  checkTelemetry(

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/performance/test/browser_perf-telemetry-04.js:52
(Diff revision 1)
> +    "DEVTOOLS_PERFTOOLS_RECORDING_FEATURES_USED", "withMarkers", [0, 1, 0], "array");
> +  checkTelemetry(
> +    "DEVTOOLS_PERFTOOLS_RECORDING_FEATURES_USED", "withMemory", [1, 0, 0], "array");
> +  checkTelemetry(
> +    "DEVTOOLS_PERFTOOLS_RECORDING_FEATURES_USED", "withAllocations", [1, 0, 0], "array");
> +  checkTelemetry(

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_button_eyedropper.js:10
(Diff revision 1)
>    "<p>browser_telemetry_button_eyedropper.js</p><div>test</div>";
> -const EYEDROPPER_OPENED = "devtools.toolbar.eyedropper.opened";
>  
>  add_task(async function() {
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_button_eyedropper.js:34
(Diff revision 1)
>  }
>  
> -function checkTelemetryResults(Telemetry) {
> -  let data = Telemetry.prototype.telemetryInfo;
> -  let results = new Map();
> -
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("devtools.")
> +  // here.
> +  checkTelemetry("devtools.toolbar.eyedropper.opened", "", 1, "scalar");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_button_paintflashing.js:16
(Diff revision 1)
>  // opened we make use of setTimeout() to create tool active times.
>  const TOOL_DELAY = 200;
>  
>  add_task(async function() {
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_button_paintflashing.js:68
(Diff revision 1)
>  }
>  
>  function checkResults(histIdFocus, Telemetry) {
> -  let result = Telemetry.prototype.telemetryInfo;
> -
> -  for (let [histId, value] of Object.entries(result)) {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_PAINTFLASHING_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_PAINTFLASHING_OPENED_COUNT", "", [1, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_button_paintflashing.js:69
(Diff revision 1)
>  
>  function checkResults(histIdFocus, Telemetry) {
> -  let result = Telemetry.prototype.telemetryInfo;
> -
> -  for (let [histId, value] of Object.entries(result)) {
> -    if (histId.startsWith("DEVTOOLS_INSPECTOR_") ||
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_PAINTFLASHING_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_PAINTFLASHING_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_PAINTFLASHING_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_button_responsive.js:37
(Diff revision 1)
>  
>  loader.lazyRequireGetter(this, "ResponsiveUIManager", "devtools/client/responsive.html/manager", true);
>  
>  add_task(async function() {
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_button_responsive.js:87
(Diff revision 1)
>  };
>  
>  function checkResults(histIdFocus, Telemetry) {
> -  let result = Telemetry.prototype.telemetryInfo;
> -
> -  for (let [histId, value] of Object.entries(result)) {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_RESPONSIVE_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_RESPONSIVE_OPENED_COUNT", "", [2, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_button_responsive.js:88
(Diff revision 1)
>  
>  function checkResults(histIdFocus, Telemetry) {
> -  let result = Telemetry.prototype.telemetryInfo;
> -
> -  for (let [histId, value] of Object.entries(result)) {
> -    if (histId.startsWith("DEVTOOLS_INSPECTOR_") ||
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_RESPONSIVE_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_RESPONSIVE_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_RESPONSIVE_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_button_scratchpad.js:16
(Diff revision 1)
>  // opened we make use of setTimeout() to create tool active times.
>  const TOOL_DELAY = 200;
>  
>  add_task(async function() {
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_button_scratchpad.js:100
(Diff revision 1)
>  }
>  
>  function checkResults(histIdFocus, Telemetry) {
> -  let result = Telemetry.prototype.telemetryInfo;
> -
> -  for (let [histId, value] of Object.entries(result)) {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_SCRATCHPAD_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_SCRATCHPAD_WINDOW_OPENED_COUNT", "", [4, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_misc.js:12
(Diff revision 1)
> +const TOOL_DELAY = 0;
> +
> +add_task(async function() {
> +  await addTab(TEST_URI);
> +
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_misc.js:23
(Diff revision 1)
> +});
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_TOOLBOX_OPENED_COUNT", "", [1, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_misc.js:24
(Diff revision 1)
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_TOOLBOX_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_INSPECTOR_OPENED_COUNT", "", [1, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_misc.js:25
(Diff revision 1)
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_TOOLBOX_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_INSPECTOR_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_OPENED_COUNT", "", [1, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_misc.js:26
(Diff revision 1)
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_TOOLBOX_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_INSPECTOR_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_TOOLBOX_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_misc.js:27
(Diff revision 1)
> +  // here.
> +  checkTelemetry("DEVTOOLS_TOOLBOX_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_INSPECTOR_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_TOOLBOX_TIME_ACTIVE_SECONDS", "", null, "hasentries");
> +  checkTelemetry("DEVTOOLS_INSPECTOR_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_misc.js:28
(Diff revision 1)
> +  checkTelemetry("DEVTOOLS_TOOLBOX_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_INSPECTOR_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_TOOLBOX_TIME_ACTIVE_SECONDS", "", null, "hasentries");
> +  checkTelemetry("DEVTOOLS_INSPECTOR_TIME_ACTIVE_SECONDS", "", null, "hasentries");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_misc.js:29
(Diff revision 1)
> +  checkTelemetry("DEVTOOLS_INSPECTOR_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_TOOLBOX_TIME_ACTIVE_SECONDS", "", null, "hasentries");
> +  checkTelemetry("DEVTOOLS_INSPECTOR_TIME_ACTIVE_SECONDS", "", null, "hasentries");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_TIME_ACTIVE_SECONDS", "", null, "hasentries");
> +  checkTelemetry(

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_misc.js:31
(Diff revision 1)
> +  checkTelemetry("DEVTOOLS_TOOLBOX_TIME_ACTIVE_SECONDS", "", null, "hasentries");
> +  checkTelemetry("DEVTOOLS_INSPECTOR_TIME_ACTIVE_SECONDS", "", null, "hasentries");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_TIME_ACTIVE_SECONDS", "", null, "hasentries");
> +  checkTelemetry(
> +    "DEVTOOLS_SCREEN_RESOLUTION_ENUMERATED_PER_USER", "", null, "hasentries");
> +  checkTelemetry("DEVTOOLS_TOOLBOX_HOST", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_sidebar.js:15
(Diff revision 1)
>  // opened we make use of setTimeout() to create tool active times.
>  const TOOL_DELAY = 200;
>  
>  add_task(async function() {
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_sidebar.js:57
(Diff revision 1)
>  }
>  
>  function checkResults(Telemetry) {
> -  let result = Telemetry.prototype.telemetryInfo;
> -
> -  for (let [histId, value] of Object.entries(result)) {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_INSPECTOR_OPENED_COUNT", "", [1, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_sidebar.js:58
(Diff revision 1)
>  
>  function checkResults(Telemetry) {
> -  let result = Telemetry.prototype.telemetryInfo;
> -
> -  for (let [histId, value] of Object.entries(result)) {
> -    if (histId.startsWith("DEVTOOLS_INSPECTOR_")) {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_INSPECTOR_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_OPENED_COUNT", "", [3, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_sidebar.js:59
(Diff revision 1)
>  function checkResults(Telemetry) {
> -  let result = Telemetry.prototype.telemetryInfo;
> -
> -  for (let [histId, value] of Object.entries(result)) {
> -    if (histId.startsWith("DEVTOOLS_INSPECTOR_")) {
> -      // Inspector stats are tested in browser_telemetry_toolboxtabs.js so we
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_INSPECTOR_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_OPENED_COUNT", "", [3, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_COMPUTEDVIEW_OPENED_COUNT", "", [2, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_sidebar.js:60
(Diff revision 1)
> -  let result = Telemetry.prototype.telemetryInfo;
> -
> -  for (let [histId, value] of Object.entries(result)) {
> -    if (histId.startsWith("DEVTOOLS_INSPECTOR_")) {
> -      // Inspector stats are tested in browser_telemetry_toolboxtabs.js so we
> -      // skip them here because we only open the inspector once for this test.
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_INSPECTOR_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_OPENED_COUNT", "", [3, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_COMPUTEDVIEW_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_LAYOUTVIEW_OPENED_COUNT", "", [2, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_sidebar.js:61
(Diff revision 1)
> -
> -  for (let [histId, value] of Object.entries(result)) {
> -    if (histId.startsWith("DEVTOOLS_INSPECTOR_")) {
> -      // Inspector stats are tested in browser_telemetry_toolboxtabs.js so we
> -      // skip them here because we only open the inspector once for this test.
> -      continue;
> +  // here.
> +  checkTelemetry("DEVTOOLS_INSPECTOR_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_OPENED_COUNT", "", [3, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_COMPUTEDVIEW_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_LAYOUTVIEW_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_FONTINSPECTOR_OPENED_COUNT", "", [2, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_sidebar.js:62
(Diff revision 1)
> -  for (let [histId, value] of Object.entries(result)) {
> -    if (histId.startsWith("DEVTOOLS_INSPECTOR_")) {
> -      // Inspector stats are tested in browser_telemetry_toolboxtabs.js so we
> -      // skip them here because we only open the inspector once for this test.
> -      continue;
> -    }
> +  checkTelemetry("DEVTOOLS_INSPECTOR_OPENED_COUNT", "", [1, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_OPENED_COUNT", "", [3, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_COMPUTEDVIEW_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_LAYOUTVIEW_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_FONTINSPECTOR_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_sidebar.js:63
(Diff revision 1)
> -    if (histId.startsWith("DEVTOOLS_INSPECTOR_")) {
> -      // Inspector stats are tested in browser_telemetry_toolboxtabs.js so we
> -      // skip them here because we only open the inspector once for this test.
> -      continue;
> -    }
> -
> +  checkTelemetry("DEVTOOLS_RULEVIEW_OPENED_COUNT", "", [3, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_COMPUTEDVIEW_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_LAYOUTVIEW_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_FONTINSPECTOR_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_TIME_ACTIVE_SECONDS", "", null, "hasentries");
> +  checkTelemetry("DEVTOOLS_COMPUTEDVIEW_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_sidebar.js:64
(Diff revision 1)
> -      // Inspector stats are tested in browser_telemetry_toolboxtabs.js so we
> -      // skip them here because we only open the inspector once for this test.
> -      continue;
> -    }
> -
> -    if (histId === "DEVTOOLS_TOOLBOX_OPENED_COUNT") {
> +  checkTelemetry("DEVTOOLS_COMPUTEDVIEW_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_LAYOUTVIEW_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_FONTINSPECTOR_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_TIME_ACTIVE_SECONDS", "", null, "hasentries");
> +  checkTelemetry("DEVTOOLS_COMPUTEDVIEW_TIME_ACTIVE_SECONDS", "", null, "hasentries");
> +  checkTelemetry("DEVTOOLS_LAYOUTVIEW_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_sidebar.js:65
(Diff revision 1)
> -      // skip them here because we only open the inspector once for this test.
> -      continue;
> -    }
> -
> -    if (histId === "DEVTOOLS_TOOLBOX_OPENED_COUNT") {
> -      is(value.length, 1, histId + " has only one entry");
> +  checkTelemetry("DEVTOOLS_LAYOUTVIEW_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_FONTINSPECTOR_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_RULEVIEW_TIME_ACTIVE_SECONDS", "", null, "hasentries");
> +  checkTelemetry("DEVTOOLS_COMPUTEDVIEW_TIME_ACTIVE_SECONDS", "", null, "hasentries");
> +  checkTelemetry("DEVTOOLS_LAYOUTVIEW_TIME_ACTIVE_SECONDS", "", null, "hasentries");
> +  checkTelemetry("DEVTOOLS_FONTINSPECTOR_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolbox.js:15
(Diff revision 1)
>  // opened we make use of setTimeout() to create tool active times.
>  const TOOL_DELAY = 200;
>  
>  add_task(async function() {
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolbox.js:26
(Diff revision 1)
>  });
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_TOOLBOX_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_TOOLBOX_OPENED_COUNT", "", [3, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolbox.js:27
(Diff revision 1)
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_TOOLBOX_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_TOOLBOX_OPENED_COUNT", "", [3, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_TOOLBOX_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolbox.js:28
(Diff revision 1)
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_TOOLBOX_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_TOOLBOX_OPENED_COUNT", "", [3, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_TOOLBOX_TIME_ACTIVE_SECONDS", "", null, "hasentries");
> +  checkTelemetry("DEVTOOLS_TOOLBOX_HOST", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_canvasdebugger.js:19
(Diff revision 1)
>    info("Activate the canvasdebugger");
>    let originalPref = Services.prefs.getBoolPref("devtools.canvasdebugger.enabled");
>    Services.prefs.setBoolPref("devtools.canvasdebugger.enabled", true);
>  
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_canvasdebugger.js:33
(Diff revision 1)
>  });
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_CANVASDEBUGGER")
> +  // here.
> +  checkTelemetry("DEVTOOLS_CANVASDEBUGGER_OPENED_COUNT", "", [2, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_canvasdebugger.js:34
(Diff revision 1)
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_CANVASDEBUGGER")
> +  // here.
> +  checkTelemetry("DEVTOOLS_CANVASDEBUGGER_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_CANVASDEBUGGER_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_inspector.js:15
(Diff revision 1)
>  // opened we make use of setTimeout() to create tool active times.
>  const TOOL_DELAY = 200;
>  
>  add_task(async function() {
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_inspector.js:26
(Diff revision 1)
>  });
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_INSPECTOR_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_INSPECTOR_OPENED_COUNT", "", [2, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_inspector.js:27
(Diff revision 1)
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_INSPECTOR_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_INSPECTOR_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_INSPECTOR_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_jsdebugger.js:15
(Diff revision 1)
>  // opened we make use of setTimeout() to create tool active times.
>  const TOOL_DELAY = 200;
>  
>  add_task(async function() {
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_jsdebugger.js:26
(Diff revision 1)
>  });
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_JSDEBUGGER_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_JSDEBUGGER_OPENED_COUNT", "", [2, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_jsdebugger.js:27
(Diff revision 1)
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_JSDEBUGGER_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_JSDEBUGGER_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_JSDEBUGGER_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_jsprofiler.js:15
(Diff revision 1)
>  // opened we make use of setTimeout() to create tool active times.
>  const TOOL_DELAY = 200;
>  
>  add_task(async function() {
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_jsprofiler.js:26
(Diff revision 1)
>  });
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_JSPROFILER")
> +  // here.
> +  checkTelemetry("DEVTOOLS_JSPROFILER_OPENED_COUNT", "", [2, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_jsprofiler.js:27
(Diff revision 1)
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_JSPROFILER")
> +  // here.
> +  checkTelemetry("DEVTOOLS_JSPROFILER_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_JSPROFILER_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_netmonitor.js:15
(Diff revision 1)
>  // opened we make use of setTimeout() to create tool active times.
>  const TOOL_DELAY = 200;
>  
>  add_task(async function() {
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_netmonitor.js:26
(Diff revision 1)
>  });
>  
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_NETMONITOR_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_NETMONITOR_OPENED_COUNT", "", [2, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_netmonitor.js:27
(Diff revision 1)
>  
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_NETMONITOR_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_NETMONITOR_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_NETMONITOR_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_options.js:15
(Diff revision 1)
>  // opened we make use of setTimeout() to create tool active times.
>  const TOOL_DELAY = 200;
>  
>  add_task(async function() {
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_options.js:26
(Diff revision 1)
>  });
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_OPTIONS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_OPTIONS_OPENED_COUNT", "", [2, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_options.js:27
(Diff revision 1)
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_OPTIONS_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_OPTIONS_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_OPTIONS_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_shadereditor.js:20
(Diff revision 1)
>    info("Active the sharer editor");
>    let originalPref = Services.prefs.getBoolPref(TOOL_PREF);
>    Services.prefs.setBoolPref(TOOL_PREF, true);
>  
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_shadereditor.js:34
(Diff revision 1)
>  });
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_SHADEREDITOR_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_SHADEREDITOR_OPENED_COUNT", "", [2, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_shadereditor.js:35
(Diff revision 1)
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_SHADEREDITOR_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_SHADEREDITOR_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_SHADEREDITOR_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_storage.js:15
(Diff revision 1)
>  // opened we make use of setTimeout() to create tool active times.
>  const TOOL_DELAY = 1000;
>  
>  add_task(async function() {
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_storage.js:26
(Diff revision 1)
>  });
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_STORAGE_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_STORAGE_OPENED_COUNT", "", [2, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_storage.js:27
(Diff revision 1)
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_STORAGE_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_STORAGE_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_STORAGE_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_styleeditor.js:15
(Diff revision 1)
>  // opened we make use of setTimeout() to create tool active times.
>  const TOOL_DELAY = 200;
>  
>  add_task(async function() {
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_styleeditor.js:26
(Diff revision 1)
>  });
>  
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_STYLEEDITOR_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_STYLEEDITOR_OPENED_COUNT", "", [2, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_styleeditor.js:27
(Diff revision 1)
>  
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_STYLEEDITOR_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_STYLEEDITOR_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_STYLEEDITOR_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_webaudioeditor.js:19
(Diff revision 1)
>    info("Activating the webaudioeditor");
>    let originalPref = Services.prefs.getBoolPref("devtools.webaudioeditor.enabled");
>    Services.prefs.setBoolPref("devtools.webaudioeditor.enabled", true);
>  
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_webaudioeditor.js:33
(Diff revision 1)
>  });
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_WEBAUDIOEDITOR")
> +  // here.
> +  checkTelemetry("DEVTOOLS_WEBAUDIOEDITOR_OPENED_COUNT", "", [2, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_webaudioeditor.js:34
(Diff revision 1)
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_WEBAUDIOEDITOR")
> +  // here.
> +  checkTelemetry("DEVTOOLS_WEBAUDIOEDITOR_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_WEBAUDIOEDITOR_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_webconsole.js:15
(Diff revision 1)
>  // opened we make use of setTimeout() to create tool active times.
>  const TOOL_DELAY = 200;
>  
>  add_task(async function() {
>    await addTab(TEST_URI);
> -  let Telemetry = loadTelemetryAndRecordLogs();
> +  startTelemetry();

Error: 'starttelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_webconsole.js:26
(Diff revision 1)
>  });
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_WEBCONSOLE_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_WEBCONSOLE_OPENED_COUNT", "", [2, 0, 0], "array");

Error: 'checktelemetry' is not defined. [eslint: no-undef]

::: devtools/client/shared/test/browser_telemetry_toolboxtabs_webconsole.js:27
(Diff revision 1)
> +
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("DEVTOOLS_WEBCONSOLE_")
> +  // here.
> +  checkTelemetry("DEVTOOLS_WEBCONSOLE_OPENED_COUNT", "", [2, 0, 0], "array");
> +  checkTelemetry("DEVTOOLS_WEBCONSOLE_TIME_ACTIVE_SECONDS", "", null, "hasentries");

Error: 'checktelemetry' is not defined. [eslint: no-undef]
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)

Comment 43

3 months ago
mozreview-review
Comment on attachment 8972953 [details]
Bug 1098374 - Telemetry: Stop all monkey patching in devtools telemetry tests

https://reviewboard.mozilla.org/r/241522/#review247468

Just a couple of nits, looks good otherwise!

::: devtools/client/inspector/grids/test/browser_grids_number-of-css-grids-telemetry.js:43
(Diff revision 3)
> -
> -  is(snapshot.counts[1], 1, "Got a count of 1 for 1 CSS Grid element seen.");
> -  is(snapshot.sum, 1, "Got the correct sum.");
>  });
> +
> +function checkresults() {

nit -> checkresults should be checkResults

::: devtools/client/inspector/test/browser_inspector_menu-02-copy-items.js:75
(Diff revision 3)
>  });
>  
> -function checkTelemetryResults(Telemetry) {
> -  let data = Telemetry.prototype.telemetryInfo;
> -  let results = new Map();
> -
> +function checkResults() {
> +  // For help generating these tests use generateTelemetryTests("devtools.")
> +  // here.
> +  // generateTelemetryTests("");

this seems to not be doing anything at the moment, is this intentional?

::: devtools/client/shared/test/browser_telemetry_button_paintflashing.js:65
(Diff revision 3)
>  
>      await clicked;
>    }
>  }
>  
>  function checkResults(histIdFocus, Telemetry) {

i think the args can be deleted since they are not used

::: devtools/client/shared/test/browser_telemetry_button_responsive.js:84
(Diff revision 3)
>      // See TOOL_DELAY for why we need setTimeout here
>      await DevToolsUtils.waitForTime(TOOL_DELAY);
>    }
>  };
>  
>  function checkResults(histIdFocus, Telemetry) {

same here, these arguments can be removed

::: devtools/client/shared/test/browser_telemetry_button_scratchpad.js:97
(Diff revision 3)
>        }
>      }, TOOL_DELAY);
>    });
>  }
>  
>  function checkResults(histIdFocus, Telemetry) {

same here, args can be removed

::: devtools/client/shared/test/browser_telemetry_sidebar.js:54
(Diff revision 3)
>        }
>      }, TOOL_DELAY);
>    });
>  }
>  
>  function checkResults(Telemetry) {

same here, arg can be removed

Comment 44

3 months ago
mozreview-review
Comment on attachment 8972953 [details]
Bug 1098374 - Telemetry: Stop all monkey patching in devtools telemetry tests

https://reviewboard.mozilla.org/r/241522/#review247494
Attachment #8972953 - Flags: review?(ystartsev)
(Assignee)

Comment 45

3 months ago
mozreview-review-reply
Comment on attachment 8972953 [details]
Bug 1098374 - Telemetry: Stop all monkey patching in devtools telemetry tests

https://reviewboard.mozilla.org/r/241522/#review247468

> this seems to not be doing anything at the moment, is this intentional?

Yes, I have added a comment to explain:
```js
  // NOTE: These telemetry probes have expired and will soon be changed to
  //       Telemetry event. For this reason this test is disabled in
  //       browser.ini. See https://bugzil.la/1459136.
```

> i think the args can be deleted since they are not used

The test was disabled on E10S so I didn't spend much time on it. It turns out that it was disabled due to a bug that was resolved 3 years ago! Anyhow, I made a tiny change and the test runs properly again.

Oh, I have removed the unused parameters.
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)

Comment 48

3 months ago
mozreview-review
Comment on attachment 8972953 [details]
Bug 1098374 - Telemetry: Stop all monkey patching in devtools telemetry tests

https://reviewboard.mozilla.org/r/241522/#review247498

Looks good to me! thanks!
Attachment #8972953 - Flags: review?(ystartsev) → review+

Comment 49

3 months ago
Pushed by mratcliffe@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a1abfeb6a79a
Telemetry: Stop all monkey patching in devtools telemetry tests r=yulia

Comment 50

3 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/a1abfeb6a79a
Status: NEW → RESOLVED
Last Resolved: 3 months ago
status-firefox61: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → Firefox 61

Updated

a month ago
Product: Firefox → DevTools
You need to log in before you can comment on or make changes to this bug.