Closed Bug 1501041 Opened 6 years ago Closed 6 years ago

Perma fail browser/modules/test/browser/browser_UsageTelemetry_content.js | Should have matching amount of events. - 0 == 1 - JS frame :: chrome://mochitests/content/browser/browser/modules/test/browser/head.js :: checkEvents :: line 150

Categories

(Toolkit :: Telemetry, defect, P5)

Product:
Toolkit
Component:
Telemetry
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla65
Tracking Flags:
Tracking Status
firefox65 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: standard8)

References

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell disabled])

Attachments

(2 files)

disable_bug1501041.patch
653 bytes, patch
standard8
: review+
Details | Diff | Splinter Review
Bug 1501041 - Fix browser_UsageTelemetry_content*.js to consistently fail if there is an issue with the navigation event probe, and disable them due to current expiry. r?adw
47 bytes, text/x-phabricator-request
Details | Review 
Filed by: ccoroiu [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=207039311&repo=mozilla-central

https://queue.taskcluster.net/v1/task/E4rJ0xqBT6yvDGhgYoO6Jg/runs/0/artifacts/public/logs/live_backing.log

[task 2018-10-22T18:36:18.011Z] 18:36:18     INFO - TEST-PASS | browser/modules/test/browser/browser_UsageTelemetry_content.js | The key other-MozSearch.contextmenu must contain 1. - 1 == 1 - 
[task 2018-10-22T18:36:18.012Z] 18:36:18     INFO - Buffered messages finished
[task 2018-10-22T18:36:18.013Z] 18:36:18     INFO - TEST-UNEXPECTED-FAIL | browser/modules/test/browser/browser_UsageTelemetry_content.js | Should have matching amount of events. - 0 == 1 - JS frame :: chrome://mochitests/content/browser/browser/modules/test/browser/head.js :: checkEvents :: line 150
[task 2018-10-22T18:36:18.014Z] 18:36:18     INFO - Stack trace:
[task 2018-10-22T18:36:18.016Z] 18:36:18     INFO - chrome://mochitests/content/browser/browser/modules/test/browser/head.js:checkEvents:150
[task 2018-10-22T18:36:18.016Z] 18:36:18     INFO - chrome://mochitests/content/browser/browser/modules/test/browser/browser_UsageTelemetry_content.js:test_context_menu:84
[task 2018-10-22T18:36:18.018Z] 18:36:18     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1093
[task 2018-10-22T18:36:18.019Z] 18:36:18     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1084
[task 2018-10-22T18:36:18.020Z] 18:36:18     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:986
[task 2018-10-22T18:36:18.021Z] 18:36:18     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:795
[task 2018-10-22T18:36:18.023Z] 18:36:18     INFO - Leaving test bound test_context_menu
[task 2018-10-22T18:36:18.024Z] 18:36:18     INFO - Entering test bound test_about_newtab
[task 2018-10-22T18:36:18.055Z] 18:36:18     INFO - Trigger a simple serch, just text + enter.
[task 2018-10-22T18:36:19.039Z] 18:36:19     INFO - TEST-PASS | browser/modules/test/browser/browser_UsageTelemetry_content.js | browser.engagement.navigation.about_newtab must be recorded. - true == true - 
[task 2018-10-22T18:36:19.055Z] 18:36:19     INFO - TEST-PASS | browser/modules/test/browser/browser_UsageTelemetry_content.js | browser.engagement.navigation.about_newtab must contain the 'search_enter' key. - true == true - 
[task 2018-10-22T18:36:19.057Z] 18:36:19     INFO - TEST-PASS | browser/modules/test/browser/browser_UsageTelemetry_content.js | browser.engagement.navigation.about_newtab['search_enter'] must contain the expected value - 1 == 1 - 
[task 2018-10-22T18:36:19.058Z] 18:36:19     INFO - TEST-PASS | browser/modules/test/browser/browser_UsageTelemetry_content.js | This search must only increment one entry in the scalar. - 1 == 1 - 
[task 2018-10-22T18:36:19.060Z] 18:36:19     INFO - TEST-PASS | browser/modules/test/browser/browser_UsageTelemetry_content.js | The histogram must contain other-MozSearch.newtab. - true == true - 
[task 2018-10-22T18:36:19.061Z] 18:36:19     INFO - TEST-PASS | browser/modules/test/browser/browser_UsageTelemetry_content.js | The key other-MozSearch.newtab must contain 1. - 1 == 1 - 
[task 2018-10-22T18:36:19.078Z] 18:36:19     INFO - Not taking screenshot here: see the one that was previously logged
[task 2018-10-22T18:36:19.079Z] 18:36:19     INFO - TEST-UNEXPECTED-FAIL | browser/modules/test/browser/browser_UsageTelemetry_content.js | Should have matching amount of events. - 0 == 1 - JS frame :: chrome://mochitests/content/browser/browser/modules/test/browser/head.js :: checkEvents :: line 150
[task 2018-10-22T18:36:19.079Z] 18:36:19     INFO - Stack trace:
[task 2018-10-22T18:36:19.080Z] 18:36:19     INFO - chrome://mochitests/content/browser/browser/modules/test/browser/head.js:checkEvents:150
[task 2018-10-22T18:36:19.082Z] 18:36:19     INFO - chrome://mochitests/content/browser/browser/modules/test/browser/browser_UsageTelemetry_content.js:test_about_newtab:120
[task 2018-10-22T18:36:19.083Z] 18:36:19     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1093
[task 2018-10-22T18:36:19.084Z] 18:36:19     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1084
[task 2018-10-22T18:36:19.086Z] 18:36:19     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:986
[task 2018-10-22T18:36:19.088Z] 18:36:19     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:795
[task 2018-10-22T18:36:19.179Z] 18:36:19     INFO - Leaving test bound test_about_newtab
[task 2018-10-22T18:36:19.602Z] 18:36:19     INFO - GECKO(4184) | JavaScript error: chrome://browser/content/contentSearchUI.js, line 615: TypeError: this._engines is undefined; can't access its "find" property


This is failing also on:  browser/modules/test/browser/browser_UsageTelemetry_content_aboutHome.js
The event family `navigation.search` expires in 65. The tests are expected to fail.

ni?everyone from bug 1316281 for what to do about it.
Flags: needinfo?(past)
Flags: needinfo?(gfritzsche)
Flags: needinfo?(dzeber)
Like I said in bug 1316281 we should just make this probe never expire. Redirecting to Justin, since Mike is on PTO.
Flags: needinfo?(past) → needinfo?(dolske)
I think this is a dupe of / same as bug 1496764?
Depends on: 1496764
navigation.search probe is being handled in bug 1496764.

My best guess for this bug, is that disabling the previous two tests in the browser.ini (in bug 1496764) has somehow allowed this to pop up and become a perma fail.

The interesting thing is, Services.telemetry.canRecordExtended hasn't been set to true (I'm assuming it is generally false), so in theory this part of the test shouldn't be run. Locally, these tests don't fail for me (even after fixing an unrelated issue where telemetry wouldn't see the histogram as expired).
Flags: needinfo?(gfritzsche)
Flags: needinfo?(dzeber)
Flags: needinfo?(dolske)
Hi! I made a patch to disable this in case it's needed. This fails quite a lot and if the fix takes time i would say to disable this in the meantime. Thanks.
Assignee: nobody → csabou
Attachment #9020501 - Flags: review?(jmaher)
Assignee: csabou → nobody
Comment on attachment 9020501 [details] [diff] [review]
disable_bug1501041.patch

Review of attachment 9020501 [details] [diff] [review]:
-----------------------------------------------------------------

Yes, lets just skip this for now, we'll re-visit once the bug this depends on is fixed.
Attachment #9020501 - Flags: review?(jmaher) → review+
Keywords: checkin-needed, leave-open
Whiteboard: [stockwell disable-recommended] → [stockwell disabled]
I did a bit more digging after having some fresh thoughts. It turns out that the nightly/asan builds have the update channel set, which means that "Services.telemetry.canRecordExtended" is true on those builds.

browser_UsageTelemetry_content.js and browser_UsageTelemetry_content_aboutHome.js are not setting that value to true, whereas the currently disabled browser_UsageTelemetry_urlbar.js and browser_UsageTelemetry_searchbar.js do set the value to true.

Hence, these two will only fail on "official" builds currently, which is wrong.

Setting canRecordExtended for these two tests causes them to fail permanently, which is due to bug 1496764, so we need to disable them as well for the time being.
Flags: needinfo?(standard8)
Assignee: nobody → standard8
Status: NEW → ASSIGNED
Keywords: leave-open
Pushed by mbanner@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f8ac0af5449a
Fix browser_UsageTelemetry_content*.js to consistently fail if there is an issue with the navigation event probe, and disable them due to current expiry. r=adw
https://hg.mozilla.org/mozilla-central/rev/f8ac0af5449a
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
status-firefox65: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla65
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: