1528255 - Intermittent failure: test-verify browser/modules/test/browser/browser_UsageTelemetry_urlbar_remotetab.js | Should not have recorded browser.engagement.navigation.urlbar - 1 == 0

Status: RESOLVED FIXED

(Firefox :: Address Bar, defect, P3)

Description

Filed by: mbanner [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=228628971&repo=try

https://queue.taskcluster.net/v1/task/JBjxsZp6STeYmZdlhNVY2w/runs/0/artifacts/public/logs/live_backing.log

For some reason, bug 1515083 has introduced an intermittent failure on test-verify Linux 64 bits.

It appears that the existing address bar attempts to record (additional?) search telemetry in some circumstances when searches shouldn't be being touched. Unfortunately I've not figured out why they might be happening yet.

As a result, I've disabled the test just for test-verify linux-64 bits.

We should try to figure out what's happening and if it affects telemetry or not.

We also need to figure out if it affects QuantumBar or not.

[task 2019-02-15T12:11:40.679Z] 12:11:40     INFO - Entering test bound test_remotetab
[task 2019-02-15T12:11:40.681Z] 12:11:40     INFO - Buffered messages finished
[task 2019-02-15T12:11:40.683Z] 12:11:40     INFO - TEST-UNEXPECTED-FAIL | browser/modules/test/browser/browser_UsageTelemetry_urlbar_remotetab.js | Should not have recorded browser.engagement.navigation.urlbar - 1 == 0 - JS frame :: chrome://mochitests/content/browser/browser/modules/test/browser/browser_UsageTelemetry_urlbar_remotetab.js :: assertSearchTelemetryEmpty :: line 22
[task 2019-02-15T12:11:40.684Z] 12:11:40     INFO - Stack trace:
[task 2019-02-15T12:11:40.686Z] 12:11:40     INFO - chrome://mochitests/content/browser/browser/modules/test/browser/browser_UsageTelemetry_urlbar_remotetab.js:assertSearchTelemetryEmpty:22
[task 2019-02-15T12:11:40.689Z] 12:11:40     INFO - chrome://mochitests/content/browser/browser/modules/test/browser/browser_UsageTelemetry_urlbar_remotetab.js:test_remotetab:156
[task 2019-02-15T12:11:40.690Z] 12:11:40     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1106
[task 2019-02-15T12:11:40.691Z] 12:11:40     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1134
[task 2019-02-15T12:11:40.692Z] 12:11:40     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:990
[task 2019-02-15T12:11:40.694Z] 12:11:40     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:803
[task 2019-02-15T12:11:40.696Z] 12:11:40     INFO - TEST-PASS | browser/modules/test/browser/browser_UsageTelemetry_urlbar_remotetab.js | The histogram must not contain other-MozSearch.urlbar. - true == true - 
[task 2019-02-15T12:11:40.698Z] 12:11:40     INFO - TEST-PASS | browser/modules/test/browser/browser_UsageTelemetry_urlbar_remotetab.js | The histogram must not contain other-MozSearch.alias. - true == true - 
[task 2019-02-15T12:11:40.700Z] 12:11:40     INFO - TEST-PASS | browser/modules/test/browser/browser_UsageTelemetry_urlbar_remotetab.js | Should not have recorded any events - 0 == 0 - 
[task 2019-02-15T12:11:40.701Z] 12:11:40     INFO - TEST-PASS | browser/modules/test/browser/browser_UsageTelemetry_urlbar_remotetab.js | unexpected counts should be zero for the histogram index 0 - 0 == 0 - 
[task 2019-02-15T12:11:40.705Z] 12:11:40     INFO - TEST-PASS | browser/modules/test/browser/browser_UsageTelemetry_urlbar_remotetab.js | expected counts should match for the histogram index 1 - 1 == 1 - 

Comment 1

[Intermittent Failures Robot]

2 failures in 2832 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• try: 1

Platform breakdown:

• linux64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1528255&startday=2019-02-11&endday=2019-02-17&tree=all

Comment 2

[Mark Banner (:standard8)]

Finally figured this out. The test was breaking because for the scalar I was checking that no scalars had been reported, rather than just the browser.engagement.navigation.urlbar scalar.

On the try servers, we'd sometimes see telemetry.accumulate_clamped_values being recorded, so the test would fail.

Comment 3

[Mark Banner (:standard8)]

Attachment: Bug 1528255 - In browser_UsageTelemetry*, correct the checks for the urlbar navigation scalar to only look at the specific scalar to avoid intermittents. r?mak

Comment 4

[Mark Banner (:standard8)]

Try push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=577e172211025b33676485738ada113b492a7e41

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e08d102e6d59
In browser_UsageTelemetry*, correct the checks for the urlbar navigation scalar to only look at the specific scalar to avoid intermittents. r=mak

Comment 6

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/e08d102e6d59