1640289 - Perma browser_UsageTelemetry_urlbar.js | unexpected counts should be zero for FX_URLBAR_SELECTED_RESULT_INDEX at index 1 - 1 == 0 when Gecko 78 merges to Beta on 2020-06-01

Status: VERIFIED FIXED

(Firefox :: Address Bar, defect, P1)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

central-as-beta simulation: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&selectedTaskRun=DdiAK-jSTsifIfoTaVGj0g-0&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=0799f0ae22aa66449b8845124b82a77f8ef2061f&searchStr=browser-chrome

Log: https://treeherder.mozilla.org/logviewer.html#?job_id=303382464&repo=try

17:07:40 INFO - TEST-UNEXPECTED-FAIL | browser/modules/test/browser/browser_UsageTelemetry_urlbar.js | unexpected counts should be zero for FX_URLBAR_SELECTED_RESULT_INDEX at index 1 - 1 == 0 - JS frame :: resource://testing-common/TelemetryTestUtils.jsm :: assertHistogram :: line 297
4290 17:07:40 INFO - TEST-UNEXPECTED-FAIL | browser/modules/test/browser/browser_UsageTelemetry_urlbar.js | expected counts should match for FX_URLBAR_SELECTED_RESULT_INDEX at index 2 - 0 == 1 - JS frame :: resource://testing-common/TelemetryTestUtils.jsm :: assertHistogram :: line 291
4301 17:07:40 INFO - TEST-UNEXPECTED-FAIL | browser/modules/test/browser/browser_UsageTelemetry_urlbar.js | unexpected counts should be zero for FX_URLBAR_SELECTED_RESULT_INDEX_BY_TYPE_2 at index 1 - 1 == 0 - JS frame :: resource://testing-common/TelemetryTestUtils.jsm :: assertKeyedHistogramValue :: line 360
4306 17:07:40 INFO - TEST-UNEXPECTED-FAIL | browser/modules/test/browser/browser_UsageTelemetry_urlbar.js | expected counts should match for FX_URLBAR_SELECTED_RESULT_INDEX_BY_TYPE_2 at index 2 - 0 == 1 - JS frame :: resource://testing-common/TelemetryTestUtils.jsm :: assertKeyedHistogramValue :: line 354

Comment 1

[Drew Willcoxon :adw]

Hmm, this didn't happen on the simulation I ran in bug 1398416 comment 47. Maybe it's intermittent? Shouldn't be though... I'll take a look.

Comment 2

[Drew Willcoxon :adw]

I think it's because the private search result is shown in Nightly but not otherwise. The test is failing because it expects the private search result to be at index 1 and the form history result to be at index 2, but the form history is actually at index 1. My simulation didn't catch it because it was still a Nightly build, so it still had the private search result. The test just needs to get the right index. There's another part of the test that clicks a remote suggestion, but it's not affected because the search happens to match a couple of history results, so the private search result isn't shown even on Nightly.

Comment 3

[Drew Willcoxon :adw]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b00d802535c1f25a6826460955a89529d0ffefc5

Comment 4

[Drew Willcoxon :adw]

Attachment: Bug 1640289 - Fix failure on late beta and release by getting the correct index of the form history result.

The private search result is shown in Nightly but not otherwise. The test is
failing because it expects the private search result to be at index 1 and the
form history result to be at index 2, but the form history is actually at index

1. The test just needs to get the right index. There's another part of the test
   that clicks a remote suggestion, but it's not affected because the search
   happens to match a couple of history results, so the private search result isn't
   shown even on Nightly.

Comment 5

[Drew Willcoxon :adw]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=772319cbd460e08e056bebd1e87f8255c10205a9

Comment 6

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=testfailed%2Cbusted%2Cexception%2Crunnable&revision=0a2a522a5a5d64fa1bd762b819bcf5aa86d12484

Comment 7

[Drew Willcoxon :adw]

Looks like it fixes it, thanks Sebastian.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e70cfe6e8da9
Fix failure on late beta and release by getting the correct index of the form history result. r=mak

Comment 9

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/e70cfe6e8da9

Comment 10

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Verified fixed in yesterday's central-as-beta simulation: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=4071359fca65d9841bc50c2481ec9296463fb8a3