Closed Bug 1417052 Opened 2 years ago Closed Last year

Intermittent browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js | Expected suggestion to have title: "select a". - Got , expected select a

Categories

(WebExtensions :: General, defect)

defect
Not set

Tracking

(firefox-esr60 wontfix, firefox62 fixed, firefox63 fixed)

RESOLVED FIXED
mozilla63
Tracking Status
firefox-esr60 --- wontfix
firefox62 --- fixed
firefox63 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: robwu)

References

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell fixed])

Attachments

(2 files)

Filed by: archaeopteryx [at] coole-files.de

https://treeherder.mozilla.org/logviewer.html#?job_id=144327860&repo=autoland

https://queue.taskcluster.net/v1/task/OgBfmvWDR1mWX-EnyUWRgw/runs/0/artifacts/public/logs/live_backing.log

[task 2017-11-13T17:30:58.789Z] 17:30:58     INFO - TEST-PASS | browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js | Expected "on-input-entered-fired" to have fired with disposition: "newBackgroundTab". - 
[task 2017-11-13T17:30:58.791Z] 17:30:58     INFO - Buffered messages logged at 17:30:57
[task 2017-11-13T17:30:58.793Z] 17:30:58     INFO - TEST-PASS | browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js | Expected "on-input-changed-fired" to have fired with text: "6". - 
[task 2017-11-13T17:30:58.795Z] 17:30:58     INFO - Buffered messages logged at 17:30:58
[task 2017-11-13T17:30:58.797Z] 17:30:58     INFO - TEST-PASS | browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js | Expected item to exist - 
[task 2017-11-13T17:30:58.799Z] 17:30:58     INFO - Buffered messages finished
[task 2017-11-13T17:30:58.800Z] 17:30:58     INFO - TEST-UNEXPECTED-FAIL | browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js | Expected suggestion to have title: "select a". - Got , expected select a
[task 2017-11-13T17:30:58.801Z] 17:30:58     INFO - Stack trace:
[task 2017-11-13T17:30:58.802Z] 17:30:58     INFO - chrome://mochikit/content/browser-test.js:test_is:1269
[task 2017-11-13T17:30:58.803Z] 17:30:58     INFO - chrome://mochitests/content/browser/browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js:expectSuggestion:239
[task 2017-11-13T17:30:58.804Z] 17:30:58     INFO - chrome://mochitests/content/browser/browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js:testSuggestions:251
[task 2017-11-13T17:30:58.805Z] 17:30:58     INFO - Not taking screenshot here: see the one that was previously logged
Since 23rd January there already are 34 failures.
:andym As you are the triage owner of this component, could you please take a look?
Flags: needinfo?(amckay)
Whiteboard: [stockwell needswork:owner]
There are 32 failures in the last week.

Summary: Intermittent browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js | Expected suggestion to have title: "select a". - Got , expected select a

The number of failures seems to be dropping.
The most failures occur on linux64-stylo-disabled & Linux x64 / debug.

Here is a recent log file:
https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-central&job_id=160952427&lineNumber=14279
Flags: needinfo?(amckay) → needinfo?(ddurst)
Over the last 7 days this bugs has 35 occurrences. These happen on Linux, Linux x64, linux32-stylo-disabled, linux64-stylo-disabled, OS X 10.10, windows10-64-ccov and Windows 7.

Here is the most recent log example: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-central&job_id=162509790&lineNumber=12455

Here is a relevant part of that log: 

[task 2018-02-15T20:51:39.892Z] 20:51:39     INFO - TEST-UNEXPECTED-FAIL | browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js | Expected suggestion to have title: "select a". - Got , expected select a
[task 2018-02-15T20:51:39.893Z] 20:51:39     INFO - Stack trace:
[task 2018-02-15T20:51:39.894Z] 20:51:39     INFO - chrome://mochikit/content/browser-test.js:test_is:1271
[task 2018-02-15T20:51:39.895Z] 20:51:39     INFO - chrome://mochitests/content/browser/browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js:expectSuggestion:252
[task 2018-02-15T20:51:39.897Z] 20:51:39     INFO - chrome://mochitests/content/browser/browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js:testSuggestions:264
[task 2018-02-15T20:51:39.899Z] 20:51:39     INFO - Not taking screenshot here: see the one that was previously logged
[task 2018-02-15T20:51:39.900Z] 20:51:39     INFO - TEST-UNEXPECTED-FAIL | browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js | Expected suggestion to have displayurl: "test a". - Got mochi.test:8888/browser/browser/components/extensions/test/browser/context.html, expected test a
[task 2018-02-15T20:51:39.904Z] 20:51:39     INFO - Stack trace:
[task 2018-02-15T20:51:39.905Z] 20:51:39     INFO - chrome://mochikit/content/browser-test.js:test_is:1271
[task 2018-02-15T20:51:39.906Z] 20:51:39     INFO - chrome://mochitests/content/browser/browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js:expectSuggestion:255
[task 2018-02-15T20:51:39.906Z] 20:51:39     INFO - chrome://mochitests/content/browser/browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js:testSuggestions:264
[task 2018-02-15T20:51:39.909Z] 20:51:39     INFO - TEST-PASS | browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js | Expected item to exist - 
[task 2018-02-15T20:51:39.911Z] 20:51:39     INFO - Not taking screenshot here: see the one that was previously logged
[task 2018-02-15T20:51:39.912Z] 20:51:39     INFO - TEST-UNEXPECTED-FAIL | browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js | Expected suggestion to have title: "select b". - Got , expected select b
[task 2018-02-15T20:51:39.913Z] 20:51:39     INFO - Stack trace:
[task 2018-02-15T20:51:39.914Z] 20:51:39     INFO - chrome://mochikit/content/browser-test.js:test_is:1271
[task 2018-02-15T20:51:39.915Z] 20:51:39     INFO - chrome://mochitests/content/browser/browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js:expectSuggestion:252
[task 2018-02-15T20:51:39.916Z] 20:51:39     INFO - chrome://mochitests/content/browser/browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js:testSuggestions:264
[task 2018-02-15T20:51:39.918Z] 20:51:39     INFO - Not taking screenshot here: see the one that was previously logged
[task 2018-02-15T20:51:39.919Z] 20:51:39     INFO - TEST-UNEXPECTED-FAIL | browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js | Expected suggestion to have displayurl: "test b". - Got mochi.test:8888/browser/browser/components/extensions/test/browser/context.html?2, expected test b
[task 2018-02-15T20:51:39.920Z] 20:51:39     INFO - Stack trace:
[task 2018-02-15T20:51:39.921Z] 20:51:39     INFO - chrome://mochikit/content/browser-test.js:test_is:1271
[task 2018-02-15T20:51:39.923Z] 20:51:39     INFO - chrome://mochitests/content/browser/browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js:expectSuggestion:255
[task 2018-02-15T20:51:39.924Z] 20:51:39     INFO - chrome://mochitests/content/browser/browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js:testSuggestions:264
[task 2018-02-15T20:51:39.925Z] 20:51:39     INFO - TEST-PASS | browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js | Expected item to exist - 
[task 2018-02-15T20:51:39.926Z] 20:51:39     INFO - Not taking screenshot here: see the one that was previously logged
[task 2018-02-15T20:51:39.927Z] 20:51:39     INFO - TEST-UNEXPECTED-FAIL | browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js | Expected suggestion to have title: "select c". - Got , expected select c
[task 2018-02-15T20:51:39.928Z] 20:51:39     INFO - Stack trace:
[task 2018-02-15T20:51:39.929Z] 20:51:39     INFO - chrome://mochikit/content/browser-test.js:test_is:1271
[task 2018-02-15T20:51:39.930Z] 20:51:39     INFO - chrome://mochitests/content/browser/browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js:expectSuggestion:252
[task 2018-02-15T20:51:39.931Z] 20:51:39     INFO - chrome://mochitests/content/browser/browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js:testSuggestions:264
[task 2018-02-15T20:51:39.932Z] 20:51:39     INFO - Not taking screenshot here: see the one that was previously logged
[task 2018-02-15T20:51:39.933Z] 20:51:39     INFO - TEST-UNEXPECTED-FAIL | browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js | Expected suggestion to have displayurl: "test c". - Got mochi.test:8888/browser/browser/components/extensions/test/browser/context.html?1, expected test c
Hi! Joel, could you please review this patch. I chose to disable only on Linux and OSX for now since in the last days there aren't too many failures on windows. Also I see that the rate has gone down in the last days but so didn't the no of pushes. If you consider to hold off disabling this for a bit it's not a problem for me. Thanks.
Attachment #8953668 - Flags: review?(jmaher)
Attachment #8953668 - Flags: review?(jmaher) → review+
Pushed by csabou@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/37efe689362d
Disable browser/components/extensions/test/browser/test-oop-extensions/browser_ext_omnibox.js for frequent failures. r=jmaher
Keywords: checkin-needed
Keywords: leave-open
Whiteboard: [stockwell disable-recommended] → [stockwell disabled]
Product: Toolkit → WebExtensions
Duplicate of this bug: 1407890
I suspect that this is caused by history items appearing in the suggestions menu. That thought is supported by the failure message from bug 1407890.

Coincidentally I am able to reproduce this bug with the patch from bug 1446956,
where the only significant change is that the test opened some page in a new tab.
The failure is: https://bugzilla.mozilla.org/show_bug.cgi?id=1446956#c17
I am able to consistently reproduce the issue using the bugs from that patch and the following command:
mach test browser/components/extensions/test/browser/browser_ext_menus_activeTab.js browser/components/extensions/test/browser/browser_ext_omnibox.js

I'll try to fix this.
Assignee: nobody → rob
Status: NEW → ASSIGNED
Flags: needinfo?(ddurst)
Comment on attachment 8996668 [details]
Bug 1417052 - Use unique keyword in browser_ext_omnibox.js

https://reviewboard.mozilla.org/r/260752/#review267894

how about just clearing the browser history at the start of the test?
Comment on attachment 8996668 [details]
Bug 1417052 - Use unique keyword in browser_ext_omnibox.js

https://reviewboard.mozilla.org/r/260752/#review267894

That was my first thought. But then the same problem may occur if there is a bookmark that contains "test". So I figured that ensuring uniqueness of the keyword is the best way to fix the problem.
Comment on attachment 8996668 [details]
Bug 1417052 - Use unique keyword in browser_ext_omnibox.js

https://reviewboard.mozilla.org/r/260752/#review267914

So we could also clear bookmarks at the start of the test?  (or just assert that we're starting with a sane/expected set of bookmarks).
I guess this is fine too, I think we could probably get away with something a little shorter but *shrug*
Attachment #8996668 - Flags: review?(aswan) → review+
Pushed by rob@robwu.nl:
https://hg.mozilla.org/integration/autoland/rev/f49f080611d6
Use unique keyword in browser_ext_omnibox.js r=aswan
Blocks: 1446956
Keywords: leave-open
Status: ASSIGNED → RESOLVED
Closed: Last year
Resolution: --- → FIXED
Target Milestone: --- → mozilla63
https://hg.mozilla.org/releases/mozilla-beta/rev/afb2c70029f6
Flags: in-testsuite+
Whiteboard: [stockwell disabled] → [stockwell fixed]
You need to log in before you can comment on or make changes to this bug.