1481344 - Perma-failing TV browser/components/search/test/browser_searchEngine_behaviors.js | "https://www.amazon.com/exec/obidos/external-search/?field-keywords=&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search"

Status: RESOLVED FIXED

(Firefox :: Search, defect, P1)

Description

[Treeherder Bug Filer]

Filed by: apavel [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=192319058&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/UmKccqivSEain-GHqVnHXw/runs/0/artifacts/public/logs/live_backing.log

18:44:38     INFO - GECKO(6616) | [Parent 6616, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x804B000A: file z:/build/build/src/obj-firefox/dist/include\nsIURIMutator.h, line 450
18:44:38     INFO - GECKO(6616) | [Parent 6616, Main Thread] WARNING: NS_ENSURE_SUCCESS(mStatus, *this) failed with result 0x804B000A: file z:/build/build/src/obj-firefox/dist/include\nsIURIMutator.h, line 605
18:44:38     INFO - GECKO(6616) | [Parent 6616, Main Thread] WARNING: NS_ENSURE_SUCCESS(mStatus, mStatus) failed with result 0x804B000A: file z:/build/build/src/obj-firefox/dist/include\nsIURIMutator.h, line 644
18:44:38     INFO - TEST-INFO | started process screenshot
18:44:38     INFO - TEST-INFO | screenshot: exit 0
18:44:38     INFO - Buffered messages logged at 18:44:37
18:44:38     INFO - Entering test bound test_setup
18:44:38     INFO - Leaving test bound test_setup
18:44:38     INFO - Entering test bound 
18:44:38     INFO - TEST-PASS | browser/components/search/test/browser_searchEngine_behaviors.js | Amazon.com is installed - {"QueryInterface":"function QueryInterface() {\n    [native code]\n}","getSubmission":"function getSubmission() {\n    [native code]\n}","addParam":"function addParam() {\n    [native code]\n}","supportsResponseType":"function supportsResponseType() {\n    [native code]\n}","getIconURLBySize":"function getIconURLBySize() {\n    [native code]\n}","getIcons":"function getIcons() {\n    [native code]\n}","speculativeConnect":"function speculativeConnect() {\n    [native code]\n}","alias":null,"description":"Amazon.com Search","hidden":false,"iconURI":{"spec":"resource://search-plugins/images/amazon.ico","scheme":"resource","prePath":"resource://search-plugins","userPass":"","username":"","password":"","hostPort":"search-plugins","host":"search-plugins","port":-1,"pathQueryRef":"/images/amazon.ico","equals":"function equals() {\n    [native code]\n}","schemeIs":"function schemeIs() {\n    [native code]\n}","resolve":"function resolve() {\n    [native code]\n}","asciiSpec":"resource://search-plugins/images/amazon.ico","asciiHostPort":"search-plugins","asciiHost":"search-plugins","ref":"","equalsExceptRef":"function equalsExceptRef() {\n    [native code]\n}","specIgnoringRef":"resource://search-plugins/images/amazon.ico","hasRef":false,"filePath":"/images/amazon.ico","query":"","displayHost":"search-plugins","displayHostPort":"search-plugins","displaySpec":"resource://search-plugins/images/amazon.ico","displayPrePath":"resource://search-plugins","mutate":"function mutate() {\n    [native code]\n}"},"name":"Amazon.com","searchForm":"https://www.amazon.com/exec/obidos/external-search/?field-keywords=&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search","identifier":"amazondotcom","getResultDomain":"function getResultDomain() {\n    [native code]\n}"} == true - 
18:44:38     INFO - TEST-PASS | browser/components/search/test/browser_searchEngine_behaviors.js | Check search URL for 'foo' - "https://www.amazon.com/exec/obidos/external-search/?field-keywords=foo&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" == "https://www.amazon.com/exec/obidos/external-search/?field-keywords=foo&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" - 
18:44:38     INFO - Buffered messages logged at 18:44:38
18:44:38     INFO - Running: context menu search
18:44:38     INFO - TEST-PASS | browser/components/search/test/browser_searchEngine_behaviors.js | "https://www.amazon.com/exec/obidos/external-search/?field-keywords=foo&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" == "https://www.amazon.com/exec/obidos/external-search/?field-keywords=foo&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" - 
18:44:38     INFO - Running: keyword search
18:44:38     INFO - TEST-PASS | browser/components/search/test/browser_searchEngine_behaviors.js | "https://www.amazon.com/exec/obidos/external-search/?field-keywords=foo&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" == "https://www.amazon.com/exec/obidos/external-search/?field-keywords=foo&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" - 
18:44:38     INFO - Running: keyword search with alias
18:44:38     INFO - TEST-PASS | browser/components/search/test/browser_searchEngine_behaviors.js | "https://www.amazon.com/exec/obidos/external-search/?field-keywords=foo&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" == "https://www.amazon.com/exec/obidos/external-search/?field-keywords=foo&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" - 
18:44:38     INFO - Running: search bar search
18:44:38     INFO - Buffered messages finished
18:44:38     INFO - TEST-UNEXPECTED-FAIL | browser/components/search/test/browser_searchEngine_behaviors.js | "https://www.amazon.com/exec/obidos/external-search/?field-keywords=&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" == "https://www.amazon.com/exec/obidos/external-search/?field-keywords=foo&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" - JS frame :: chrome://mochitests/content/browser/browser/components/search/test/browser_searchEngine_behaviors.js :: testSearchEngine :: line 184
18:44:38     INFO - Stack trace:
18:44:38     INFO - chrome://mochitests/content/browser/browser/components/search/test/browser_searchEngine_behaviors.js:testSearchEngine:184
18:44:38     INFO - chrome://mochitests/content/browser/browser/components/search/test/browser_searchEngine_behaviors.js:null:90
18:44:38     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1106
18:44:38     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1097
18:44:38     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:994
18:44:38     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:795
18:44:38     INFO - Running: new tab search
18:44:38     INFO - GECKO(6616) | ++DOMWINDOW == 9 (00000196643AA000) [pid = 3868] [serial = 9] [outer = 0000019663AE5400]
18:44:38     INFO - GECKO(6616) | --DOCSHELL 0000022860C48800 == 1 [pid = 6548] [id = {cdcb5c5c-d5a9-47e8-98d3-2b0f67b1ad9f}]
18:44:39     INFO - TEST-PASS | browser/components/search/test/browser_searchEngine_behaviors.js | "https://www.amazon.com/exec/obidos/external-search/?field-keywords=foo&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" == "https://www.amazon.com/exec/obidos/external-search/?field-keywords=foo&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" - 
18:44:39     INFO - Console message: [JavaScript Error: "onboarding-session-end should pass valid session_key" {file: "resource://onboarding/modules/OnboardingTelemetry.jsm" line: 378}]

Comment 1

[Intermittent Failures Robot]

50 failures in 350 pushes (0.143 failures/push) were associated with this bug yesterday.

Repository breakdown:
* autoland: 11
* mozilla-inbound: 22
* mozilla-central: 17

Platform breakdown:
* windows10-64-nightly: 1
* linux64: 10
* windows10-64-msvc: 1
* windows7-32: 12
* windows10-64: 15
* linux64-nightly: 1
* windows7-32-nightly: 1
* macosx64-nightly: 1
* windows7-32-msvc: 1
* osx-10-10: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1481344&startday=2018-08-06&endday=2018-08-06&tree=all

Comment 2

[Intermittent Failures Robot]

20 failures in 463 pushes (0.043 failures/push) were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 20

Platform breakdown:
* osx-10-10: 3
* windows10-64: 6
* windows7-32: 6
* linux64: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1481344&startday=2018-08-08&endday=2018-08-08&tree=all

Comment 3

[Intermittent Failures Robot]

23 failures in 549 pushes (0.042 failures/push) were associated with this bug yesterday.

Repository breakdown:
* mozilla-central: 8
* autoland: 4
* mozilla-inbound: 11

Platform breakdown:
* windows10-64-nightly: 1
* windows7-32-msvc: 1
* linux64: 2
* osx-10-10: 2
* windows10-64-msvc: 2
* windows10-64: 9
* windows7-32: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1481344&startday=2018-08-09&endday=2018-08-09&tree=all

Comment 4

[Tiberius Oros[:tiberius_oros]]

There have been 92 failures in the last 7 days.

Failures per platform and build type:
- windows10-64: 30
- windows7-32 / all builds except asan: 24
- linux64 / all builds except asan: 17
- osx-10-10 /  opt & debug: 12
- windows10-64-msvc / opt & debug: 3
- windows7-32-msvc: 2
- windows10-64-nightly: 2
- macosx64-nightly: 1
- linux64-nightly: 1


Recent log file:
https://treeherder.mozilla.org/logviewer.html#?job_id=193010286&repo=mozilla-central&lineNumber=1363

snippet with the failure:

11:59:40     INFO - TEST-PASS | browser/components/search/test/browser_searchEngine_behaviors.js | "https://www.amazon.com/exec/obidos/external-search/?field-keywords=foo&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" == "https://www.amazon.com/exec/obidos/external-search/?field-keywords=foo&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" - 
11:59:40     INFO - Running: keyword search with alias
11:59:40     INFO - TEST-PASS | browser/components/search/test/browser_searchEngine_behaviors.js | "https://www.amazon.com/exec/obidos/external-search/?field-keywords=foo&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" == "https://www.amazon.com/exec/obidos/external-search/?field-keywords=foo&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" - 
11:59:40     INFO - Running: search bar search
11:59:40     INFO - Buffered messages finished
11:59:40     INFO - TEST-UNEXPECTED-FAIL | browser/components/search/test/browser_searchEngine_behaviors.js | "https://www.amazon.com/exec/obidos/external-search/?field-keywords=&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" == "https://www.amazon.com/exec/obidos/external-search/?field-keywords=foo&ie=UTF-8&mode=blended&tag=mozilla-20&sourceid=Mozilla-search" - JS frame :: chrome://mochitests/content/browser/browser/components/search/test/browser_searchEngine_behaviors.js :: testSearchEngine :: line 184
11:59:40     INFO - Stack trace:
11:59:40     INFO - chrome://mochitests/content/browser/browser/components/search/test/browser_searchEngine_behaviors.js:testSearchEngine:184
11:59:40     INFO - chrome://mochitests/content/browser/browser/components/search/test/browser_searchEngine_behaviors.js:null:90
11:59:40     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1103
11:59:40     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1094

Comment 5

[Intermittent Failures Robot]

92 failures in 2561 pushes (0.036 failures/push) were associated with this bug in the last 7 days.

This is the #11 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. ** 

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. ** 

Repository breakdown:
* mozilla-inbound: 52
* autoland: 15
* mozilla-central: 25

Platform breakdown:
* windows10-64: 30
* windows10-64-msvc: 3
* linux64-nightly: 1
* windows7-32: 24
* osx-10-10: 12
* windows7-32-msvc: 2
* macosx64-nightly: 1
* windows7-32-nightly: 1
* windows10-64-nightly: 2
* linux64: 16

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1481344&startday=2018-08-05&endday=2018-08-11&tree=all

Comment 6

[Intermittent Failures Robot]

92 failures in 2579 pushes (0.036 failures/push) were associated with this bug in the last 7 days.

This is the #13 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. ** 

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 15
* mozilla-inbound: 52
* mozilla-central: 25

Platform breakdown:
* windows10-64-nightly: 2
* windows7-32-msvc: 2
* windows10-64: 30
* linux64: 16
* macosx64-nightly: 1
* osx-10-10: 12
* windows10-64-msvc: 3
* windows7-32-nightly: 1
* windows7-32: 24
* linux64-nightly: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1481344&startday=2018-08-06&endday=2018-08-12&tree=all

Comment 7

[Mark Banner (:standard8)]

I'm not quite sure of the exact cause of this, but from the output of the log when the test is run for a second time in the same instance, it looks like there's no value in the search bar.

From test inspection, it looks like we're queuing four registerCleanupFunction calls to reset the search bar value. I suspect that there could be some async delays in executing those, and hence we're somehow still clearing the bar when it gets to the next test run.

Reorganising the test it runs just fine in --verify, so I'll put up a patch.

Comment 8

[Mark Banner (:standard8)]

Attachment: Bug 1481344 - Fix perma-fail in browser_searchEngine_behaviors.js, improve how the search bar is cleared.

MozReview-Commit-ID: IgT9aU41BHW

Comment 9

[Mike Kaply [:mkaply]]

Comment on attachment 9002401 [details]
Bug 1481344 - Fix perma-fail in browser_searchEngine_behaviors.js, improve how the search bar is cleared.

Mike Kaply [:mkaply] has approved the revision.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e20c185652fe
Fix perma-fail in browser_searchEngine_behaviors.js, improve how the search bar is cleared. r=mkaply

Comment 12

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/e20c185652fe