613664 - Test failure 'subject.isEngineNameInContextMenu == true' in testSearchSelection.js | testSearchSelectionViaContextMenu

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

MODULES: firefox/testSearch/testSearchSelection.js
TEST: testSearchSelectionViaContextMenu
ERROR: controller.assertJS: Failed for 'subject.isEngineNameInContextMenu == true'
BRANCH: default

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Exception message:

assert((function () {return eval(expression);}),"controller.assertJS: Failed for 'subject.isEngineNameInContextMenu == true'")@resource://mozmill/modules/utils.js:395 ("subject.isEngineNameInContextMenu == true",[object Object])@resource://mozmill/modules/controller.js:897 ([object Object],"Wikipedia (en)",true)@resource://mozmill/modules/frame.js -> file:///tmp/tmpz7G3eo.mozmill-tests/firefox/testSearch/testSearchSelection.js:117 ()@resource://mozmill/modules/frame.js -> file:///tmp/tmpz7G3eo.mozmill-tests/firefox/testSearch/testSearchSelection.js:82 ((function () {var engines = searchBar.visibleEngines;var engineName = engines[engines.length - 1].name;searchBar.selectedEngine = engineName;controller.open(LOCAL_TEST_PAGE);controller.waitForPageLoad();var textElem = new elementslib.ID(controller.tabs.activeTab, "mission_statement");startSearch(textElem, engineName, true);startSearch(textElem, engineName, false);}))@resource://mozmill/modules/frame.js:530 ([object Object])@resource://mozmill/modules/frame.js:598 ([object Object])@resource://mozmill/modules/frame.js:641 ("/tmp/tmpz7G3eo.mozmill-tests/firefox")@resource://mozmill/modules/frame.js:471 ("/tmp/tmpz7G3eo.mozmill-tests/firefox")@resource://mozmill/modules/frame.js:647 ((function (dir, invokedFromIDE) {var runner = new Runner(new Collector, invokedFromIDE);runner.runTestDirectory(dir);runner.end();return true;}),[object Proxy])@resource://jsbridge/modules/server.js:164 ("ce632ac2-f3f7-11df-b9dc-002500f4ec3d",(function (dir, invokedFromIDE) {var runner = new Runner(new Collector, invokedFromIDE);runner.runTestDirectory(dir);runner.end();return true;}),[object Proxy])@resource://jsbridge/modules/server.js:168 @:0 @resource://jsbridge/modules/server.js:249

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Unable to reproduce this on Mac OS X 10.6.5. It's been failing the last few days on Mac OS X 10.6.4 http://mozmill-release.brasstacks.mozilla.com/#/general/failure?branch=4.0&platform=Mac&from=2010-11-12&to=2010-11-19&test=firefox%2FtestSearch%2FtestSearchSelection.js&func=testSearchSelectionViaContextMenu

The failure appears to be that the selected search engine is not displayed in the context-menu for highlighted text.

Whilst investigating I have discovered the following strange behaviour:

If I let Mozmill dump the search engine providers it lists:
* Google
* Yahoo
* Bing
* Amazon.com
* Wikipedia (en)
* eBay

If I check myself in the drop down list, Wikipedia and eBay are the other way around. Also, if I check whilst the test is running then the dumped values have the same difference.

This may be relevant as the test selects the last engine, which is eBay when the test runs locally, but reported as Wikipedia in the exception.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Do we know which engine fails here? I think we should give you the access to qa-mozmill and qa-horus for investigation. Should be easier to reproduce it on the box itself, if you can't do it locally.

But not sure why it happens. I could imagine the problem is the "(en)" in Wikipedia, if it's the last engine.

Comment 4

[Dave Hunt [:davehunt] [he/him] ⌚BST]

The engine that's failing on qa-horus is 'Wikipedia (en)' however when run locally it's using 'eBay' and passing.

I've also tried to replicate this on a local Mac OS X 10.6.4 without success. Access to qa-mozmill and qa-horus would be really useful, what do I need to get this?

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

So my question is, why is Horus using Wikipedia for an en-US build? Why the order is different. Dave, please modify your local test to also use wikipedia, just by using the 2nd last entry. I expect that it will fail at the final test with the same error.

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Passes locally when forced to use Wikipedia (en) on both 10.6.5 and 10.6.4

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Then you should get access to qa-horus. Talk with Geo about it.

Comment 8

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Finally was able to replicate and narrow this down. Raised regression bug 614152.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Good find Dave! Which tests have to be executed to see this behavior?

Comment 10

[Dave Hunt [:davehunt] [he/him] ⌚BST]

You need to run testPrivateBrowsing/testDisabledPermissions.js followed by testSearch/testSearchSelection.js

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The failure has disappeared after the check-in of the patch on bug 614152.