701901 - Failure in testSearch::testGetMoreSearchEngines.js

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[u279076]

MODULE: tests/functional/testSearch/testGetMoreSearchEngines.js
TEST: testGetMoreEngines
FAIL: TimeoutError("Modal dialog has been found and processed")@resource://mozmill/modules/utils.js:429 waitFor([object Proxy],"Modal dialog has been found and processed",30000,100,[object Proxy])@resource://mozmill/modules/utils.js:467 
BRANCH: mozilla-release, mozilla-beta, mozilla-aurora, default

Comment 1

[Remus Pop (:RemusPop)]

The only fail closer to this is the waitForPageLoad error. I cannot reproduce the other one.
Anyway the cause for this would likely be no response from the network when we click the Add to Firefox button. If this would be the case, the solution lies in installing a search engine from localhost.
Any other info would be appreciated.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This should be tested on qa-horus if it can't be reproduced locally. Does it happen all the time? Only for the daily testrun or also when the test module or the folder gets executed? Which platforms are affected? This information I kinda miss in the initial comments all over the place now. :/

Comment 3

[Remus Pop (:RemusPop)]

Last week (11th to 18th of November) only one fail occured for this test: 
* controller.waitForPageLoad(): Timeout waiting for page loaded
which I mentioned earlier.

Comment 4

[u279076]

I'm only seeing 2 instances of this failure in the last week. Once on 10.0a2 and another time with 8.0.1. I'm thinking this is most likely due to a connectivity glitch.

Resolving WONTFIX for now. We can reopen if this starts happening much more frequently.

Comment 5

[u279076]

Reopening this due to failure in the 9.0b5 testruns today:
http://mozmill-release.brasstacks.mozilla.com/#/functional/report/e6d78e062edb343a967207c43d2b8595

This only happened once and given the history of this bug I think it's safe to say this is a very intermittent failure.

Remus, please submit a patch to disable this test immediately. It is unreliable in it's current form and will likely need to be refactored.

Comment 6

[Alex Lakatos[:AlexLakatos]]

Attachment: patch v1.0

(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #5)
> Remus, please submit a patch to disable this test immediately. It is
> unreliable in it's current form and will likely need to be refactored.
Remus had some work with other tests. Seeing as this was marked as "immediately", I took the liberty of skipping it. Hope it's ok.

Comment 7

[u279076]

Attachment: disable test (default/aurora/beta) [checked-in]

Your commit message is backwards -- please use the correct format. I've uploaded a corrected patch.

Comment 8

[u279076]

Comment on attachment 580096 [details] [diff] [review]
disable test (default/aurora/beta) [checked-in]

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/18fcf9ce8996 (default)
http://hg.mozilla.org/qa/mozmill-tests/rev/a5d218c82cab (mozilla-aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/e8048efe8d39 (mozilla-beta)

Comment 9

[Remus Pop (:RemusPop)]

No failures for this one, so here are the reports from the testruns:

Windows:
http://mozmill-crowd.blargon7.com/#/functional/report/f87375a634b1a5ba746e5f763a2b5928

Linux:
http://mozmill-crowd.blargon7.com/#/functional/report/f87375a634b1a5ba746e5f763a2b6fba

Mac:
http://mozmill-crowd.blargon7.com/#/functional/report/f87375a634b1a5ba746e5f763a2b6fcf

I will follow up with a patch to enable the test.

Comment 10

[Remus Pop (:RemusPop)]

Attachment: enable test v1 (all branches) [checked-in]

This enables the test.

Comment 11

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Remus: Could you resubmit results for Linux? The test is skipped in the report you've linked to in comment 9. Once I've seen that it's passing I can r+ this and land it.

Comment 12

[Remus Pop (:RemusPop)]

It seems I may forgot to qrefresh before the testrun. So here are the new results. testGetMoreSearchEngines is included.

Comment 13

[Remus Pop (:RemusPop)]

Linux:
http://mozmill-crowd.blargon7.com/#/functional/report/f87375a634b1a5ba746e5f763a463754

Comment 14

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Thanks Remus. Landed as:
http://hg.mozilla.org/qa/mozmill-tests/rev/01dbffb740b4

Once we have this passing we can transplant it to the remaining branches.

Comment 15

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Landed as:
http://hg.mozilla.org/qa/mozmill-tests/rev/bcd8e9c7df6b (mozilla-aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/061749338f7f (mozilla-beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/800ac3301dca (mozilla-release)
http://hg.mozilla.org/qa/mozmill-tests/rev/bf47b768f54d (mozilla-esr10)

Remus: I just noticed that you set the reviewer as dave.hunt. Please use dhunt when I am the reviewer in the future.

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Yay! So lets now get the appropriate Litmus tests updated (if necessary).

Comment 17

[Remus Pop (:RemusPop)]

Dave: wrote it down.

Here are the links from Litmus for this test:

Release:
https://litmus.mozilla.org/show_test.cgi?id=50825

Beta:
https://litmus.mozilla.org/show_test.cgi?id=55695

Aurora:
https://litmus.mozilla.org/show_test.cgi?id=15702

Comment 18

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This also needs an update of the 10.0 Litmus test.

Comment 19

[Remus Pop (:RemusPop)]

Missed it. 
Firefox ESR (10.0):
https://litmus.mozilla.org/show_test.cgi?id=40800

Comment 20

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 625048 [details] [diff] [review]
enable test v1 (all branches) [checked-in]

Missed setting the r+ when I landed this.