Last Comment Bug 701901 - Failure in testSearch::testGetMoreSearchEngines.js
: Failure in testSearch::testGetMoreSearchEngines.js
Status: RESOLVED FIXED
[mozmill-test-failure][qa-]
:
Product: Mozilla QA
Classification: Other
Component: Mozmill Tests (show other bugs)
: unspecified
: All All
: -- normal (vote)
: ---
Assigned To: Remus Pop (:RemusPop)
:
Mentors:
http://mozmill-release.brasstacks.moz...
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-11-11 15:17 PST by Anthony Hughes (:ashughes) [GFX][QA][Mentor]
Modified: 2012-08-14 14:30 PDT (History)
4 users (show)
remus.pop: in‑litmus+
See Also:
QA Whiteboard:
Iteration: ---
Points: ---
fixed
fixed
fixed
fixed
fixed


Attachments
patch v1.0 (1.43 KB, patch)
2011-12-08 08:33 PST, Alex Lakatos[:AlexLakatos]
anthony.s.hughes: review+
Details | Diff | Splinter Review
disable test (default/aurora/beta) [checked-in] (1.45 KB, patch)
2011-12-08 10:35 PST, Anthony Hughes (:ashughes) [GFX][QA][Mentor]
anthony.s.hughes: review+
Details | Diff | Splinter Review
enable test v1 (all branches) [checked-in] (1.40 KB, patch)
2012-05-18 02:54 PDT, Remus Pop (:RemusPop)
dave.hunt: review+
Details | Diff | Splinter Review

Description Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2011-11-11 15:17:51 PST
MODULE: tests/functional/testSearch/testGetMoreSearchEngines.js
TEST: testGetMoreEngines
FAIL: TimeoutError("Modal dialog has been found and processed")@resource://mozmill/modules/utils.js:429 waitFor([object Proxy],"Modal dialog has been found and processed",30000,100,[object Proxy])@resource://mozmill/modules/utils.js:467 
BRANCH: mozilla-release, mozilla-beta, mozilla-aurora, default
Comment 1 Remus Pop (:RemusPop) 2011-11-16 04:44:43 PST
The only fail closer to this is the waitForPageLoad error. I cannot reproduce the other one.
Anyway the cause for this would likely be no response from the network when we click the Add to Firefox button. If this would be the case, the solution lies in installing a search engine from localhost.
Any other info would be appreciated.
Comment 2 Henrik Skupin (:whimboo) 2011-11-17 03:53:02 PST
This should be tested on qa-horus if it can't be reproduced locally. Does it happen all the time? Only for the daily testrun or also when the test module or the folder gets executed? Which platforms are affected? This information I kinda miss in the initial comments all over the place now. :/
Comment 3 Remus Pop (:RemusPop) 2011-11-18 01:00:27 PST
Last week (11th to 18th of November) only one fail occured for this test: 
* controller.waitForPageLoad(): Timeout waiting for page loaded
which I mentioned earlier.
Comment 4 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2011-11-21 18:45:43 PST
I'm only seeing 2 instances of this failure in the last week. Once on 10.0a2 and another time with 8.0.1. I'm thinking this is most likely due to a connectivity glitch.

Resolving WONTFIX for now. We can reopen if this starts happening much more frequently.
Comment 5 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2011-12-07 17:27:50 PST
Reopening this due to failure in the 9.0b5 testruns today:
http://mozmill-release.brasstacks.mozilla.com/#/functional/report/e6d78e062edb343a967207c43d2b8595

This only happened once and given the history of this bug I think it's safe to say this is a very intermittent failure.

Remus, please submit a patch to disable this test immediately. It is unreliable in it's current form and will likely need to be refactored.
Comment 6 Alex Lakatos[:AlexLakatos] 2011-12-08 08:33:31 PST
Created attachment 580054 [details] [diff] [review]
patch v1.0

(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #5)
> Remus, please submit a patch to disable this test immediately. It is
> unreliable in it's current form and will likely need to be refactored.
Remus had some work with other tests. Seeing as this was marked as "immediately", I took the liberty of skipping it. Hope it's ok.
Comment 7 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2011-12-08 10:35:20 PST
Created attachment 580096 [details] [diff] [review]
disable test (default/aurora/beta) [checked-in]

Your commit message is backwards -- please use the correct format. I've uploaded a corrected patch.
Comment 8 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2011-12-08 10:38:17 PST
Comment on attachment 580096 [details] [diff] [review]
disable test (default/aurora/beta) [checked-in]

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/18fcf9ce8996 (default)
http://hg.mozilla.org/qa/mozmill-tests/rev/a5d218c82cab (mozilla-aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/e8048efe8d39 (mozilla-beta)
Comment 9 Remus Pop (:RemusPop) 2012-05-18 02:51:14 PDT
No failures for this one, so here are the reports from the testruns:

Windows:
http://mozmill-crowd.blargon7.com/#/functional/report/f87375a634b1a5ba746e5f763a2b5928

Linux:
http://mozmill-crowd.blargon7.com/#/functional/report/f87375a634b1a5ba746e5f763a2b6fba

Mac:
http://mozmill-crowd.blargon7.com/#/functional/report/f87375a634b1a5ba746e5f763a2b6fcf

I will follow up with a patch to enable the test.
Comment 10 Remus Pop (:RemusPop) 2012-05-18 02:54:18 PDT
Created attachment 625048 [details] [diff] [review]
enable test v1 (all branches) [checked-in]

This enables the test.
Comment 11 Dave Hunt (:davehunt) 2012-05-18 07:00:57 PDT
Remus: Could you resubmit results for Linux? The test is skipped in the report you've linked to in comment 9. Once I've seen that it's passing I can r+ this and land it.
Comment 12 Remus Pop (:RemusPop) 2012-05-21 06:08:31 PDT
It seems I may forgot to qrefresh before the testrun. So here are the new results. testGetMoreSearchEngines is included.
Comment 14 Dave Hunt (:davehunt) 2012-05-21 06:26:31 PDT
Thanks Remus. Landed as:
http://hg.mozilla.org/qa/mozmill-tests/rev/01dbffb740b4

Once we have this passing we can transplant it to the remaining branches.
Comment 15 Dave Hunt (:davehunt) 2012-05-22 12:41:24 PDT
Landed as:
http://hg.mozilla.org/qa/mozmill-tests/rev/bcd8e9c7df6b (mozilla-aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/061749338f7f (mozilla-beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/800ac3301dca (mozilla-release)
http://hg.mozilla.org/qa/mozmill-tests/rev/bf47b768f54d (mozilla-esr10)

Remus: I just noticed that you set the reviewer as dave.hunt. Please use dhunt when I am the reviewer in the future.
Comment 16 Henrik Skupin (:whimboo) 2012-05-22 15:10:01 PDT
Yay! So lets now get the appropriate Litmus tests updated (if necessary).
Comment 17 Remus Pop (:RemusPop) 2012-05-23 00:01:43 PDT
Dave: wrote it down.

Here are the links from Litmus for this test:

Release:
https://litmus.mozilla.org/show_test.cgi?id=50825

Beta:
https://litmus.mozilla.org/show_test.cgi?id=55695

Aurora:
https://litmus.mozilla.org/show_test.cgi?id=15702
Comment 18 Henrik Skupin (:whimboo) 2012-05-23 00:44:58 PDT
This also needs an update of the 10.0 Litmus test.
Comment 19 Remus Pop (:RemusPop) 2012-05-23 00:51:21 PDT
Missed it. 
Firefox ESR (10.0):
https://litmus.mozilla.org/show_test.cgi?id=40800
Comment 20 Dave Hunt (:davehunt) 2012-05-28 17:36:09 PDT
Comment on attachment 625048 [details] [diff] [review]
enable test v1 (all branches) [checked-in]

Missed setting the r+ when I landed this.

Note You need to log in before you can comment on or make changes to this bug.