Failure in testSearch::testGetMoreSearchEngines.js

RESOLVED FIXED

Status

Mozilla QA
Mozmill Tests
RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: ashughes, Assigned: RemusPop)

Tracking

unspecified
Bug Flags:
in-litmus +

Firefox Tracking Flags

(firefox12 fixed, firefox13 fixed, firefox14 fixed, firefox15 fixed, firefox-esr10 fixed)

Details

(Whiteboard: [mozmill-test-failure][qa-], URL)

Attachments

(2 attachments, 1 obsolete attachment)

MODULE: tests/functional/testSearch/testGetMoreSearchEngines.js
TEST: testGetMoreEngines
FAIL: TimeoutError("Modal dialog has been found and processed")@resource://mozmill/modules/utils.js:429 waitFor([object Proxy],"Modal dialog has been found and processed",30000,100,[object Proxy])@resource://mozmill/modules/utils.js:467 
BRANCH: mozilla-release, mozilla-beta, mozilla-aurora, default
(Assignee)

Updated

6 years ago
Assignee: nobody → remus.pop
(Assignee)

Comment 1

6 years ago
The only fail closer to this is the waitForPageLoad error. I cannot reproduce the other one.
Anyway the cause for this would likely be no response from the network when we click the Add to Firefox button. If this would be the case, the solution lies in installing a search engine from localhost.
Any other info would be appreciated.
This should be tested on qa-horus if it can't be reproduced locally. Does it happen all the time? Only for the daily testrun or also when the test module or the folder gets executed? Which platforms are affected? This information I kinda miss in the initial comments all over the place now. :/
(Assignee)

Comment 3

6 years ago
Last week (11th to 18th of November) only one fail occured for this test: 
* controller.waitForPageLoad(): Timeout waiting for page loaded
which I mentioned earlier.
I'm only seeing 2 instances of this failure in the last week. Once on 10.0a2 and another time with 8.0.1. I'm thinking this is most likely due to a connectivity glitch.

Resolving WONTFIX for now. We can reopen if this starts happening much more frequently.
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → WONTFIX
Reopening this due to failure in the 9.0b5 testruns today:
http://mozmill-release.brasstacks.mozilla.com/#/functional/report/e6d78e062edb343a967207c43d2b8595

This only happened once and given the history of this bug I think it's safe to say this is a very intermittent failure.

Remus, please submit a patch to disable this test immediately. It is unreliable in it's current form and will likely need to be refactored.
Status: RESOLVED → REOPENED
Resolution: WONTFIX → ---
Created attachment 580054 [details] [diff] [review]
patch v1.0

(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #5)
> Remus, please submit a patch to disable this test immediately. It is
> unreliable in it's current form and will likely need to be refactored.
Remus had some work with other tests. Seeing as this was marked as "immediately", I took the liberty of skipping it. Hope it's ok.
Assignee: remus.pop → alex.lakatos
Attachment #580054 - Flags: review?(anthony.s.hughes)
Attachment #580054 - Flags: review?(anthony.s.hughes) → review+
Created attachment 580096 [details] [diff] [review]
disable test (default/aurora/beta) [checked-in]

Your commit message is backwards -- please use the correct format. I've uploaded a corrected patch.
Attachment #580054 - Attachment is obsolete: true
Attachment #580096 - Flags: review+
Status: REOPENED → ASSIGNED
Summary: Failure in testGetMoreSearchEngines.js: Modal dialog has been found and processed → Failure in testSearch::testGetMoreSearchEngines.js
Whiteboard: [mozmill-test-failure] → [mozmill-test-failure][mozmill-test-skipped]
Comment on attachment 580096 [details] [diff] [review]
disable test (default/aurora/beta) [checked-in]

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/18fcf9ce8996 (default)
http://hg.mozilla.org/qa/mozmill-tests/rev/a5d218c82cab (mozilla-aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/e8048efe8d39 (mozilla-beta)
Attachment #580096 - Attachment description: disable test (default/aurora/beta) → disable test (default/aurora/beta) [checked-in]
(Assignee)

Comment 9

5 years ago
No failures for this one, so here are the reports from the testruns:

Windows:
http://mozmill-crowd.blargon7.com/#/functional/report/f87375a634b1a5ba746e5f763a2b5928

Linux:
http://mozmill-crowd.blargon7.com/#/functional/report/f87375a634b1a5ba746e5f763a2b6fba

Mac:
http://mozmill-crowd.blargon7.com/#/functional/report/f87375a634b1a5ba746e5f763a2b6fcf

I will follow up with a patch to enable the test.
(Assignee)

Comment 10

5 years ago
Created attachment 625048 [details] [diff] [review]
enable test v1 (all branches) [checked-in]

This enables the test.
Attachment #625048 - Flags: review?(dave.hunt)
Remus: Could you resubmit results for Linux? The test is skipped in the report you've linked to in comment 9. Once I've seen that it's passing I can r+ this and land it.
Assignee: alex.lakatos → remus.pop
(Assignee)

Comment 12

5 years ago
It seems I may forgot to qrefresh before the testrun. So here are the new results. testGetMoreSearchEngines is included.
(Assignee)

Comment 13

5 years ago
Linux:
http://mozmill-crowd.blargon7.com/#/functional/report/f87375a634b1a5ba746e5f763a463754
Thanks Remus. Landed as:
http://hg.mozilla.org/qa/mozmill-tests/rev/01dbffb740b4

Once we have this passing we can transplant it to the remaining branches.
status-firefox-esr10: --- → affected
status-firefox12: --- → affected
status-firefox13: --- → affected
status-firefox14: --- → affected
status-firefox15: --- → fixed
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago5 years ago
Resolution: --- → FIXED
Landed as:
http://hg.mozilla.org/qa/mozmill-tests/rev/bcd8e9c7df6b (mozilla-aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/061749338f7f (mozilla-beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/800ac3301dca (mozilla-release)
http://hg.mozilla.org/qa/mozmill-tests/rev/bf47b768f54d (mozilla-esr10)

Remus: I just noticed that you set the reviewer as dave.hunt. Please use dhunt when I am the reviewer in the future.
status-firefox-esr10: affected → fixed
status-firefox12: affected → fixed
status-firefox13: affected → fixed
status-firefox14: affected → fixed
Yay! So lets now get the appropriate Litmus tests updated (if necessary).
Flags: in-litmus?(remus.pop)
Whiteboard: [mozmill-test-failure][mozmill-test-skipped] → [mozmill-test-failure]
(Assignee)

Comment 17

5 years ago
Dave: wrote it down.

Here are the links from Litmus for this test:

Release:
https://litmus.mozilla.org/show_test.cgi?id=50825

Beta:
https://litmus.mozilla.org/show_test.cgi?id=55695

Aurora:
https://litmus.mozilla.org/show_test.cgi?id=15702
Flags: in-litmus?(remus.pop) → in-litmus+
This also needs an update of the 10.0 Litmus test.
Flags: in-litmus+ → in-litmus?(remus.pop)
(Assignee)

Comment 19

5 years ago
Missed it. 
Firefox ESR (10.0):
https://litmus.mozilla.org/show_test.cgi?id=40800
(Assignee)

Updated

5 years ago
Flags: in-litmus?(remus.pop) → in-litmus+
Comment on attachment 625048 [details] [diff] [review]
enable test v1 (all branches) [checked-in]

Missed setting the r+ when I landed this.
Attachment #625048 - Flags: review?(dave.hunt) → review+
(Assignee)

Updated

5 years ago
Attachment #625048 - Attachment description: enable test v1 (all branches) → enable test v1 (all branches) [checked-in]
Whiteboard: [mozmill-test-failure] → [mozmill-test-failure][qa-]
You need to log in before you can comment on or make changes to this bug.