1120461 - Test failures "Suite progress indicator is green - got 'false'", "No tests have failed - '1' should equal '0'" and "No error messages present - '1' should equal '0'" in testAssertAlertPresentPasses.js and testVerifyAlertPresentPasses.js

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[Mihaela Velimiroviciu (:mihaelav)]

Modules:    testAssertAlertPresentCommandPasses
            testVerifyAlertNotPresentCommandPasses 
            testVerifyAlertPresentCommandPasses

Tests:      * /[e-mail]/tests/testCommands/testAlertPresent/testAssertAlertPresentPasses.js 
            * /[e-mail]/tests/testCommands/testAlertPresent/testVerifyAlertNotPresentPasses.js
            * /[e-mail]/tests/testCommands/testAlertPresent/testVerifyAlertPresentPasses.js

Failure:    Suite progress indicator is green - got 'false'
            No tests have failed - '1' should equal '0'
            No error messages present - '1' should equal '0'

Branches:  default
Platforms: Ubuntu 

Reports: 
* http://mozmill-daily.blargon7.com/#/addons/report/c0d2f578c4e131660e84b4e0e20e90ac
* http://mozmill-daily.blargon7.com/#/addons/report/c0d2f578c4e131660e84b4e0e20d4fdc
* http://mozmill-daily.blargon7.com/#/addons/report/c0d2f578c4e131660e84b4e0e20d4f20

Comment 1

[Teodor Druta]

This is the the Selenium log with a failure.

>    [info] Playing test case Untitled
>    [info] Executing: |assertAlertNotPresent | | |
>    [info] Executing: |assertAlertNotPresentecho | final command | |
>    [error] Unknown command: 'assertAlertNotPresentecho'
>    [info] Test case failed 

We actually fail to wait for the tree node child row to change, when clicking it.
http://hg.mozilla.org/qa/mozmill-tests/file/00f0b67f6e3d/firefox/tests/addons/ide%40seleniumhq.org/lib/selenium.js#l118

Comment 2

[Teodor Druta]

Attachment: fixaddons.patch

I think that we should wait for the right row to be selected in clickTreeCell() function of the widgets.js lib.

Just a small note regarding as setting this Bug as blocker to Bug 1120462, at first I thought that it should be the same related issue, now I'm not so sure, I'll continue investigating that failure more.

Comment 3

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 8551381 [details] [diff] [review]
fixaddons.patch

Review of attachment 8551381 [details] [diff] [review]:
-----------------------------------------------------------------

So you reproduced this and with this fix it's not failing anymore?

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Comment on attachment 8551381 [details] [diff] [review]
fixaddons.patch

Review of attachment 8551381 [details] [diff] [review]:
-----------------------------------------------------------------

r=me with the nit addressed.

::: lib/ui/widgets.js
@@ +47,1 @@
>    aController.sleep(0);

By this change I assume we can get rid of at least the final sleep(0) call, now that waitFor() is used. Not sure about the one in line 40, which is also out of scope for this bug. So please remove that sleep call here.

Comment 5

[Teodor Druta]

My patch indeed lowers the failure rate, but it seems that it doesn't completely fix the issue, I'll investigate this more.

Comment 6

[Daniela Domnici]

Failed again on Nightly with Ubuntu 14.04.

Reports:
http://mozmill-daily.blargon7.com/#/addons/report/52e1dd05f3d97380e39f88e69c4cfb97

Comment 7

[Teodor Druta]

Attachment: addons_final_fix.patch

What I was missing in my fix, was cleaning the fields in addCommand(*) method between tests.

Comment 8

[Daniela Domnici]

Failed again on Nightly

Reports:
http://mozmill-daily.blargon7.com/#/addons/report/6b1e6b9bf618fb20da2bf87274637aa7
http://mozmill-daily.blargon7.com/#/addons/report/6b1e6b9bf618fb20da2bf8727465d607
http://mozmill-daily.blargon7.com/#/addons/report/6b1e6b9bf618fb20da2bf8727465e76e

Comment 9

[Andreea Matei [:AndreeaMatei]]

https://hg.mozilla.org/qa/mozmill-tests/rev/cee05b43eee0 (default)

Comment 10

[Andreea Matei [:AndreeaMatei]]

No failures today with nightly, yey! No need to backport as we run this on nightly only.