Closed Bug 1247468 Opened 6 years ago Closed 6 years ago

Intermittent testBookmarklets | Alert was shown for bookmarklet - got false, expected true

Categories

(Firefox for Android Graveyard :: General, defect)

defect
Not set
normal

Tracking

(firefox47 fixed, firefox48 fixed)

RESOLVED FIXED
Firefox 48
Tracking Status
firefox47 --- fixed
firefox48 --- fixed

People

(Reporter: KWierso, Assigned: gbrown)

References

Details

(Keywords: intermittent-failure)

Attachments

(1 file)

Blocks: 1244966
Any ideas? Maybe we just need to increase this timeout value?
Flags: needinfo?(mark.finkle)
 21:53:33     INFO -  TEST-START | testBookmarklets
 21:53:33     INFO -  TEST-PASS | testBookmarklets | Robocop tests need the test device screen to be powered on. -
 21:53:33     INFO -  TEST-PASS | testBookmarklets | Given message occurred for registered event: {"type":"Gecko:Ready"} - Gecko:Ready should equal Gecko:Ready
 21:53:33     INFO -  EventExpecter: no longer listening for Gecko:Ready
 21:53:33     INFO -  TEST-PASS | testBookmarklets | Given message occurred for registered event: {"errorType":"","bgColor":"transparent","type":"DOMContentLoaded","tabID":0} - DOMContentLoaded should equal DOMContentLoaded
 21:53:33     INFO -  EventExpecter: no longer listening for DOMContentLoaded
 21:53:33     INFO -  TEST-PASS | testBookmarklets | waiting for urlbar text to gain focus - urlbar text gained focus
 21:53:33     INFO -  TEST-PASS | testBookmarklets | URL typed properly - javascript:alert(12 + 10) should equal javascript:alert(12 + 10)
 21:53:33     INFO -  waitForCondition timeout after 3000 ms.
 21:53:33     INFO -  TEST-PASS | testBookmarklets | Alert was not shown for user-entered bookmarklet - false should equal false
 21:53:33     INFO -  waitForCondition timeout after 3000 ms.
 21:53:33  WARNING -  TEST-UNEXPECTED-FAIL | testBookmarklets | Alert was shown for bookmarklet - got false, expected true
 21:53:33     INFO -  0 ERROR Exception caught during test! - junit.framework.AssertionFailedError: TEST-UNEXPECTED-FAIL | testBookmarklets | Alert was shown for bookmarklet - got false, expected true
 21:53:33     INFO -  	at junit.framework.Assert.fail(Assert.java:50)
 21:53:33     INFO -  	at org.mozilla.gecko.FennecMochitestAssert._logMochitestResult(FennecMochitestAssert.java:128)
 21:53:33     INFO -  	at org.mozilla.gecko.FennecMochitestAssert.ok(FennecMochitestAssert.java:150)
 21:53:33     INFO -  	at org.mozilla.gecko.FennecMochitestAssert.is(FennecMochitestAssert.java:156)
 21:53:33     INFO -  	at org.mozilla.gecko.tests.testBookmarklets.testBookmarklets(testBookmarklets.java:44)
 21:53:33     INFO -  	at java.lang.reflect.Method.invokeNative(Native Method)
 21:53:33     INFO -  	at java.lang.reflect.Method.invoke(Method.java:525)
 21:53:33     INFO -  	at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
 21:53:33     INFO -  	at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
 21:53:33     INFO -  	at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
 21:53:33     INFO -  	at org.mozilla.gecko.tests.BaseRobocopTest.runTest(BaseRobocopTest.java:188)
 21:53:33     INFO -  	at junit.framework.TestCase.runBare(TestCase.java:134)


Screenshots show an alert saying "The page at ... says: 22"

eg. http://mozilla-releng-blobs.s3.amazonaws.com/blobs/mozilla-inbound/sha512/2316483a3a621e14f8ca84fae3121b2621d38f6635670dc1807fbcf5ae4b4f93a38a6bf3d52cb457af77abf5f99eff9d805a1c36be15c2891b5ab294f15c82ad


Successful runs of this test do not normally have the 2nd timeout:

04:16:59     INFO -  SimpleTest START
04:16:59     INFO -  TEST-START | testBookmarklets
04:16:59     INFO -  TEST-PASS | testBookmarklets | Robocop tests need the test device screen to be powered on. -
04:16:59     INFO -  TEST-PASS | testBookmarklets | Given message occurred for registered event: {"type":"Gecko:Ready"} - Gecko:Ready should equal Gecko:Ready
04:16:59     INFO -  EventExpecter: no longer listening for Gecko:Ready
04:16:59     INFO -  TEST-PASS | testBookmarklets | Given message occurred for registered event: {"errorType":"","bgColor":"transparent","type":"DOMContentLoaded","tabID":0} - DOMContentLoaded should equal DOMContentLoaded
04:16:59     INFO -  EventExpecter: no longer listening for DOMContentLoaded
04:16:59     INFO -  TEST-PASS | testBookmarklets | waiting for urlbar text to gain focus - urlbar text gained focus
04:16:59     INFO -  TEST-PASS | testBookmarklets | URL typed properly - javascript:alert(12 + 10) should equal javascript:alert(12 + 10)
04:16:59     INFO -  waitForCondition timeout after 3000 ms.
04:16:59     INFO -  TEST-PASS | testBookmarklets | Alert was not shown for user-entered bookmarklet - false should equal false
04:16:59     INFO -  TEST-PASS | testBookmarklets | Alert was shown for bookmarklet - true should equal true
04:16:59     INFO -  TEST-OK | testBookmarklets | took 129521ms
(I see now - the first 3000 ms timeout is expected; the second is not.)

Comparing timestamps in the logcats for successful runs show that it often takes 5 to 7 seconds to go from "Alert was not shown..." to "Alert was shown...". There's more than just the 3 second wait in that interval, but I think it's another indication that increasing the 2nd timeout might be effective.
Attached patch increase timeoutSplinter Review
Assignee: nobody → gbrown
Attachment #8739837 - Flags: review?(margaret.leibovic)
Attachment #8739837 - Flags: review?(margaret.leibovic) → review+
https://hg.mozilla.org/mozilla-central/rev/e1adfa4a39c0
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 48
It looks like increasing the timeout was effective.
Flags: needinfo?(mark.finkle)
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.