Intermittent testHomeBanner | banner text - correct text appeared in the home banner

RESOLVED FIXED in Firefox 29

Status

()

Firefox for Android
General
RESOLVED FIXED
4 years ago
3 years ago

People

(Reporter: emorley, Assigned: Margaret)

Tracking

({intermittent-failure})

Trunk
Firefox 29
ARM
Android
intermittent-failure
Points:
---

Firefox Tracking Flags

(firefox27 unaffected, firefox28 unaffected, firefox29 fixed, firefox-esr24 unaffected)

Details

Attachments

(2 attachments)

(Reporter)

Description

4 years ago
Android 4.0 Panda mozilla-inbound opt test robocop-2 on 2014-01-29 08:03:54 PST for push 523a1736de8f

slave: panda-0805

https://tbpl.mozilla.org/php/getParsedLog.php?id=33747048&tree=Mozilla-Inbound

{
08:25:11     INFO -  1 INFO TEST-START | testHomeBanner
08:25:11     INFO -  2 INFO TEST-PASS | testHomeBanner | Given message occurred for registered event: {"type":"Gecko:Ready"} - Gecko:Ready should equal Gecko:Ready
08:25:11     INFO -  EventExpecter: no longer listening for Gecko:Ready
08:25:11     INFO -  3 INFO TEST-PASS | testHomeBanner | waiting for urlbar text to gain focus - urlbar text gained focus
08:25:11     INFO -  4 INFO TEST-PASS | testHomeBanner | URL typed properly - chrome://roboextender/content/robocop_home_banner.html#addMessage should equal chrome://roboextender/content/robocop_home_banner.html#addMessage
08:25:11     INFO -  5 INFO TEST-PASS | testHomeBanner | Given message occurred for registered event: {"errorType":"","bgColor":"transparent","type":"DOMContentLoaded","tabID":0} - DOMContentLoaded should equal DOMContentLoaded
08:25:11     INFO -  EventExpecter: no longer listening for DOMContentLoaded
08:25:11     INFO -  6 INFO TEST-PASS | testHomeBanner | Given message occurred for registered event: {"type":"TestHomeBanner:MessageAdded"} - TestHomeBanner:MessageAdded should equal TestHomeBanner:MessageAdded
08:25:11     INFO -  7 INFO TEST-PASS | testHomeBanner | Given message occurred for registered event: {"type":"TestHomeBanner:MessageShown"} - TestHomeBanner:MessageShown should equal TestHomeBanner:MessageShown
08:25:11     INFO -  8 INFO TEST-PASS | testHomeBanner | waiting for urlbar text to gain focus - urlbar text gained focus
08:25:11     INFO -  9 INFO TEST-PASS | testHomeBanner | URL typed properly - about:home should equal about:home
08:25:11     INFO -  10 INFO TEST-PASS | testHomeBanner | Given message occurred for registered event: {"errorType":"","bgColor":"transparent","type":"DOMContentLoaded","tabID":0} - DOMContentLoaded should equal DOMContentLoaded
08:25:11     INFO -  EventExpecter: no longer listening for DOMContentLoaded
08:25:11     INFO -  waitForText timeout on The quick brown fox jumps over the lazy dog.
08:25:11     INFO -  11 INFO TEST-UNEXPECTED-FAIL | testHomeBanner | banner text - correct text appeared in the home banner
08:25:11     INFO -  Exception caught during test!
08:25:11     INFO -  junit.framework.AssertionFailedError: 11 INFO TEST-UNEXPECTED-FAIL | testHomeBanner | banner text - correct text appeared in the home banner
08:25:11     INFO -  	at junit.framework.Assert.fail(Assert.java:47)
08:25:11     INFO -  	at org.mozilla.gecko.FennecMochitestAssert._logMochitestResult(FennecMochitestAssert.java:110)
08:25:11     INFO -  	at org.mozilla.gecko.FennecMochitestAssert.ok(FennecMochitestAssert.java:139)
08:25:11     INFO -  	at org.mozilla.gecko.tests.testHomeBanner.testHomeBanner(testHomeBanner.java:30)
08:25:11     INFO -  	at java.lang.reflect.Method.invokeNative(Native Method)
08:25:11     INFO -  	at java.lang.reflect.Method.invoke(Method.java:511)
08:25:11     INFO -  	at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
08:25:11     INFO -  	at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
08:25:11     INFO -  	at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
08:25:11     INFO -  	at org.mozilla.gecko.tests.BaseTest.runTest(BaseTest.java:150)
08:25:11     INFO -  	at junit.framework.TestCase.runBare(TestCase.java:127)
08:25:11     INFO -  	at junit.framework.TestResult$1.protect(TestResult.java:106)
08:25:11     INFO -  	at junit.framework.TestResult.runProtected(TestResult.java:124)
08:25:11     INFO -  	at junit.framework.TestResult.run(TestResult.java:109)
08:25:11     INFO -  	at junit.framework.TestCase.run(TestCase.java:118)
08:25:11     INFO -  	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)
08:25:11     INFO -  	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)
08:25:11     INFO -  	at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:545)
08:25:11     INFO -  	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1551)
08:25:11     INFO -  12 INFO TEST-UNEXPECTED-FAIL | testHomeBanner | Exception caught - junit.framework.AssertionFailedError: 11 INFO TEST-UNEXPECTED-FAIL | testHomeBanner | banner text - correct text appeared in the home banner
08:25:11     INFO -  13 INFO TEST-END | testHomeBanner | finished in 32629ms
}
(Reporter)

Comment 1

4 years ago
https://tbpl.mozilla.org/php/getParsedLog.php?id=33744720&tree=B2g-Inbound
(Reporter)

Comment 2

4 years ago
https://tbpl.mozilla.org/php/getParsedLog.php?id=33748265&tree=Fx-Team
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Assignee)

Comment 8

4 years ago
I suspect this may be caused by the fact that we recently pushed some test snippets to production, since I'm seeing snippets appear when running tests locally right now.

If that is the case, I'm not sure the best way to fix this. I don't want to totally disable snippets for testing, because we did find some performance regressions related to them. However, maybe we should change the update URL pref to point to some local test server rather than the snippets server.

I wonder if this is only intermittently green because the snippets fail to update sometimes.
Assignee: nobody → margaret.leibovic
(Assignee)

Comment 9

4 years ago
Created attachment 8367741 [details] [diff] [review]
disable snippets for mochitests

I think we should try to just disable snippets for mochitests. I ran this locally, and it looks like it fixed the problem.
Attachment #8367741 - Flags: review?(gbrown)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Assignee)

Comment 25

4 years ago
My first try push accidentally included a patch that broke other things. Here's a clean try push:
https://tbpl.mozilla.org/?tree=Try&rev=b74251a1a7f4

I really think that the snippets must have caused this. I also think it would be good to disable them for mochitests in general, so that we avoid making network requests during test runs.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment on attachment 8367741 [details] [diff] [review]
disable snippets for mochitests

Review of attachment 8367741 [details] [diff] [review]:
-----------------------------------------------------------------

Disabling snippets definitely seems like a good idea.

This change only disables them for Robocop tests, while your comment says "for mochitests", and it might be best to disable for all test suites. I think you want to update testing/profiles/prefs_general.js instead...but I'm not 100% sure that will be picked up by Robocop.
Attachment #8367741 - Flags: review?(gbrown) → review+
Comment hidden (Treeherder Robot)
(Assignee)

Comment 35

4 years ago
(In reply to Geoff Brown [:gbrown] from comment #33)
> Comment on attachment 8367741 [details] [diff] [review]
> disable snippets for mochitests
> 
> Review of attachment 8367741 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Disabling snippets definitely seems like a good idea.
> 
> This change only disables them for Robocop tests, while your comment says
> "for mochitests", and it might be best to disable for all test suites. I
> think you want to update testing/profiles/prefs_general.js instead...but I'm
> not 100% sure that will be picked up by Robocop.

Oh, I misinterpreted what I was doing then :)

I can also make a try run flipping the pref in prefs_general.js.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Assignee)

Comment 41

4 years ago
Try run looking green so I pushed to fx-team:
https://hg.mozilla.org/integration/fx-team/rev/335c1799e1c0

Here's a try push that disables snippets in prefs_general.js:
https://tbpl.mozilla.org/?tree=Try&rev=145a0754c23c
(Assignee)

Comment 42

4 years ago
Created attachment 8368062 [details] [diff] [review]
disable snippets in default testing profile

Here's the patch I pushed to try.

Although I now realize I should probably do a try push that runs more than just robocop tests, since this affects more than that.
Attachment #8368062 - Flags: review?(gbrown)
Comment hidden (Treeherder Robot)
(Assignee)

Comment 44

4 years ago
Try run that runs all unit tests:
https://tbpl.mozilla.org/?tree=Try&rev=5be485ff07fd
Comment hidden (Treeherder Robot)

Updated

4 years ago
Attachment #8368062 - Flags: review?(gbrown) → review+
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
https://hg.mozilla.org/mozilla-central/rev/335c1799e1c0
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 29
Comment hidden (Treeherder Robot)
(Assignee)

Comment 74

4 years ago
Try run was green, so I pushed the patch to disable snippets more globally:
https://hg.mozilla.org/integration/fx-team/rev/cbcf7d2e9cb3
status-firefox27: --- → unaffected
status-firefox28: --- → unaffected
status-firefox29: --- → fixed
status-firefox-esr24: --- → unaffected
https://hg.mozilla.org/mozilla-central/rev/cbcf7d2e9cb3
(Reporter)

Updated

3 years ago
Depends on: 1036028
You need to log in before you can comment on or make changes to this bug.