Intermittent testBookmarklets,testHistory,testTabHistory,testBookmarkKeyword | Page title is correct - got http://mochi.test:8888/tests/robocop/robocop_blank_01.html, expected Browser Blank Page 01

RESOLVED INACTIVE

Status

()

Firefox for Android
Awesomescreen
P3
normal
RESOLVED INACTIVE
5 years ago
4 hours ago

People

(Reporter: RyanVM, Assigned: liuche)

Tracking

({intermittent-failure})

Trunk
Firefox 27
ARM
Android
intermittent-failure
Points:
---
Bug Flags:
in-testsuite -

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [test disabled][leave open])

Attachments

(2 attachments)

(Reporter)

Description

5 years ago
https://tbpl.mozilla.org/php/getParsedLog.php?id=27705779&tree=Fx-Team

Android 2.2 Tegra fx-team opt test robocop-1 on 2013-09-11 08:14:15 PDT for push a1a846de1a8a
slave: tegra-094

0 INFO SimpleTest START
1 INFO TEST-START | testBookmarklets
EventExpecter: no longer listening for Gecko:Ready
2 INFO TEST-PASS | testBookmarklets | waiting for urlbar text to gain focus - urlbar text gained focus
3 INFO TEST-PASS | testBookmarklets | URL typed properly - http://mochi.test:8888/tests/robocop/robocop_blank_01.html should equal http://mochi.test:8888/tests/robocop/robocop_blank_01.html
EventExpecter: no longer listening for DOMContentLoaded
waitForCondition timeout after 3000 ms.
4 INFO TEST-UNEXPECTED-FAIL | testBookmarklets | Page title is correct - got http://mochi.test:8888/tests/robocop/robocop_blank_01.html, expected Browser Blank Page 01
Exception caught during test!
junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testBookmarklets | Page title is correct - got http://mochi.test:8888/tests/robocop/robocop_blank_01.html, expected Browser Blank Page 01
	at junit.framework.Assert.fail(Assert.java:47)
	at org.mozilla.fennec.FennecMochitestAssert._logMochitestResult(FennecMochitestAssert.java:111)
	at org.mozilla.fennec.FennecMochitestAssert.ok(FennecMochitestAssert.java:140)
	at org.mozilla.fennec.FennecMochitestAssert.is(FennecMochitestAssert.java:146)
	at org.mozilla.fennec.tests.BaseTest.verifyPageTitle(BaseTest.java:528)
	at org.mozilla.fennec.tests.testBookmarklets.testBookmarklets(testBookmarklets.java:26)
	at java.lang.reflect.Method.invokeNative(Native Method)
	at java.lang.reflect.Method.invoke(Method.java:521)
	at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204)
	at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194)
	at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186)
	at org.mozilla.fennec.tests.BaseTest.runTest(BaseTest.java:162)
	at junit.framework.TestCase.runBare(TestCase.java:127)
	at junit.framework.TestResult$1.protect(TestResult.java:106)
	at junit.framework.TestResult.runProtected(TestResult.java:124)
	at junit.framework.TestResult.run(TestResult.java:109)
	at junit.framework.TestCase.run(TestCase.java:118)
	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)
	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)
	at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:520)
	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447)
5 INFO TEST-UNEXPECTED-FAIL | testBookmarklets | Exception caught - junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testBookmarklets | Page title is correct - got http://mochi.test:8888/tests/robocop/robocop_blank_01.html, expected Browser Blank Page 01
6 INFO TEST-END | testBookmarklets | finished in 36127ms
7 INFO TEST-START | Shutdown
8 INFO Passed: 2
9 INFO Failed: 2
10 INFO Todo: 0
11 INFO SimpleTest FINISHED
(Reporter)

Comment 1

5 years ago
https://tbpl.mozilla.org/php/getParsedLog.php?id=27705526&tree=Fx-Team
Summary: Intermittent testBookmarklets | Page title is correct - got http://mochi.test:8888/tests/robocop/robocop_blank_01.html, expected Browser Blank Page 01 → Intermittent testBookmarklets,testHistory | Page title is correct - got http://mochi.test:8888/tests/robocop/robocop_blank_01.html, expected Browser Blank Page 01
(Reporter)

Comment 2

5 years ago
https://tbpl.mozilla.org/php/getParsedLog.php?id=27708995&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=27712012&tree=Mozilla-Inbound
Summary: Intermittent testBookmarklets,testHistory | Page title is correct - got http://mochi.test:8888/tests/robocop/robocop_blank_01.html, expected Browser Blank Page 01 → Intermittent testBookmarklets,testHistory,testTabHistory | Page title is correct - got http://mochi.test:8888/tests/robocop/robocop_blank_01.html, expected Browser Blank Page 01
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Reporter)

Comment 5

5 years ago
https://tbpl.mozilla.org/php/getParsedLog.php?id=27724058&tree=Fx-Team

Mark, this seems to be a very frequent, recent regression. Can someone on your team please look at this ASAP?
Flags: needinfo?(mark.finkle)
Summary: Intermittent testBookmarklets,testHistory,testTabHistory | Page title is correct - got http://mochi.test:8888/tests/robocop/robocop_blank_01.html, expected Browser Blank Page 01 → Intermittent testBookmarklets,testHistory,testTabHistory,testBookmarkKeyword | Page title is correct - got http://mochi.test:8888/tests/robocop/robocop_blank_01.html, expected Browser Blank Page 01

Comment 6

5 years ago
Looks like a possible problem with verifyPageTitle. Regression caused by bug 912590?
Flags: needinfo?(liuche)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(In reply to :Margaret Leibovic from comment #6)
> Looks like a possible problem with verifyPageTitle. Regression caused by bug
> 912590?

That does seem likely, considering that the url is verified as being typed correctly, but the urlbar text is still the url, instead of the page title, and this happens much more frequently on the (slower) 2.2 devices.

I'm looking over those patches again; having some trouble verifying locally.
Assignee: nobody → liuche
Flags: needinfo?(liuche)
Looking at the logs a little more carefully, this is definitely what's happening.

waitForCondition timeout after 3000 ms.
6 INFO TEST-UNEXPECTED-FAIL | testBookmarkKeyword | Page title is correct - got http://mochi.test:8888/tests/robocop/robocop_blank_01.html, expected Browser Blank Page 01
Exception caught during test!
junit.framework.AssertionFailedError: 6 INFO TEST-UNEXPECTED-FAIL | testBookmarkKeyword | Page title is correct - got http://mochi.test:8888/tests/robocop/robocop_blank_01.html, expected Browser Blank Page 01
	at junit.framework.Assert.fail(Assert.java:47)

This is failing because we're timing out waiting for the url to switch to the title (on load completion). I'm not entirely sure why we aren't timing out waiting for DOMContentLoaded instead though...

It's possible that the Robotium 4.2 APIs are not as good as the old BooleanTest that we were using; however, these failures also started appearing a day or two after bug 912590 landed, so it could be that we are just taking more time to load pages on Android 2.2.

I'll experiment with some timeouts, and see what works, and then upload a patch.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
I've been running tests on try, and a timeout of 4500 seems to avoid any of the oranges of this bug.
https://tbpl.mozilla.org/?tree=Try&rev=1002cc99bd15

For some reason, a lot of these orange reports don't have screenshots. Going through some of them, the full log from comment #12 has a screenshot from the failure does contain a screenshot (search for data:image), but the screenshot displays the right text (content is black, though).

I'll upload the 4500 timeout patch and we can land that and reopen this bug if this is still a problem. I'm not convinced this is simply a timeout problem though.
Created attachment 804813 [details] [diff] [review]
Patch: increase timeout to 4500
Attachment #804813 - Flags: review?(gbrown)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment on attachment 804813 [details] [diff] [review]
Patch: increase timeout to 4500

Review of attachment 804813 [details] [diff] [review]:
-----------------------------------------------------------------

Reluctantly...(I don't have a better idea.)
Attachment #804813 - Flags: review?(gbrown) → review+
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Landed on fx-team - let's see if this fixes this orange. I'll also do some more investigation to see if there's another way to fix this, better than just increasing the timeout.

https://hg.mozilla.org/integration/fx-team/rev/e1dfb1746f96
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Reporter)

Comment 146

5 years ago
https://hg.mozilla.org/mozilla-central/rev/e1dfb1746f96
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Flags: in-testsuite-
Resolution: --- → FIXED
Target Milestone: --- → Firefox 27
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)

Updated

5 years ago
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Most of these don't have screenshots, unfortunately. Comment #152 contains one, but the url is only partially typed in that screenshot. This means that at least some of these oranges are due to timeouts - it's still worth trying to find out why some of these are taking so much longer to load.

The correct approach is to listen for the page to finish loading, with a timeout.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
testBookmarklets, testHistory, testTabHistory & testBookmarkKeyword disabled until we get these failures under control (sorry):
https://hg.mozilla.org/integration/mozilla-inbound/rev/54d4889555ca
Whiteboard: [test disabled][leave open]
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Flags: needinfo?(mark.finkle)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)