Intermittent testBookmark | bookmark added successfully - got false, expected true

RESOLVED FIXED in Firefox 26

Status

()

Firefox for Android
General
RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: philor, Assigned: capella)

Tracking

({intermittent-failure})

Trunk
Firefox 27
ARM
Android
intermittent-failure
Points:
---

Firefox Tracking Flags

(firefox25 wontfix, firefox26 fixed, firefox27 fixed, firefox-esr24 wontfix)

Details

Attachments

(1 attachment)

(Reporter)

Description

5 years ago
https://tbpl.mozilla.org/php/getParsedLog.php?id=20404470&tree=Mozilla-Inbound
Android Tegra 250 mozilla-inbound opt test robocop-1 on 2013-03-06 21:07:24 PST for push 4b9f83fccf6b
slave: tegra-082

10 INFO TEST-UNEXPECTED-FAIL | testBookmark | bookmark added successfully - got false, expected true
junit.framework.AssertionFailedError: 10 INFO TEST-UNEXPECTED-FAIL | testBookmark | bookmark added successfully - got false, expected true
11 INFO TEST-UNEXPECTED-FAIL | testBookmark | Exception caught - junit.framework.AssertionFailedError: 10 INFO TEST-UNEXPECTED-FAIL | testBookmark | bookmark added successfully - got false, expected true
03-06 21:27:36.496 I/Robocop ( 3444): 10 INFO TEST-UNEXPECTED-FAIL | testBookmark | bookmark added successfully - got false, expected true
03-06 21:27:36.516 I/Robocop ( 3444): junit.framework.AssertionFailedError: 10 INFO TEST-UNEXPECTED-FAIL | testBookmark | bookmark added successfully - got false, expected true
03-06 21:27:36.516 I/Robocop ( 3444): 11 INFO TEST-UNEXPECTED-FAIL | testBookmark | Exception caught - junit.framework.AssertionFailedError: 10 INFO TEST-UNEXPECTED-FAIL | testBookmark | bookmark added successfully - got false, expected true
03-06 21:27:37.536 I/TestRunner( 3444): junit.framework.AssertionFailedError: 11 INFO TEST-UNEXPECTED-FAIL | testBookmark | Exception caught - junit.framework.AssertionFailedError: 10 INFO TEST-UNEXPECTED-FAIL | testBookmark | bookmark added successfully - got false, expected true
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Assignee)

Comment 47

5 years ago
all tegra (2.2)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Assignee)

Comment 54

5 years ago
This is interesting / weird in several ways ... 

The test that fails basically opens the menu list, clicks the bookmarks item and waits to see if a toast message appears that the page was bookmarked ...

This TestBookmark fails intermittently and apparently only on the tegras:
http://mxr.mozilla.org/mozilla-central/source/mobile/android/base/tests/testBookmark.java.in#152

But we have another testHistoryTab that does logically the same thing using different code here and I can't find a bugzilla complaining of it failing:
http://mxr.mozilla.org/mozilla-central/source/mobile/android/base/tests/testHistoryTab.java.in#54

margaret suggested I cc: wesj as he was involved with the code in the second test, and I added sriram as he's familiar with the GeckoMenu and I wonder if maybe the way the menu is inflated, or the timing of it is tricking out the the first test.

I'm tempted to copy over the code from testHistoryTab into testBookmark and was looking for input ... ?
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Reporter)

Updated

4 years ago
Blocks: 923764
(Assignee)

Comment 62

4 years ago
This may not be the same bug as 923764 ... testing a fix ...
(Assignee)

Comment 63

4 years ago
Created attachment 817136 [details] [diff] [review]
bug848671

Following up on testBookmark robocop issues, If I don't use the following patch, I can't run this test at all on my GS3 ... I receive the stacktrace @ bottom every time I try ... Running locally on my Asus Nexus 7 the test runs fine with or without the patch ...

( Based on this comment would we maybe want to replace / put in the patch? )
http://mxr.mozilla.org/mozilla-central/source/mobile/android/base/tests/BaseTest.java.in#713 


click: unable to find view 2131165374
waitForText timeout on Bookmark added
15 INFO TEST-UNEXPECTED-FAIL | testBookmark | bookmark added successfully - got false, expected true
Exception caught during test!
junit.framework.AssertionFailedError: 15 INFO TEST-UNEXPECTED-FAIL | testBookmark | bookmark added successfully - got false, expected true
	at junit.framework.Assert.fail(Assert.java:50)
	at org.mozilla.fennec.FennecMochitestAssert._logMochitestResult(FennecMochitestAssert.java:111)
	at org.mozilla.fennec.FennecMochitestAssert.ok(FennecMochitestAssert.java:140)
	at org.mozilla.fennec.FennecMochitestAssert.is(FennecMochitestAssert.java:146)
	at org.mozilla.fennec.tests.testBookmark.setUpBookmark(testBookmark.java:74)
	at org.mozilla.fennec.tests.testBookmark.runMenuTest(testBookmark.java:32)
	at org.mozilla.fennec.tests.testBookmark.testBookmark(testBookmark.java:27)
	at java.lang.reflect.Method.invokeNative(Native Method)
	at java.lang.reflect.Method.invoke(Method.java:511)
	at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
	at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
	at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
	at org.mozilla.fennec.tests.BaseTest.runTest(BaseTest.java:149)
Attachment #817136 - Flags: feedback?(lucasr.at.mozilla)
Comment on attachment 817136 [details] [diff] [review]
bug848671

Review of attachment 817136 [details] [diff] [review]:
-----------------------------------------------------------------

Clearing review flag to get requested information.

::: mobile/android/base/tests/testBookmark.java.in
@@ +66,5 @@
>      private void setUpBookmark() {
>          // Bookmark a page for the test
>          loadUrl(BOOKMARK_URL);
>          waitForText(StringHelper.ROBOCOP_BLANK_PAGE_01_TITLE);
> +        toggleBookmark();

Where's this toggleBookmark() defined? I could not find it in AboutHomeTest.
Attachment #817136 - Flags: feedback?(lucasr.at.mozilla)
(Assignee)

Comment 65

4 years ago
In the comment I had
http://mxr.mozilla.org/mozilla-central/source/mobile/android/base/tests/BaseTest.java.in#713

The new bugzilla lnks are hard to see
(Assignee)

Comment 67

4 years ago
Introduced here Bug 897970 - Robocop testBookmark fails consistently on Android x86 emulators
(Assignee)

Updated

4 years ago
Assignee: nobody → markcapella
Status: NEW → ASSIGNED
(Assignee)

Comment 68

4 years ago
Comment on attachment 817136 [details] [diff] [review]
bug848671

This patch is demonstrably required for GS3 4.1 devices so I'd like to move it.

Also, it hasn't random oranged in 6 weeks asof current date, so I'm thinking a) it's been resolved, or b) this will resolve it based on randomness of target test devices in the pool, and c) we can open a new bug if it recurs (?)

some TRY tests: https://tbpl.mozilla.org/?tree=Try&rev=a6b6e7565d9e
Attachment #817136 - Flags: review?(lucasr.at.mozilla)
Comment on attachment 817136 [details] [diff] [review]
bug848671

Review of attachment 817136 [details] [diff] [review]:
-----------------------------------------------------------------

Good.
Attachment #817136 - Flags: review?(lucasr.at.mozilla) → review+
https://hg.mozilla.org/mozilla-central/rev/4975704635c9
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 27
status-firefox25: --- → wontfix
status-firefox26: --- → affected
status-firefox27: --- → fixed
status-firefox-esr24: --- → wontfix
You need to log in before you can comment on or make changes to this bug.