Closed
Bug 817970
Opened 8 years ago
Closed 8 years ago
Robocop: Add tests for Bookmark remove and edit from context menu
Categories
(Firefox for Android Graveyard :: General, defect)
Tracking
(Not tracked)
RESOLVED
FIXED
Firefox 21
People
(Reporter: AdrianT, Assigned: AdrianT)
References
Details
Attachments
(1 file, 3 obsolete files)
6.79 KB,
patch
|
jmaher
:
review+
|
Details | Diff | Splinter Review |
This bug will track the progress for adding Robocop test for the remove bookmark and edit bookmarks from content menu tests that are part of the Firefox for Android Smoke test suite. Remove bookmark from context menu: https://moztrap.mozilla.org/manage/case/316/ Edit bookmark from context menu: https://moztrap.mozilla.org/manage/case/315/
Assignee | ||
Updated•8 years ago
|
QA Contact: adrian.tamas
Assignee | ||
Updated•8 years ago
|
Assignee: nobody → adrian.tamas
Assignee | ||
Comment 1•8 years ago
|
||
Attachment #689703 -
Flags: review?(jmaher)
Comment 2•8 years ago
|
||
Comment on attachment 689703 [details] [diff] [review] Edit and Remove bookmark from the context menu v1 Review of attachment 689703 [details] [diff] [review]: ----------------------------------------------------------------- additional tests would be to test invalid characters and the flow of error messages. ::: mobile/android/base/tests/testBookmarksTab.java.in @@ +198,2 @@ > // back again to exit the awesomebar > mActions.sendSpecialKey(Actions.SpecialKey.BACK); is this back still valid? Also are we still in the awesomebar? @@ +237,5 @@ > + } else { > + mSolo.clickOnText("Cancel"); > + return false; > + } > + } both of these functions would make a good library function for other tests to use (not needed for this bug, but please file a followup bug)
Attachment #689703 -
Flags: review?(jmaher) → review+
Assignee | ||
Comment 3•8 years ago
|
||
(In reply to Joel Maher (:jmaher) from comment #2) > Comment on attachment 689703 [details] [diff] [review] > Edit and Remove bookmark from the context menu v1 > > Review of attachment 689703 [details] [diff] [review]: > ----------------------------------------------------------------- > > additional tests would be to test invalid characters and the flow of error > messages. > > ::: mobile/android/base/tests/testBookmarksTab.java.in > @@ +198,2 @@ > > // back again to exit the awesomebar > > mActions.sendSpecialKey(Actions.SpecialKey.BACK); > > is this back still valid? Also are we still in the awesomebar? The BACK from above this just closes the Custom Menu which was opened to test the bookmark title. This closes the Awesomebar although the test ends after this so this may not be necessary but it is part of the original test. > > @@ +237,5 @@ > > + } else { > > + mSolo.clickOnText("Cancel"); > > + return false; > > + } > > + } > > both of these functions would make a good library function for other tests > to use (not needed for this bug, but please file a followup bug) Follow-up bug logged - bug 820276 Please check in the patch as I do not have the permissions to do that.
Comment 4•8 years ago
|
||
I was not able to run this test locally: 0 INFO SimpleTest START 1 INFO TEST-START | testBookmarksTab 2 INFO TEST-PASS | testBookmarksTab | Inserted at: - content://org.mozilla.fennec_jmaher.db.browser/bookmarks/10?profile=default 3 INFO TEST-PASS | testBookmarksTab | checking that bookmarks list exists - android.widget.ListView@443830f0 4 INFO TEST-UNEXPECTED-FAIL | testBookmarksTab | bookmarks list has 5 children (defaults + a folder) - got 0, expected 5 Exception caught during test! junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testBookmarksTab | bookmarks list has 5 children (defaults + a folder) - got 0, expected 5 at junit.framework.Assert.fail(Assert.java:47) at org.mozilla.fennec_jmaher.FennecMochitestAssert._logMochitestResult(FennecMochitestAssert.java:107) at org.mozilla.fennec_jmaher.FennecMochitestAssert.ok(FennecMochitestAssert.java:136) at org.mozilla.fennec_jmaher.FennecMochitestAssert.is(FennecMochitestAssert.java:142) at org.mozilla.fennec_jmaher.tests.testBookmarksTab.testList(testBookmarksTab.java:65) at org.mozilla.fennec_jmaher.tests.testBookmarksTab.testBookmarksTab(testBookmarksTab.java:52) at java.lang.reflect.Method.invokeNative(Native Method) at java.lang.reflect.Method.invoke(Method.java:521) at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204) at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194) at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186) at org.mozilla.fennec_jmaher.tests.BaseTest.runTest(BaseTest.java:124) at junit.framework.TestCase.runBare(TestCase.java:127) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154) at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:520) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447) 5 INFO TEST-UNEXPECTED-FAIL | testBookmarksTab | Exception caught - junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testBookmarksTab | bookmarks list has 5 children (defaults + a folder) - got 0, expected 5 6 INFO TEST-END | testBookmarksTab | finished in 80041ms 7 INFO TEST-START | Shutdown 8 INFO Passed: 2 9 INFO Failed: 2 10 INFO Todo: 0
Assignee | ||
Comment 5•8 years ago
|
||
I did not modify that part of the original bookmark test. I believe that Bug 817414 already covers this. Should I try and look into this?
Comment 6•8 years ago
|
||
After a dozen runs I couldn't get this to pass locally or on try server, so I can't land it quite yet. We need to get bug 817414 fixed first.
Assignee | ||
Comment 7•8 years ago
|
||
Joel, can you please retest this now that bug 817414 has been fixed?
Comment 8•8 years ago
|
||
this is what I get now: FIRE PROC: '"MOZ_CRASHREPORTER=1,XPCOM_DEBUG_BREAK=stack,MOZ_HIDE_RESULTS_TABLE=1,MOZ_CRASHREPORTER_NO_REPORT=1,NO_EM_RESTART=1,MOZ_PROCESS_LOG=/tmp/tmpkzErS_pidlog,XPCOM_MEM_BLOAT_LOG=/tmp/tmpt0w6Lz/runtests_leaks.log" am instrument -w -e deviceroot /mnt/sdcard/tests -e class org.mozilla.fennec_jmaher.tests.testBookmarksTab org.mozilla.roboexample.test/org.mozilla.fennec_jmaher.FennecInstrumentationTestRunner' Robocop derived process name: org.mozilla.fennec_jmaher INFO | automation.py | Application pid: 0 0 INFO SimpleTest START 1 INFO TEST-START | testBookmarksTab 2 INFO TEST-PASS | testBookmarksTab | Inserted at: - content://org.mozilla.fennec_jmaher.db.browser/bookmarks/10?profile=default 3 INFO TEST-PASS | testBookmarksTab | checking that bookmarks list exists - android.widget.ListView@44306820 should not equal null 4 INFO TEST-PASS | testBookmarksTab | bookmarks list has 5 children (defaults + a folder) - 5 should equal 5 5 INFO TEST-PASS | testBookmarksTab | TextView is filled in - Firefox: Support 6 INFO TEST-PASS | testBookmarksTab | TextView is filled in - http://support.mozilla.org/en-US/mobile 7 INFO TEST-PASS | testBookmarksTab | Correct number of ImageViews visible - 1 should equal 1 8 INFO TEST-PASS | testBookmarksTab | Correct number of TextViews visible - 2 should equal 2 9 INFO TEST-PASS | testBookmarksTab | TextView is filled in - Firefox: Customize with add-ons 10 INFO TEST-PASS | testBookmarksTab | TextView is filled in - https://addons.mozilla.org/en-US/android/ 11 INFO TEST-PASS | testBookmarksTab | Correct number of ImageViews visible - 1 should equal 1 12 INFO TEST-PASS | testBookmarksTab | Correct number of TextViews visible - 2 should equal 2 13 INFO TEST-PASS | testBookmarksTab | TextView is filled in - Firefox Start 14 INFO TEST-PASS | testBookmarksTab | TextView is filled in - about:home 15 INFO TEST-PASS | testBookmarksTab | Correct number of ImageViews visible - 1 should equal 1 16 INFO TEST-PASS | testBookmarksTab | Correct number of TextViews visible - 2 should equal 2 17 INFO TEST-PASS | testBookmarksTab | TextView is filled in - Firefox: About your browser 18 INFO TEST-PASS | testBookmarksTab | TextView is filled in - about:firefox 19 INFO TEST-PASS | testBookmarksTab | Correct number of ImageViews visible - 1 should equal 1 20 INFO TEST-PASS | testBookmarksTab | Correct number of TextViews visible - 2 should equal 2 21 INFO TEST-PASS | testBookmarksTab | TextView is filled in - Desktop Bookmarks 22 INFO TEST-PASS | testBookmarksTab | Correct number of ImageViews visible - 1 should equal 1 23 INFO TEST-PASS | testBookmarksTab | Correct number of TextViews visible - 1 should equal 1 24 INFO TEST-PASS | testBookmarksTab | desktop folder has correct number of children - 4 should equal 4 25 INFO TEST-PASS | testBookmarksTab | TextView is filled in - Unsorted Bookmarks 26 INFO TEST-PASS | testBookmarksTab | Correct number of ImageViews visible - 1 should equal 1 27 INFO TEST-PASS | testBookmarksTab | Correct number of TextViews visible - 1 should equal 1 28 INFO TEST-PASS | testBookmarksTab | TextView is filled in - Bookmarks Menu 29 INFO TEST-PASS | testBookmarksTab | Correct number of ImageViews visible - 1 should equal 1 30 INFO TEST-PASS | testBookmarksTab | Correct number of TextViews visible - 1 should equal 1 31 INFO TEST-PASS | testBookmarksTab | TextView is filled in - Bookmarks Toolbar 32 INFO TEST-PASS | testBookmarksTab | Correct number of ImageViews visible - 1 should equal 1 33 INFO TEST-PASS | testBookmarksTab | Correct number of TextViews visible - 1 should equal 1 34 INFO TEST-PASS | testBookmarksTab | TextView is filled in - Desktop Bookmarks 35 INFO TEST-PASS | testBookmarksTab | Correct number of ImageViews visible - 0 should equal 0 36 INFO TEST-PASS | testBookmarksTab | Correct number of TextViews visible - 1 should equal 1 37 INFO TEST-PASS | testBookmarksTab | toolbar folder has correct number of children - 2 should equal 2 38 INFO TEST-PASS | testBookmarksTab | TextView is filled in - BOOKMARK_TITLE 39 INFO TEST-PASS | testBookmarksTab | TextView is filled in - http://www.example.com 40 INFO TEST-PASS | testBookmarksTab | Correct number of ImageViews visible - 1 should equal 1 41 INFO TEST-PASS | testBookmarksTab | Correct number of TextViews visible - 2 should equal 2 42 INFO TEST-PASS | testBookmarksTab | TextView is filled in - Bookmarks Toolbar 43 INFO TEST-PASS | testBookmarksTab | Correct number of ImageViews visible - 0 should equal 0 44 INFO TEST-PASS | testBookmarksTab | Correct number of TextViews visible - 1 should equal 1 45 INFO TEST-PASS | testBookmarksTab | Back moved up one level - 46 INFO TEST-PASS | testBookmarksTab | Back moved up one level - 47 INFO TEST-PASS | testBookmarksTab | found TextView: - Firefox: About your browser 48 INFO TEST-PASS | testBookmarksTab | found TextView: - about:firefox clickLongOnView: android.widget.RelativeLayout@44433708 49 INFO TEST-PASS | testBookmarksTab | Context menu has New Tab option - Open in New Tab 50 INFO TEST-PASS | testBookmarksTab | Context menu has Share option - Share 51 INFO TEST-PASS | testBookmarksTab | Context menu has Edit option - Edit 52 INFO TEST-PASS | testBookmarksTab | Context menu has Remove option - Remove 53 INFO TEST-PASS | testBookmarksTab | Context menu has Add to Home Screen option - Add to Home Screen 54 INFO TEST-PASS | testBookmarksTab | Folders have no context menu - false should equal false Exception caught during test! junit.framework.AssertionFailedError: The text: Edit is not found! at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at com.jayway.android.robotium.solo.Clicker.clickOnText(Clicker.java:291) at com.jayway.android.robotium.solo.Solo.clickOnText(Solo.java:823) at org.mozilla.fennec_jmaher.tests.testBookmarksTab.editBookmark(testBookmarksTab.java:207) at org.mozilla.fennec_jmaher.tests.testBookmarksTab.testContextMenu(testBookmarksTab.java:160) at org.mozilla.fennec_jmaher.tests.testBookmarksTab.testBookmarksTab(testBookmarksTab.java:52) at java.lang.reflect.Method.invokeNative(Native Method) at java.lang.reflect.Method.invoke(Method.java:521) at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204) at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194) at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186) at org.mozilla.fennec_jmaher.tests.BaseTest.runTest(BaseTest.java:130) at junit.framework.TestCase.runBare(TestCase.java:127) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154) at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:520) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447) 55 INFO TEST-UNEXPECTED-FAIL | testBookmarksTab | Exception caught - junit.framework.AssertionFailedError: The text: Edit is not found! 56 INFO TEST-END | testBookmarksTab | finished in 122209ms 57 INFO TEST-START | Shutdown 58 INFO Passed: 53 59 INFO Failed: 1 60 INFO Todo: 0 61 INFO SimpleTest FINISHED INFO | automation.py | Application ran for: 0:02:06.885968 INFO | automation.py | Reading PID log: /tmp/tmpkzErS_pidlog getting files in '/mnt/sdcard/tests/profile/minidumps/' really close!
Assignee | ||
Comment 9•8 years ago
|
||
Removed the calls to press "Back" in order to close the vkb since now when the user enters the Bookmarks tab the vkb is dismissed by default. Also changed the patch to get the Bookmarks list once since when building the ListView it seems that the order is not always the same and getChildAt(1) would select different bookmarks not always the first one.
Attachment #689703 -
Attachment is obsolete: true
Attachment #701822 -
Flags: review?(jmaher)
Comment 10•8 years ago
|
||
Comment on attachment 701822 [details] [diff] [review] Edit and Remove bookmark from the context menu v2 Review of attachment 701822 [details] [diff] [review]: ----------------------------------------------------------------- this looks good, I will send it to try server as and land it if it works great. Locally it passed.
Attachment #701822 -
Flags: review?(jmaher) → review+
Comment 11•8 years ago
|
||
this has failures on try server: https://tbpl.mozilla.org/?tree=Try&rev=e0b581d23864 53 INFO TEST-PASS | testBookmarksTab | Context menu has Add to Home Screen option - Add to Home Screen 54 INFO TEST-PASS | testBookmarksTab | Folders have no context menu - false should equal false 55 INFO TEST-UNEXPECTED-FAIL | testBookmarksTab | Bookmark Name was changed - got false, expected true Exception caught during test! junit.framework.AssertionFailedError: 55 INFO TEST-UNEXPECTED-FAIL | testBookmarksTab | Bookmark Name was changed - got false, expected true at junit.framework.Assert.fail(Assert.java:47) at org.mozilla.fennec.FennecMochitestAssert._logMochitestResult(FennecMochitestAssert.java:107) at org.mozilla.fennec.FennecMochitestAssert.ok(FennecMochitestAssert.java:136) at org.mozilla.fennec.FennecMochitestAssert.is(FennecMochitestAssert.java:142) at org.mozilla.fennec.tests.testBookmarksTab.testContextMenu(testBookmarksTab.java:157) at org.mozilla.fennec.tests.testBookmarksTab.testBookmarksTab(testBookmarksTab.java:46) at java.lang.reflect.Method.invokeNative(Native Method) at java.lang.reflect.Method.invoke(Method.java:521) at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204) at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194) at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186) at org.mozilla.fennec.tests.BaseTest.runTest(BaseTest.java:129) at junit.framework.TestCase.runBare(TestCase.java:127) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154) at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:520) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447) 56 INFO TEST-UNEXPECTED-FAIL | testBookmarksTab | Exception caught - junit.framework.AssertionFailedError: 55 INFO TEST-UNEXPECTED-FAIL | testBookmarksTab | Bookmark Name was changed - got false, expected true 57 INFO TEST-END | testBookmarksTab | finished in 83371ms 58 INFO TEST-START | Shutdown 59 INFO Passed: 53 60 INFO Failed: 2 61 INFO Todo: 0
Assignee | ||
Comment 12•8 years ago
|
||
I am unable to reproduce the fail at all in about 20+ tries on an HTC Desire running Android 2.2. Best guess at a fix for this: making sure the Edit Bookmark pop-up is opened before checking the text and closing the vkb after exiting Bookmark Edit pop-up to ensure that the hole bookmark list is visible.
Attachment #701822 -
Attachment is obsolete: true
Attachment #702319 -
Flags: review?(jmaher)
Comment 13•8 years ago
|
||
Comment on attachment 702319 [details] [diff] [review] Edit and Remove bookmark from the context menu v2.1 sorry to say this, but using the latest patch still causes me to fail on a production tegra.
Attachment #702319 -
Flags: review?(jmaher) → review-
Assignee | ||
Comment 14•8 years ago
|
||
I believe I have identified the issue. On Gingerbread the keyboard is opened when entering the text in the edit bookmark pop-up but on ICS it's not so I added an OS specific restriction. Also I am clearing the fields before adding the text in order to make sure there are no issues with the text. Tested this version of the patch on the HTC Desire (Android 2.2), Samsung Galaxy Tab 2 7.0 (Android 4.0.4) and I managed to get a Kindle Fire for half an hour to test on. The Kindle has a custom Android build but it is based on the 2.3 version so it may be somewhat relevant and comparable to the Tegra boards.
Attachment #702319 -
Attachment is obsolete: true
Attachment #704526 -
Flags: review?(jmaher)
Comment 15•8 years ago
|
||
Comment on attachment 704526 [details] [diff] [review] Edit and Remove bookmark from the context menu v3 Review of attachment 704526 [details] [diff] [review]: ----------------------------------------------------------------- Thanks for adding this. Looks great.
Attachment #704526 -
Flags: review?(jmaher) → review+
Comment 16•8 years ago
|
||
https://hg.mozilla.org/mozilla-central/rev/0770079f15c7
Status: NEW → RESOLVED
Closed: 8 years ago
Flags: in-testsuite+
Resolution: --- → FIXED
Target Milestone: --- → Firefox 21
Updated•2 months ago
|
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•