Closed Bug 823165 Opened 9 years ago Closed 9 years ago

fix robocop tests to work on panda boards

Categories

(Testing :: General, defect)

ARM
Android
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jmaher, Assigned: jmaher)

References

Details

(Whiteboard: [leave open])

Attachments

(2 files, 1 obsolete file)

there are two failures I see in the logs on m-c:
* testAwesomebarSwipes
* testBookmarklets

I have fixed these for the panda board and tested them locally on my tegra as well.
Assignee: nobody → jmaher
Status: NEW → ASSIGNED
Attachment #693974 - Flags: review?(gbrown)
Comment on attachment 693974 [details] [diff] [review]
fix two test cases which fail on panda boards (1.0)

Review of attachment 693974 [details] [diff] [review]:
-----------------------------------------------------------------

::: mobile/android/base/tests/BaseTest.java.in
@@ +78,5 @@
> +
> +        // This is required for us to pass testBookmarklets, otherwise we fail either with
> +        // StrictMode or something related to sending the ENTER key
> +        int applicationFlags = getActivity().getApplicationInfo().flags;
> +

Odd, very odd - but if you are sure you need it...

::: mobile/android/base/tests/testAwesomebarSwipes.java.in
@@ +35,5 @@
>          mAsserter.is(pager.getCurrentItem(), 0, "All pages is selected");
>  
> +        int width = mDriver.getGeckoWidth() / 2;
> +        int y = mDriver.getGeckoHeight() / 2;
> +        mActions.drag(width, 0, y, y);

nit: Can we find better names for width and y? I like midX and midY.
Attachment #693974 - Flags: review?(gbrown) → review+
I tried running testBookmarklets on my local panda. Without this patch, I reproduced a failure immediately following a StrictMode error. But I was unable to verify that the patch fixes the test -- I keep hitting failures and timeouts in sutAgent (I may need to re-image my pandaboard).
Geoff,

You might want to reflash your sdcard with this:
http://people.mozilla.org/~jmaher/panda/panda_2gb_v2.img.bz2

dd if=panda_2gb_v2.img of=/dev/sdb bs=100M
I am closer to the root cause as to why testBookmarklets fails on panda board.  It boils down to this:
http://mxr.mozilla.org/mozilla-central/source/build/automation.py.in#545

                        '(?::(\\\\\\\\d+))?/');

if I change that to:
                        '(?::(\\\\d+))?/');

the test passes.  I need to look at this test case and see what the urls and network access is doing.  There has to be something there causing this confusion.
even more debugging shows that on the tegra when we display the awesomebar, the testAlertBookmarklet is shown in the list and the virtual keyboard is not shown.

on the panda board, the testAlertBookmarklet is in the awesomebar, but we have to scroll down to see it, presumably because the virtual keyboard is shown.
this patch does some rework of testBookmarklets so it will work reliably on the panda board.  This also refactors the testBookmarksTab as there was duplicated code.

there are still 2 failing robocop tests on the panda boards (which were not failing last month), but we are getting closer.
Attachment #704080 - Flags: feedback?(gbrown)
Comment on attachment 704080 [details] [diff] [review]
refactor bookmarks to allow tests to run on panda boards (1.0)

Review of attachment 704080 [details] [diff] [review]:
-----------------------------------------------------------------

I like the re-factoring -- thanks.

::: mobile/android/base/tests/testBookmarklets.java.in
@@ +58,5 @@
> +                    mSolo.clickOnView(bookmarks.getChildAt(i));
> +                }
> +            }
> +        }
> +        mAsserter.is(found, true, "Found bookmarklet and clicked on it");

Extra credit for reporting what was clicked on here (the url / js).

Even so, if the "Found bookmarklet" assertion fails, it leaves a lot of questions: was bookmarks null, did the waitForText fail, which bookmarks were in the list? I would like to see a clearer connection between failure and assertion, or maybe just additional diagnostics in the failure case...but you have to balance that against making this too complex.
Attachment #704080 - Flags: feedback?(gbrown) → feedback+
updated with feedback.  Green on try server.
Attachment #704080 - Attachment is obsolete: true
Attachment #704139 - Flags: review?(gbrown)
Comment on attachment 704139 [details] [diff] [review]
refactor bookmarks to allow tests to run on panda boards (1.1)

Review of attachment 704139 [details] [diff] [review]:
-----------------------------------------------------------------

::: mobile/android/base/tests/BaseTest.java.in
@@ +465,5 @@
> +            mAsserter.ok(true, "Inserted at: ", uri.toString());
> +            return ContentUris.parseId(uri);
> +       }
> +
> +       return ContentUris.parseId(bookmarksUri);

nit: spacing is off by one here

::: mobile/android/base/tests/testBookmarklets.java.in
@@ +57,5 @@
> +                    mSolo.clickOnView(bookmarks.getChildAt(i));
> +                }
> +            }
> +        } else {
> +            mAsserter.is(true, true, "Did not find the title '" + title + "' of the bookmark in the list");

Shouldn't that be: mAsserter.is(true, false, ... ?
Attachment #704139 - Flags: review?(gbrown) → review+
it appears that testVkbOverlap started failing after we pushed:
https://bugzilla.mozilla.org/show_bug.cgi?id=725018

After looking into this, when the keyboard shows up we don't focus on the element and it goes offscreen.

I am not sure if this is a test issue or a product issue.  Something tells me this is a product issue.

I cc'd wesj since he did the changes.
Hard to tell without the failure. We're showing the keyboard in response to the element being focused so the sentence "when the keyboard shows up we don't focus on the element" doesn't make sense to me. We fixed this test to work with the new code.
here is a link to a log which shows the errors:
https://tbpl.mozilla.org/php/getParsedLog.php?id=19074132&tree=Cedar&full=1

These are the same on any tree.  The key here is this is on the panda boards with ICS, not the tegras.
I believe the focused element gets covered by the keyboard and not scrolled up into the visible area.
fwiw, this is the only test stopping us from going green.  I tested on try by removing this test from the ini file and we were green.
(In reply to Joel Maher (:jmaher) from comment #8)
> I am closer to the root cause as to why testBookmarklets fails on panda
> board.  It boils down to this:
> http://mxr.mozilla.org/mozilla-central/source/build/automation.py.in#545
> 
>                         '(?::(\\\\\\\\d+))?/');
> 
> if I change that to:
>                         '(?::(\\\\d+))?/');
> 
> the test passes.  I need to look at this test case and see what the urls and
> network access is doing.  There has to be something there causing this
> confusion.

Odd. I had to make this same change on desktop when mozbase-ifying mochitest's profile.  I'm not sure if that is related and interesting or inane and a coincidence.
Depends on: 837318
We have these working in general.  They are hidden right now due to some timeouts, but these same timeouts are seen on tegras, just not as frequently.
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.