If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Robocop exception handling is skipped in UITests

RESOLVED FIXED in Firefox 37

Status

()

Firefox for Android
Testing
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: gbrown, Assigned: gbrown)

Tracking

unspecified
Firefox 37
x86_64
Linux
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Assignee)

Description

3 years ago
In bug 1085837, a NullPointerException appears in the logcat for a Robocop test but does not fail the test with an appropriate message in the main test log. Notice that the exception stack trace does not include BaseTest.runTest:

04:08:13     INFO -  11-26 03:51:42.233 I/TestRunner( 1355): failed: testAppMenuPathways(org.mozilla.gecko.tests.testAppMenuPathways)
04:08:13     INFO -  11-26 03:51:42.246 I/TestRunner( 1355): ----- begin exception -----
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): java.lang.NullPointerException
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at org.mozilla.gecko.tests.components.AppMenuComponent.assertMenuItemIsDisabledAndVisible(AppMenuComponent.java:128)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at org.mozilla.gecko.tests.testAppMenuPathways._testSaveAsPDFPathway(testAppMenuPathways.java:26)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at org.mozilla.gecko.tests.testAppMenuPathways.testAppMenuPathways(testAppMenuPathways.java:21)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at java.lang.reflect.Method.invokeNative(Native Method)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at java.lang.reflect.Method.invoke(Method.java:507)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at junit.framework.TestCase.runBare(TestCase.java:127)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at junit.framework.TestResult$1.protect(TestResult.java:106)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at junit.framework.TestResult.runProtected(TestResult.java:124)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at junit.framework.TestResult.run(TestResult.java:109)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at junit.framework.TestCase.run(TestCase.java:118)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:529)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): 	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1448)
04:08:13     INFO -  11-26 03:51:42.303 I/TestRunner( 1355): ----- end exception -----

Is the stack misleading, or is BaseTest.runTest no longer invoked?
(Assignee)

Comment 1

3 years ago
Oh, I see! testAppMenuPathways (and others) extend UITest, which inherits from BaseRobocopTest, bypassing BaseTest and the exception handling in runTest. This is probably a bad thing....
(Assignee)

Updated

3 years ago
Summary: Is Robocop's BaseTest.runTest invoked? → Exception handling in Robocop BaseTest.runTest is sometimes skipped
(Assignee)

Updated

3 years ago
Assignee: nobody → gbrown
(Assignee)

Updated

3 years ago
Depends on: 1105792
(Assignee)

Updated

3 years ago
Depends on: 1105795
(Assignee)

Updated

3 years ago
Summary: Exception handling in Robocop BaseTest.runTest is sometimes skipped → Robocop exception handling is skipped in UITests
(Assignee)

Comment 2

3 years ago
Created attachment 8529914 [details] [diff] [review]
implement runTest in UITest

This just copies BaseTest.runTest into UITest. (Cannot be implemented in BaseRobocopTest without moving mSolo too.)
Attachment #8529914 - Flags: review?(michael.l.comella)
Blocks: 1106903
Comment on attachment 8529914 [details] [diff] [review]
implement runTest in UITest

Review of attachment 8529914 [details] [diff] [review]:
-----------------------------------------------------------------

(In reply to Geoff Brown [:gbrown] from comment #2)
> This just copies BaseTest.runTest into UITest. (Cannot be implemented in
> BaseRobocopTest without moving mSolo too.)

Seems do-able - I filed bug 1106900 and bug 1106903.
Attachment #8529914 - Flags: review?(michael.l.comella) → review+
(Assignee)

Updated

3 years ago
Depends on: 1085837
See Also: bug 1085837
(Assignee)

Comment 4

3 years ago
https://treeherder.mozilla.org/#/jobs?repo=try&revision=9dd49c68402d

https://hg.mozilla.org/integration/mozilla-inbound/rev/c292aca6244e
https://hg.mozilla.org/mozilla-central/rev/c292aca6244e
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 37
Thanks for pushing this forward, Geoff! Hearts!
You need to log in before you can comment on or make changes to this bug.