1341527 - Intermittent android-test | Execution failed for task ':app:testAutomationDebugUnitTest'.

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Testing, defect, P1)

Description

[Iris Hsiao [:ihsiao]]

18:40:25     INFO -  :app:testAutomationDebugUnitTest
18:40:25     INFO -  org.mozilla.gecko.icons.preparation.TestSuggestedSitePreparer > testSuggestedIconsAdded FAILED
18:40:25     INFO -      java.lang.NullPointerException at TestSuggestedSitePreparer.java:37
18:40:25     INFO -  org.mozilla.gecko.icons.preparation.TestSuggestedSitePreparer > testNonSuggestedURL FAILED
18:40:25     INFO -      java.lang.NullPointerException at TestSuggestedSitePreparer.java:37
18:41:33     INFO -  715 tests completed, 2 failed, 2 skipped
18:41:34     INFO -  :app:testAutomationDebugUnitTest FAILED
18:41:34     INFO -  TEST-UNEXPECTED-FAIL | android-test | There were failing tests. See the report at: https://queue.taskcluster.net/v1/task/BMTT5pQbTtimYIAJfOjHvw/runs/0/artifacts/public/android/unittest/automationDebug/index.html
18:41:34     INFO -  FAILURE: Build failed with an exception.
18:41:34     INFO -  * What went wrong:
18:41:34     INFO -  Execution failed for task ':app:testAutomationDebugUnitTest'.
18:41:34     INFO -  > There were failing tests. See the report at: file:///home/worker/workspace/build/src/obj-firefox/gradle/build/mobile/android/app/reports/tests/automationDebug/index.html
18:41:34     INFO -  * Try:
18:41:34     INFO -  Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
18:41:34     INFO -  BUILD FAILED
18:41:34     INFO -  Total time: 2 mins 3.744 secs
18:41:34    ERROR - Return code: 1
18:41:34    ERROR - 1 not in success codes: [0]
18:41:34  WARNING - setting return code to 2
18:41:34    FATAL - Halting on failure while running ['python2.7', 'mach', '--log-no-times', 'gradle', 'app:testAutomationDebugUnitTest']
18:41:34    FATAL - Running post_fatal callback...
18:41:34    FATAL - Exiting 2
18:41:34     INFO - [mozharness: 2017-02-15 18:41:34.497043Z] Finished build step (failed)
18:41:34     INFO - Running post-run listener: _summarize
18:41:34    ERROR - # TBPL FAILURE #
18:41:34     INFO - [mozharness: 2017-02-15 18:41:34.497776Z] FxDesktopBuild summary:
18:41:34    ERROR - # TBPL FAILURE #


https://treeherder.mozilla.org/logviewer.html#?job_id=77679227&repo=autoland&lineNumber=3724

https://public-artifacts.taskcluster.net/BMTT5pQbTtimYIAJfOjHvw/0/public/logs/live_backing.log

https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=0cfcc10ad05c6bb6d8b6494b5648950169d588ff&selectedJob=77679227

Comment 1

[Andrzej Hunt :ahunt]

Attachment: Bug 1341527 - Add restrictions mocking to SuggestedSitePreparer test

Review commit: https://reviewboard.mozilla.org/r/115006/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/115006/

Comment 2

[Andrzej Hunt :ahunt]

Attachment: Bug 1341527 - Move UserManager mocking into helper class

We need this for more than one test, let's have one copy we can call
from wherever it's needed.

Review commit: https://reviewboard.mozilla.org/r/115080/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/115080/

Comment 3

[Andrzej Hunt :ahunt]

Comment on attachment 8840519 [details]
Bug 1341527 - Add restrictions mocking to SuggestedSitePreparer test

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/115006/diff/1-2/

Comment 4

[Intermittent Failures Robot]

19 failures in 812 pushes (0.023 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* autoland: 9
* mozilla-inbound: 7
* mozilla-central: 2
* try: 1

Platform breakdown:
* android-4-0-armv7-api15: 19

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341527&startday=2017-02-20&endday=2017-02-26&tree=all

Comment 5

[Sebastian Kaspari (:sebastian; :pocmo)]

Comment on attachment 8840629 [details]
Bug 1341527 - Move UserManager mocking into helper class

https://reviewboard.mozilla.org/r/115080/#review117750

::: mobile/android/tests/background/junit4/src/org/mozilla/gecko/helpers/MockUserManager.java:24
(Diff revision 1)
> +        // This seems to be done automatically in newer versions of RoboElectric, we need our own
> +        // version for now:

Is it an option to just update Robolectric? This is easier than our app dependencies. Let's file a bug if this is the way to go and we can do it after Focus. :)

Comment 6

[Sebastian Kaspari (:sebastian; :pocmo)]

Comment on attachment 8840519 [details]
Bug 1341527 - Add restrictions mocking to SuggestedSitePreparer test

https://reviewboard.mozilla.org/r/115006/#review117752

Comment 7

[Andrzej Hunt :ahunt]

Comment on attachment 8840629 [details]
Bug 1341527 - Move UserManager mocking into helper class

https://reviewboard.mozilla.org/r/115080/#review117876

::: mobile/android/tests/background/junit4/src/org/mozilla/gecko/helpers/MockUserManager.java:24
(Diff revision 1)
> +        // This seems to be done automatically in newer versions of RoboElectric, we need our own
> +        // version for now:

Filed as bug 1343556 !

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9f67d8a50747
Move UserManager mocking into helper class r=sebastian
https://hg.mozilla.org/integration/autoland/rev/a0d02ea27731
Add restrictions mocking to SuggestedSitePreparer test r=sebastian

Comment 9

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/9f67d8a50747
https://hg.mozilla.org/mozilla-central/rev/a0d02ea27731

Comment 10

[Intermittent Failures Robot]

16 failures in 783 pushes (0.02 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* autoland: 7
* mozilla-inbound: 5
* mozilla-central: 3
* try: 1

Platform breakdown:
* android-4-0-armv7-api15: 16

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341527&startday=2017-02-27&endday=2017-03-05&tree=all