1476315 - java-exception android.app.RemoteServiceException: Context.startForegroundService did not then call Service.startForeground() at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1778)

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: General, defect, P5)

Description

[Bob Clary [:bc] (inactive)]

Beginning with Bug 1474961 - Change StumblerService to a foreground service when targeting Android Oreo mda1 and mda3 began perma failing on Android Hardware Pixel 2 running Android 8.0.

https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=c6763cfa891a86b897a214a14e7c226361650bb4&filter-tier=1&filter-tier=2&filter-tier=3&filter-searchStr=android-hw

PROCESS-CRASH | dom/media/test/test_autoplay_policy.html | java-exception android.app.RemoteServiceException: Context.startForegroundService() did not then call Service.startForeground() at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1778)

among other similar errors

Comment 1

[Bob Clary [:bc] (inactive)]

Note mda2 occasionally fails with the java-exception as well.

Comment 2

[Intermittent Failures Robot]

99 failures in 119 pushes (0.832 failures/push) were associated with this bug yesterday.    

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. ** 

Repository breakdown:
* autoland: 47
* mozilla-inbound: 38
* mozilla-central: 14

Platform breakdown:
* android-hw-p2-8-0-arm7-api-16: 99

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1476315&startday=2018-07-18&endday=2018-07-18&tree=trunk

Comment 3

[Natalia Csoregi [:nataliaCs]]

There has been a total of 230 tier 3 failures since the failure has started appearing, on Android 8.0 Pixel2 opt & debug.
Recent failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=188827317&repo=mozilla-inbound&lineNumber=2740
22:13:43     INFO -  63 INFO TEST-UNEXPECTED-ERROR | dom/media/tests/mochitest/test_getUserMedia_basicVideoAudio.html | unexpected-crash-dump-found - This test left crash dumps behind, but we weren't expecting it to!
22:13:43     INFO -  64 INFO Found unexpected crash dump file /storage/emulated/0/tests/profile/minidumps/246709c7-b51a-b40b-6453-6140d9c6e422.dmp.
22:14:04     INFO -  INFO | automation.py | Application ran for: 0:00:53.299469
22:14:04     INFO -  INFO | zombiecheck | Reading PID log: /tmp/tmpDxFzAupidlog
22:14:06     INFO -  /data/tombstones does not exist; tombstone check skipped
22:14:06  WARNING -  PROCESS-CRASH | dom/media/tests/mochitest/test_getUserMedia_basicVideoAudio.html | java-exception android.app.RemoteServiceException: Context.startForegroundService() did not then call Service.startForeground() at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1778)
22:14:06     INFO -  Stopping web server
22:14:06     INFO -  Stopping web socket server
22:14:06     INFO -  Stopping ssltunnel
22:14:06     INFO -  websocket/process bridge listening on port 8191
22:14:06     INFO -  Stopping websocket/process bridge
22:14:06     INFO -  leakcheck | refcount logging is off, so leaks can't be detected!
22:14:06     INFO -  runtests.py | Running tests: end.
22:14:07     INFO -  Buffered messages finished
22:14:08     INFO -  0 INFO TEST-START | Shutdown
22:14:08     INFO -  1 INFO Passed:  0
22:14:08     INFO -  2 INFO Failed:  0
22:14:08     INFO -  3 INFO Todo:    0
22:14:08     INFO -  4 INFO Mode:    non-e10s
22:14:08     INFO -  5 INFO SimpleTest FINISHED
22:14:08     INFO -  --------- beginning of system
22:14:08     INFO -  07-18 15:12:52.875 I/ActivityManager( 1173): Force stopping org.mozilla.fennec_aurora appid=10120 user=0: from pid 5438
22:14:08     INFO -  07-18 15:12:54.694 D/NetlinkEvent(  711): Unknown ND option type 31
22:14:08     INFO -  07-18 15:13:00.555 I/ActivityManager( 1173): Killing 4193:com.google.android.gm/u0a89 (adj 906): empty #17
22:14:08     INFO -  07-18 15:13:00.631 D/NetlinkEvent(  711): Unknown ND option type 31
22:14:08     INFO -  07-18 15:13:04.931 I/ActivityManager( 1173): Killing 4380:com.google.android.videos/u0a77 (adj 906): empty #17
22:14:08     INFO -  07-18 15:13:10.151 D/NetlinkEvent(  711): Unknown ND option type 31
22:14:08     INFO -  07-18 15:13:11.663 I/ActivityManager( 1173): START u0 {act=android.intent.action.VIEW dat=http://mochi.test:8888/... flg=0x10000000 cmp=org.mozilla.fennec_aurora/org.mozilla.gecko.BrowserApp (has extras)} from uid 2000
22:14:08     INFO -  07-18 15:13:11.685 I/ActivityManager( 1173): Start proc 5506:org.mozilla.fennec_aurora/u0a120 for activity org.mozilla.fennec_aurora/org.mozilla.gecko.BrowserApp
22:14:08     INFO -  07-18 15:13:14.457 D/NetlinkEvent(  711): Unknown ND option type 31
22:14:08     INFO -  07-18 15:13:16.236 W/ActivityManager( 1173): Permission Denial: Accessing service ComponentInfo{com.google.android.music/com.google.android.music.dial.DialMediaRouteProviderService} from pid=5506, uid=10120 that is not exported from uid 10087
22:14:08     INFO -  07-18 15:13:21.344 E/NotificationService( 1173): Muting recently noisy 0|org.mozilla.fennec_aurora|2131296516|null|10120
22:14:08     INFO -  07-18 15:13:23.129 E/NotificationService( 1173): Muting recently noisy 0|org.mozilla.fennec_aurora|2131296516|null|10120
22:14:08     INFO -  07-18 15:13:23.152 D/NetlinkEvent(  711): Unknown ND option type 31
22:14:08     INFO -  07-18 15:13:27.288 E/NotificationService( 1173): Muting recently noisy 0|org.mozilla.fennec_aurora|2131296516|null|10120

Comment 4

[Intermittent Failures Robot]

75 failures in 126 pushes (0.595 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 35
* mozilla-inbound: 29
* mozilla-central: 11

Platform breakdown:
* android-hw-p2-8-0-arm7-api-16: 75

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1476315&startday=2018-07-19&endday=2018-07-19&tree=trunk

Comment 5

[Vlad Baicu]

It appears that this issue is caused because the service is stopped too fast (couple of millis) before it has a chance to start and call startForeground. Looking into fixes

Comment 6

[Intermittent Failures Robot]

97 failures in 117 pushes (0.829 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 55
* mozilla-inbound: 32
* mozilla-central: 10

Platform breakdown:
* android-hw-p2-8-0-arm7-api-16: 97

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1476315&startday=2018-07-20&endday=2018-07-20&tree=trunk

Comment 7

[Intermittent Failures Robot]

604 failures in 653 pushes (0.925 failures/push) were associated with this bug in the last 7 days. 

This is the #2 most frequent failure this week. 

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **   

Repository breakdown:
* autoland: 261
* mozilla-inbound: 258
* mozilla-central: 85

Platform breakdown:
* android-hw-p2-8-0-arm7-api-16: 604

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1476315&startday=2018-07-16&endday=2018-07-22&tree=trunk

Comment 8

[Intermittent Failures Robot]

85 failures in 126 pushes (0.675 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 38
* mozilla-inbound: 32
* mozilla-central: 15

Platform breakdown:
* android-hw-p2-8-0-arm7-api-16: 85

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1476315&startday=2018-07-23&endday=2018-07-23&tree=trunk

Comment 9

[Cristina Coroiu [:ccoroiu]]

:Vlad Vaicu do you have any updates?

Comment 10

[Vlad Baicu]

Currently working on a patch for it, should be up for review soon

Comment 11

[Vlad Baicu]

Submitted a patch on https://bugzilla.mozilla.org/show_bug.cgi?id=1477041 which also fixes these tests, tracking there for now.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=aea5d4da4a9ba9dafc72d3ea41a5116812b97a69

Comment 12

[Bob Clary [:bc] (inactive)]

Vlad: You need to view the Tier 3 jobs to see the Android Hardware tests. I see lots of ava-exception java.lang.NullPointerException: Attempt to invoke virtual method 'android.content.pm.ApplicationInfo android.content.Context.getApplicationInfo()' on a null object reference at android.content.ContextWrapper.getApplicationInfo(ContextWrapper.java:151) errors in your try run for Android hardware and the mda tests.

Comment 13

[Intermittent Failures Robot]

89 failures in 152 pushes (0.586 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 37
* mozilla-inbound: 33
* mozilla-central: 19

Platform breakdown:
* android-hw-p2-8-0-arm7-api-16: 89

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1476315&startday=2018-07-24&endday=2018-07-24&tree=trunk

Comment 14

[Vlad Baicu]

I've caught those before pushing that review and modified it, this is try result for mda jobs on the latest revision
 
https://treeherder.mozilla.org/#/jobs?repo=try&revision=10c7249a2b6844ff3876400144de12ea958c4adb&filter-tier=1&filter-tier=2&filter-tier=3

Comment 15

[Bob Clary [:bc] (inactive)]

\o/

Comment 16

[Intermittent Failures Robot]

27 failures in 138 pushes (0.196 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 18
* mozilla-inbound: 9

Platform breakdown:
* android-hw-p2-8-0-arm7-api-16: 27

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1476315&startday=2018-07-25&endday=2018-07-25&tree=trunk

Comment 17

[Firefox Bug Husbandry Bot]

Re-triaging per https://bugzilla.mozilla.org/show_bug.cgi?id=1473195

Needinfo :susheel if you think this bug should be re-triaged.

Comment 18

[Intermittent Failures Robot]

107 failures in 142 pushes (0.754 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 52
* mozilla-inbound: 51
* mozilla-central: 4

Platform breakdown:
* android-hw-p2-8-0-arm7-api-16: 107

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1476315&startday=2018-07-26&endday=2018-07-26&tree=trunk

Comment 19

[Intermittent Failures Robot]

123 failures in 134 pushes (0.918 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 59
* mozilla-inbound: 49
* mozilla-central: 15

Platform breakdown:
* android-hw-p2-8-0-arm7-api-16: 123

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1476315&startday=2018-07-27&endday=2018-07-27&tree=trunk

Comment 20

[Intermittent Failures Robot]

47 failures in 41 pushes (1.146 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* mozilla-inbound: 20
* autoland: 20
* mozilla-central: 7

Platform breakdown:
* android-hw-p2-8-0-arm7-api-16: 47

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1476315&startday=2018-07-28&endday=2018-07-28&tree=trunk

Comment 21

[Intermittent Failures Robot]

790 failures in 760 pushes (1.039 failures/push) were associated with this bug in the last 7 days. 

This is the #3 most frequent failure this week. 

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **   

Repository breakdown:
* autoland: 342
* mozilla-inbound: 311
* mozilla-central: 137

Platform breakdown:
* android-hw-p2-8-0-arm7-api-16: 790

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1476315&startday=2018-07-23&endday=2018-07-29&tree=trunk

Comment 22

[Bob Clary [:bc] (inactive)]

all green now. thanks.

Comment 23

[Intermittent Failures Robot]

76 failures in 114 pushes (0.667 failures/push) were associated with this bug yesterday.    

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. ** 

Repository breakdown:
* mozilla-inbound: 40
* autoland: 25
* mozilla-central: 11

Platform breakdown:
* android-hw-p2-8-0-arm7-api-16: 76

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1476315&startday=2018-07-30&endday=2018-07-30&tree=trunk