1188935 - Autophone - Intermittent Java.lang.NullPointerException at android.net.nsd.NsdManager$ServiceHandler.handleMessage(NsdManager.java:350)

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: General, defect)

Description

[Bob Clary [:bc] (inactive)]

Attachment: Nexus 4 Stack

An example is:

Nexus 4 Android 4.2 s1s2-blank-remote

https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=237e7aac9def&exclusion_profile=false&filter-searchStr=autophone

https://autophone.s3.amazonaws.com/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android-api-11/1438129122/autophone-s1s2-s1s2-blank-remote.ini-1-nexus-4-jdq39-2-f6f4c4a4-7f12-48cf-9bd8-11641fe3a7c6-logcat.log

I attempted a manual search using Treeherder to find the initial regression range. It does appear this may be limited to Nexus 4 Android 4.2 but I am not certain. I am downloading the logs now and will attempt to identify when this first appeared.

Comment 1

[Geoff Brown [:gbrown]]

Bug 1158029 recently landed changes related to NsdManager.

Comment 2

[Bob Clary [:bc] (inactive)]

The first mozilla-inbound build where this was seen was https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-searchStr=autophone&exclusion_profile=false&revision=e58aba8b984b

http://autophone.s3.amazonaws.com/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android-api-11/1437410848/autophone-s1s2-s1s2-blank-remote.ini-1-nexus-4-jdq39-2-488865d9-b6ef-449b-b628-2a2dbf1366aa-logcat.log

I've only seen this 5 times in July so that is consistent with Bug 1158029. I have also only seen this on Nexus 4 Android 4.2

Gary, do you know what might be going on?

Comment 3

[Bob Clary [:bc] (inactive)]

Gary, can we back this out due to the failures in Autophone?

Comment 4

[Gary Chen [:xeonchen]]

Yes, Bob. Thank you.

Comment 5

[Bob Clary [:bc] (inactive)]

This appears to have gotten better Friday afternoon/evening. See bug 1190000 comment 4. Lets wait a little bit to see how it plays out.

Comment 8

[Gary Chen [:xeonchen]]

I found a similar issue was reported as an Android bug[1].
Although it is marked as released, but still reported reproducible on Android 4.4.

Another discussion[2] also mentioned about this issue.

I don't have a strong evidence yet to support they are the same.
I'm trying to see if I can have a workaround on this,
or maybe make this feature disabled if we cannot fix it.

[1] https://code.google.com/p/android/issues/detail?id=35585
[2] http://stackoverflow.com/questions/16666604/internal-error-using-network-service-discovery-in-android

Comment 9

[Kartikaya Gupta (email:[e-mail])]

Thanks for the links. Based on those I rebooted my phone and now Nightly starts up again.

Comment 10

[Bob Clary [:bc] (inactive)]

Gary, if we can't find a fix, please lets go ahead and either disable this or back out both patches which landed regarding nsd.

Comment 11

[Gary Chen [:xeonchen]]

According to [1] and [2], this issue should have been fixed in Android 4.4.3_r1

[1] https://android.googlesource.com/platform/frameworks/base/+/3fc376b7336cfbddbb10ce3f93a853a927f6513a
[2] https://android.googlesource.com/platform/frameworks/base/+/android-4.4.3_r1/core/java/android/net/nsd/NsdManager.java

Comment 12

[Gary Chen [:xeonchen]]

In Comment 11, I suppose the NullPointerException is a bug of Android, this might affects almost all Android 4.x devices.

It's not a issue of garbage collection, and the call stack is in the Android code base,
so we can't just have a workaround of it.

Richard,
do you think we should enable this feature only to 5.x and after (API level >= 21)?

Bob,
We don't have autophone test on Android 5.x platform, do we?

Comment 13

[Bob Clary [:bc] (inactive)]

Not on 5 yet. But that is coming soon.

Comment 14

[Richard Newman [:rnewman]]

(In reply to Gary Chen [:xeonchen] from comment #12)

> so we can't just have a workaround of it.

There seem to be *some* workarounds, but not good or complete ones. It's up to you whether it's worth investigating those.

> Richard,
> do you think we should enable this feature only to 5.x and after (API level
> >= 21)?

5.x currently has a little less than 20% of the market. But given that this causes crashes on the rest, there's not too much we can do. Yup, turn it off for earlier versions. You'll need to use the Versions.* conditional, and make sure consumers fail fast. 

:/

Comment 15

[Gary Chen [:xeonchen]]

(In reply to Richard Newman [:rnewman] from comment #14)
> There seem to be *some* workarounds, but not good or complete ones. It's up
> to you whether it's worth investigating those.

Although it may not change the result of this issue, I'd like to know about that very much.
Would you give some possible workarounds if it won't take too much time?

Comment 16

[Gary Chen [:xeonchen]]

Attachment: Part 1: enable NsdMulticastDNSManager only if API level >= 21

Comment 17

[Gary Chen [:xeonchen]]

Attachment: Part 2: enable mDNS module on Android only if API level >= 21

Comment 18

[Gary Chen [:xeonchen]]

Attachment: Part 3: register MulticastDNSDeviceProvider on Android only if API level >= 21

Comment 19

[Richard Newman [:rnewman]]

(In reply to Gary Chen [:xeonchen] from comment #15)
> Although it may not change the result of this issue, I'd like to know about
> that very much.
> Would you give some possible workarounds if it won't take too much time?

https://code.google.com/p/android/issues/detail?id=35585 has some suggestions to explore:

* Use UDP if the protocol allows it. (Untested.)
* Don't stop discovery soon after a service has been discovered.
* Add a delay between fetching NsdManager and using it.

Comment 20

[Kartikaya Gupta (email:[e-mail])]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #9)
> Thanks for the links. Based on those I rebooted my phone and now Nightly
> starts up again.

FWIW today my Fennec got into a startup crash loop again. I'm not really sure when it happened (I unlocked my device at some point and was greeted with the crash reporter dialog) but I suspect it happened while I was outside and disconnected from wi-fi. Rebooting the device fixed it again.

Comment 21

[Gary Chen [:xeonchen]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=10326e6202a9&exclusion_profile=false

Comment 22

[Pulsebot]

https://hg.mozilla.org/integration/fx-team/rev/60c13eb01a6c
https://hg.mozilla.org/integration/fx-team/rev/a338aeb2e9be
https://hg.mozilla.org/integration/fx-team/rev/67b330295145

Comment 23

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/60c13eb01a6c
https://hg.mozilla.org/mozilla-central/rev/a338aeb2e9be
https://hg.mozilla.org/mozilla-central/rev/67b330295145