1084023 - SIM randomly gets permanently stuck in "unknown" state, survives reboots

Status: RESOLVED FIXED

(Firefox OS Graveyard :: RIL, defect)

Description

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

This has happened to me twice in the last two weeks.  I get the SIM card stuck in an "unknown" state, at which point I can't make phone calls or send SMSs.  Rebooting the device has no effect.  My data connection still works though.

When this happens, if I do a full flash and restore profile data from before this happened everything works again.  If I do a full flash and restore profile data from after it happens it is still broken.  And if I restore known good profile data after it happens without doing a full flash it is still broken.

Comment 1

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Attachment: changed files in my profile

This is git diff --stat good bad run on the git repository that has my b2g profile stored in it.

Comment 2

[Hsin-Yi Tsai (she/her) [:hsinyi]]

Discussed with teams but cannot thinking of anything obviously suspicious. Do you think you are able to provide us log with ril messages? https://wiki.mozilla.org/B2G/QA/Tips_And_Tricks#RIL_Debugging

Comment 3

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Attachment: RIL Log

Comment 4

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Did you find anything in the log?  This happened to me again tonight, and it's super annoying.

Comment 5

[Edgar Chen [:edgar]]

I found some interesting things in your log, the timing of some messages is suspicious, but I still can not figure out what is the root cause of 'unknown' state. 

Would you mind help to provide the normal log for us to compare (device boots up log)? And have you tried to switch airplane mode on/off when it happens?

Thank you.

ni? me for tracking.

Comment 6

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Attachment: good-ril.log

Comment 7

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Attachment: bad-ril.log

Comment 8

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

(In reply to Edgar Chen [:edgar][:echen] from comment #5)
> Would you mind help to provide the normal log for us to compare (device
> boots up log)? And have you tried to switch airplane mode on/off when it
> happens?

Airplane mode doesn't help, and neither does rebooting the device.  The only way to fix it is to reflash from scratch.

I narrowed the "broken" part of the profile down to the settings database.  I can provide a copy of that if it would be helpful.

Comment 9

[Edgar Chen [:edgar]]

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #8)
> (In reply to Edgar Chen [:edgar][:echen] from comment #5)
> > Would you mind help to provide the normal log for us to compare (device
> > boots up log)? And have you tried to switch airplane mode on/off when it
> > happens?
> 
> Airplane mode doesn't help, and neither does rebooting the device.  The only
> way to fix it is to reflash from scratch.
> 
> I narrowed the "broken" part of the profile down to the settings database. 
> I can provide a copy of that if it would be helpful.

Having more information is always helpful. :)
Thank you, Kyle.
Rg

Comment 10

[Edgar Chen [:edgar]]

Comment on attachment 8509723 [details]
bad-ril.log

Ah, somehow device switches to cdma mode, so unable to recognize a gsm card.

> I/Gecko   (  185): RIL Worker: [0] Handling parcel as REQUEST_VOICE_RADIO_TECH
> I/Gecko   (  185): RIL Worker: [0] Radio tech is set to: is95a, it is a cdma technology

Device reports card state as 'unknown'.

> I/Gecko   (  185): -*- RadioInterface[0]: Received message from worker: {"rilMessageType":"cardstatechange","cardState":0,"rilMessageClientId":0}
> I/Gecko   (  185): -*- RILContentHelper: Received message 'RIL:CardStateChanged': {"clientId":0,"data":{"rilMessageType":"cardstatechange","cardState":0,"rilMessageClientId":0}}

Comment 11

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Any progress?

Comment 12

[Edgar Chen [:edgar]]

Somehow device is initialized in cdma mode.
And then sends an unsolicited event switching to gsm mode.
(This unsolicited event is usually used for world phone)

But moz ril doesn't support dynamical switching phone mode yet [2].

> I/Gecko   (  185): RIL Worker: [0] Handling parcel as REQUEST_VOICE_RADIO_TECH
> I/Gecko   (  185): RIL Worker: [0] Radio tech is set to: is95a, it is a cdma technology
> 
> I/Gecko   (  185): RIL Worker: [0] Unsolicited response for request type 1035
> I/Gecko   (  185): RIL Worker: Parcel handler didn't consume whole parcel, 8 bytes left over

Probably because the preferred network type is set to `lte/wcdma/gsm/cdma/evdo`? But in good-ril.log, device has the same configuration and it works good. Have no idea why.

> I/Gecko   (  185): RIL Worker: [0] Received chrome message {"type":"lte/wcdma/gsm/cdma/evdo","rilMessageClientId":0,"rilMessageToken":10,"rilMessageType":"setPreferredNetworkType"}

Could you try to set system property, `ro.moz.ril.0.network_types`, to `gsm,wcdma,lte` to see if it helps? 
This can hide the cdma options in Gaia (you probably still need to clear settings database).

Thank you.

[1] https://github.com/mozilla-b2g/platform_hardware_ril/blob/master/include/telephony/ril.h#L3801-L3811
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=866038

Comment 13

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

So I didn't make any of the changes from comment 12 yet, but I also haven't seen this in several weeks.  Will update if I see it again.

Comment 14

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

This happened again with the network set to just LTE in the Gaia::Settings.  Next time it happens I will try setting the system property.

Comment 15

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Messing with the network types manually doesn't seem to help.

I've gone back to Android for the time being.

Comment 16

[Kevin Grandon :kgrandon]

Is there anything else we can do here? It sucks to be losing dogfooders =( I was also considering switching to using a nexus5, so this is super painful to see.

Comment 17

[Edgar Chen [:edgar]]

Hmm, looks like restricting network types doesn't help too much here. Let me provide a quick patch for supporting RIL_UNSOL_VOICE_RADIO_TECH_CHANGED to see if it fixes this annoying issue. Keeps ni? to me for tracking.

Comment 18

[Hsin-Yi Tsai (she/her) [:hsinyi]]

(In reply to Kevin Grandon :kgrandon (In Europe/Conf until 11/24) from comment #16)
> Is there anything else we can do here? 

Discussed with Edgar, we have some ideas in mind. He will first provide a proof-of-concept patch to update sim info when radio tech is somewhat switched, and see how thing goes. However, the gaia implementation of handling these update events might be a concern, not sure if we need gaia work as well, but we will see. 

> It sucks to be losing dogfooders =( I
> was also considering switching to using a nexus5, so this is super painful
> to see.

Understood. This is also painful for us to deal with different modem behaviours :( I don't want to see we lose dogfooders, and I am very thankful of you being using mozril and reporting drawbacks. But please kindly understand we have such limited bandwidth as we have to take care of the reference phone (and partner devices) first, so the progress might be relatively slow. :(

Comment 19

[Edgar Chen [:edgar]]

Attachment: Patch, v1

I verified this patch in emulator by hacking it to simulate this behaviour, it works good. But I still would like to have a test in real device.
Hi Kyle, could you help to test this patch for me to see if it fixes the issue? Thank you.

*Note that this patch is just a quick solution for supporting RIL_UNSOL_VOICE_RADIO_TECH_CHANGED in order to fix this issue, we still need a full solution for world phone (bug 866038).

Comment 20

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

The earliest I could test this would be after Portland, and it usually requires several days of use for the issue to manifest.  If you think it is likely to fix the problem please just get it reviewed and landed.

Comment 21

[Edgar Chen [:edgar]]

Comment on attachment 8526715 [details] [diff] [review]
Patch, v1

Hi Hsinyi, may I have your review? Thank you.

Comment 22

[Hsin-Yi Tsai (she/her) [:hsinyi]]

Comment on attachment 8526715 [details] [diff] [review]
Patch, v1

Review of attachment 8526715 [details] [diff] [review]:
-----------------------------------------------------------------

Thank you :)

::: dom/system/gonk/ril_worker.js
@@ +6972,5 @@
>    this.setRadioEnabled({enabled: false});
>  };
> +RilObject.prototype[UNSOLICITED_VOICE_RADIO_TECH_CHANGED] = function UNSOLICITED_VOICE_RADIO_TECH_CHANGED(length) {
> +  // This unsolicited response will be sent when the technology of a multi-tech
> +  // modem is changed, ex. switch between gsm and cdma. Currently the only known

The line of "Currently the only know..." doesn't look 100% right, nexus 5 is an example. Could we remove or revise this? Thank you.

@@ +6975,5 @@
> +  // This unsolicited response will be sent when the technology of a multi-tech
> +  // modem is changed, ex. switch between gsm and cdma. Currently the only known
> +  // multi-tech modem is emulator (qemu can support both gsm and cdma).
> +  // TODO: We may need to reset some data when switching between gsm and cdma
> +  //       mode, like IMEI, ESN ... etc. See Bug 866038.

With Bug 1110619, we don't need to worry about updates of IMEI, ESN, etc.

The patch fixes the accidental change, and agree that to fully support world phone case, we need  to do more on updating iccInfo and iccType.

Comment 23

[Hsin-Yi Tsai (she/her) [:hsinyi]]

This is not a regression but triggered by different device behaviour.

Comment 24

[Edgar Chen [:edgar]]

Attachment: Patch, v2, r=hsinyi

Address review comment #22.

Comment 25

[Edgar Chen [:edgar]]

Try result looks good: https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=c2cc4259ee80

Comment 26

[Edgar Chen [:edgar]]

https://hg.mozilla.org/integration/b2g-inbound/rev/776f432e007d

Comment 27

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/776f432e007d