1038496 - Should control radio power after RILPROXY reconnected

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::System, defect)

Description

[sam.hua]

Attachment: 0-radio-13-35-46.log

1. RILD is dead and reboot.
2. UNSOL_RIL_CONNECTED is sent to gecko.
3. gecko send RADIO_POWER off to RILC.
4. But gecko won't send RADIO_POWER ON to RILC.

except:
RILC send Power on to RILC

Comment 1

[sam.hua]

07-13 14:09:33.495  1479  1484 D use-Rlog/RLOG-AT: [w] Channel0: AT< +CSQ: 31,0
07-13 14:09:33.495  1479  1484 D use-Rlog/RLOG-RILC: [w] [UNSL]< UNSOL_SIGNAL_STRENGTH {[signalStrength=31,bitErrorRate=0,                CDMA_SS.dbm=-1,CDMA_SSecio=-1,                EVDO_SS.dbm=-1,EVDO_SS.ecio=-1,                EVDO_SS.signalNoiseRatio=-1,                LTE_SS.signalStrength=99,LTE_SS.rsrp=2147483647,LTE_SS.rsrq=2147483647,                LTE_SS.rssnr=2147483647,LTE_SS.cqi=2147483647]}
07-13 14:09:33.705   164   164 D use-Rlog/RLOG-RILPROXY: Connecting to socket rild
07-13 14:09:33.705   164   164 D use-Rlog/RLOG-RILPROXY: Connected to socket rild
07-13 14:09:33.705  1479  1480 I use-Rlog/RLOG-RILC: [w] libril: new connection
07-13 14:09:33.705  1479  1480 D use-Rlog/RLOG-RILC: [w] [UNSL]< UNSOL_RIL_CONNECTED {9}
07-13 14:09:33.705  1479  1480 D use-Rlog/RLOG-RILC: [w] [UNSL]< UNSOL_RESPONSE_RADIO_STATE_CHANGED  {RADIO_SIM_READY}
07-13 14:09:33.705  1479  1480 D use-Rlog/RLOG-RILC: [w] [UNSL]< UNSOL_RESPONSE_SIM_STATUS_CHANGED 
07-13 14:09:33.705  1479  1480 I use-Rlog/RLOG-RILC: [w] RIL Daemon version: android reference-ril 1.0
07-13 14:09:33.705  1479  1480 I use-Rlog/RLOG-RILC: [w] enter processCommandsCallback
07-13 14:09:33.705  1479  1480 I use-Rlog/RLOG-RILC: [w] PCC  alloc one command p_record
07-13 14:09:33.715  1479  1482 D use-Rlog/RLOG-RILC: [w] PCB request code 23 token 1465
07-13 14:09:33.715  1479  1482 D use-Rlog/RLOG-RILC: [w] [1465]> RADIO_POWER (0)

Comment 2

[Ken Chang[:kenkai]]

(In reply to sam.hua from comment #0)
> Created attachment 8455859 [details]
> 0-radio-13-35-46.log
> 
> 1. RILD is dead and reboot.
If RILD is dead and rebooted, I think gecko should be rebooted as well.
When the status of RILD is different from the one of gecko, I wonder if it could cause some problems.

Comment 3

[shawn ku [:sku]]

Hi Ken:
 RADIO_POWER request was issued by gaia, gecko only fire RADIO_POWER off when socket get connected.

Meanwhile, since RILD can be dead for some reasons, we need recovery mechanism when hitting this case, (not re-plug-in battery to rescue this problem).

Comment 4

[shawn ku [:sku]]

Attachment: RADIO_POWER on after kill RILD manually, but no command response back.

Comment 5

[shawn ku [:sku]]

Hi Sam:
 Could you please help check why there is no response back for "06-20 21:44:23.610  1115  1118 D use-Rlog/RLOG-RILC: [w] [0137]> RADIO_POWER (1)"

Thanks!!
Shawn

Comment 6

[sam.hua]

Hi Shawn，
RILPROXY reconnect after it
06-20 21:44:51.878   109   109 E use-Rlog/RLOG-RILPROXY: Failed to read from rild socket, closing...
06-20 21:44:51.878   109   109 D use-Rlog/RLOG-RILPROXY: Waiting on socket
06-20 21:44:51.878   961   961 I Gonk    : RIL[0]: OnDisconnect
06-20 21:44:51.888   109   109 D use-Rlog/RLOG-RILPROXY: Socket connected
06-20 21:44:51.888   109   109 D use-Rlog/RLOG-RILPROXY: Connecting to socket rild
06-20 21:44:51.888   109   109 E use-Rlog/RLOG-RILPROXY: Could not connect to rild socket, retrying: Connection refused

Comment 7

[Edgar Chen [:edgar] (Parental leave until Oct. 09.)]

(In reply to shawn ku [:sku] from comment #5)
> Hi Sam:
>  Could you please help check why there is no response back for "06-20
> 21:44:23.610  1115  1118 D use-Rlog/RLOG-RILC: [w] [0137]> RADIO_POWER (1)"
> 
> Thanks!!
> Shawn

(In reply to sam.hua from comment #6)
> Hi Shawn，
> RILPROXY reconnect after it
> 06-20 21:44:51.878   109   109 E use-Rlog/RLOG-RILPROXY: Failed to read from
> rild socket, closing...
> 06-20 21:44:51.878   109   109 D use-Rlog/RLOG-RILPROXY: Waiting on socket
> 06-20 21:44:51.878   961   961 I Gonk    : RIL[0]: OnDisconnect
> 06-20 21:44:51.888   109   109 D use-Rlog/RLOG-RILPROXY: Socket connected
> 06-20 21:44:51.888   109   109 D use-Rlog/RLOG-RILPROXY: Connecting to
> socket rild
> 06-20 21:44:51.888   109   109 E use-Rlog/RLOG-RILPROXY: Could not connect
> to rild socket, retrying: Connection refused

Looks like RILD crash again

06-20 21:44:23.610  1115  1116 I use-Rlog/RLOG-RILC: [w] enter processCommandsCallback
06-20 21:44:23.610  1115  1118 D use-Rlog/RLOG-RILC: [w] PCB request code 23 token 137
06-20 21:44:23.610  1115  1118 D use-Rlog/RLOG-RILC: [w] [0137]> RADIO_POWER (1)
06-20 21:44:23.610  1115  1118 D use-Rlog/RLOG-RIL: [w] onRequest: RADIO_POWER sState=0

06-20 21:44:51.878   961   961 I Gonk    : RIL[0]: OnDisconnect
06-20 21:44:51.888   961   961 I Gonk    : RIL[0]: OnConnectSuccess

// The process id of RILD is changed from 1115 to 1138.
06-20 21:44:51.938  1138  1138 D use-Rlog/RLOG-RILC: [w] it's w modem rild0 libril
06-20 21:44:51.938  1138  1138 E use-Rlog/RLOG-RILC: [w] RIL_register: RIL version 9
06-20 21:44:51.938  1138  1138 I use-Rlog/RLOG-RILC: [w] Ril.cpp: 3 CommandThread create
06-20 21:44:51.938  1138  1140 I use-Rlog/RLOG-RIL: [w] AT channel [0] open successfully

Comment 8

[sam.hua]

Hi .

please send radio_power on to RILC after RILC completed its initialization.
when u find the following logs, it means that RILC has completed initialization.
06-20 21:45:13.109  1138  1143 D use-Rlog/RLOG-AT: [w] Channel1: AT< OK
06-20 21:45:13.109  1138  1139 D use-Rlog/RLOG-AT: [w] Channel1: AT> AT+SPVIDEOTYPE=3
06-20 21:45:13.119  1138  1143 D use-Rlog/RLOG-AT: [w] Channel1: AT< OK
06-20 21:45:13.119  1138  1139 D use-Rlog/RLOG-AT: [w] Channel1: AT> AT+SPDVTDCI="000001B000000001B5090000010000000120008440FA282C2090A21F"
06-20 21:45:13.119  1138  1143 D use-Rlog/RLOG-AT: [w] Channel1: AT< OK
06-20 21:45:13.119  1138  1139 D use-Rlog/RLOG-AT: [w] Channel1: AT> AT+SPDVTTEST=2,650
06-20 21:45:13.129  1138  1143 D use-Rlog/RLOG-AT: [w] Channel1: AT< OK
06-20 21:45:13.129  1138  1139 D use-Rlog/RLOG-AT: [w] Channel1: AT> AT+SPAURC="10011011111000000000100001000011111111"
06-20 21:45:13.129  1138  1143 D use-Rlog/RLOG-AT: [w] Channel1: AT< OK

Comment 9

[shawn ku [:sku]]

Attachment: simply the STR.

1. kill rild
2. wait for SOCKET_CONNECTED
3. GECKO RADIO_POWER (0)
4. wait radioState to off
5. GAIA RADIO_POWER (1) again

Comment 10

[shawn ku [:sku]]

no response back for "[0145]> RADIO_POWER (1)"

06-20 22:53:03.259   106   106 I Gonk    : RIL[0]: OnDisconnect
06-20 22:53:04.290   667   670 D use-Rlog/RLOG-RILC: [w] [0127]> RADIO_POWER (0)
06-20 22:53:04.480   667   670 D use-Rlog/RLOG-RILC: [w] [0127]< RADIO_POWER
06-20 22:53:04.650   106   106 I Gecko   : -*- RadioInterface[0]: Received message from worker: {"rilMessageType":"radiostatechange","radioState":"off","rilMessageClientId":0}
06-20 22:53:04.650   106   106 I Gecko   : -*- RILContentHelper: Received message 'RIL:RadioStateChanged': {"clientId":0,"data":"disabled"}
06-20 22:53:04.720   106   106 I Gecko   : -*- RILContentHelper: Received message 'RIL:RadioStateChanged': {"clientId":0,"data":"enabling"}
...
06-20 22:53:04.750   667   670 D use-Rlog/RLOG-RILC: [w] [0145]> RADIO_POWER (1)

Comment 11

[sam.hua]

1. kill rild
2. wait for SOCKET_CONNECTED
3. turn on airplane mode
4. turn off airplane mode.
5. rilc works

and 
1. turn on airplane mode
2. kill rild/rild_sp/rild_sp
3. turn off airplane mode
4. rilc works

Comment 12

[shawn ku [:sku]]

hi Sam:
 Airplane mode on/off can recover rild crash issue.
In other word, there are manual recover mechanisms by following your steps in comment 11.

No RADIO_POWER (1) response back after kill rild manually is still a question/problem!!

Thanks!!
Shawn

Comment 13

[sam.hua]

Hi Shawn,
I use ur patch and now it can work now.
I can't reproduce the case in the "simply str" log.

Comment 14

[shawn ku [:sku]]

I am not sure which parts went wrong.
Besides, I am currently OOO, not able to try further.

Please request arthur chen's help to check how can we move on.

Comment 15

[Ivan Tsay (:ITsay)]

This is the blocker according to the symptom.

Hi Arthur,

Could you help on this radio issue?

Comment 16

[Arthur Chen [:arthurcc][inactive after 6/30]]

Attachment: PR against master

EJ, Alive, could you help review that patch? The patch was refined from Shawn's patch.

Sam, could you help test the patch again?

Thanks.

Comment 17

[sam.hua]

ok,i will test it today.

Comment 18

[shawn ku [:sku]]

(In reply to Arthur Chen [:arthurcc] from comment #16)
> Created attachment 8458505 [details]
> PR against master
> 
> EJ, Alive, could you help review that patch? The patch was refined from
> Shawn's patch.
> 
> Sam, could you help test the patch again?
> 
> Thanks.

Hi Arthur:
 Only patch apps/system/js/airplane_mode.js seems not work because radio.js will do the state check too, the setter of "window.Radio.enabled = true;" will be ignored (if my memory is correct).

Please correct me if anything is wrong.

Comment 19

[Arthur Chen [:arthurcc][inactive after 6/30]]

Comment on attachment 8458505 [details]
PR against master

Clear the review request per comment 18. Thanks for pointing that out, Shawn!

Comment 20

[Hsin-Yi Tsai (she/her) [:hsinyi]]

Looks like a gaia system issue.

Comment 21

[sam.hua]

Hi Arthur.
any difference between your patch and shawn's?

Comment 22

[Arthur Chen [:arthurcc][inactive after 6/30]]

Per comment 18, I'll propose another patch doing modifications on system/js/radio.js. And this should be a gaia issue.

Comment 23

[Keven Kuo [:kkuo]]

Hi! Arthur,

Any progress?

--
Keven

Comment 24

[Arthur Chen [:arthurcc][inactive after 6/30]]

The new patch is ready. However, it did not succeed every time when I re-enable the radio from gaia. Sam, could you help check from the platform side using the PR? Thanks.

Comment 25

[shawn ku [:sku]]

(In reply to Arthur Chen [:arthurcc] from comment #24)
> The new patch is ready. However, it did not succeed every time when I
> re-enable the radio from gaia. Sam, could you help check from the platform
> side using the PR? Thanks.

That's the similiar result as my previous trial.
I guess there might be some thing missed regrading modem.

Comment 26

[sam.hua]

Hi Shawn,

Please give me the new patch and i will check it.

Comment 27

[Arthur Chen [:arthurcc][inactive after 6/30]]

I've updated the patch to the original PR: https://bugzilla.mozilla.org/attachment.cgi?id=8458505

Comment 28

[Evelyn Hung]

ni Sam for the result of testing the patch.

Comment 29

[thomas tsai]

Hi Sam and James, how about the test result?

Comment 30

[Brogan Zumwalt [Inactive]]

Not enough information with current STR to write test case to address bug.

Comment 31

[sam.hua]

it could work in dolphin.

Comment 32

[shawn ku [:sku]]

Hi Arthur:
 Could you please help proceeding review/land code process since partner claims the patch works on 1.4?

Thanks!
Shawn

Comment 33

[Arthur Chen [:arthurcc][inactive after 6/30]]

Comment on attachment 8458505 [details]
PR against master

The patch has been proved works per comment 31. Alive, could you help review the patch? Thanks.

Comment 34

[(inactive after 6/18) Alive Kuo [:alive]]

Comment on attachment 8458505 [details]
PR against master

Looks ok

Comment 35

[Arthur Chen [:arthurcc][inactive after 6/30]]

Thanks.

master: 0de152ddeae5a5efc30ddf6393dd4253967c5141

Comment 36

[Ryan VanderMeulen [:RyanVM]]

v1.4: https://github.com/mozilla-b2g/gaia/commit/f63cdae1a06c808443ed14de3cd13b61311426e0

Please nominate this for v2.0 uplift if it's needed there as well (1.4 blocking status doesn't grant auto-approval).

Comment 37

[Arthur Chen [:arthurcc][inactive after 6/30]]

Comment on attachment 8458505 [details]
PR against master

NOTE: Please see https://wiki.mozilla.org/Release_Management/B2G_Landing to better understand the B2G approval process and landings.

[Approval Request Comment]
[Bug caused by] (feature/regressing bug #): N/A
[User impact] if declined: Users are not able to connect to network after rild crashes.
[Testing completed]: Manual tests and unit tests.
[Risk to taking this patch] (and alternatives if risky): None
[String changes made]: None

Comment 38

[Ryan VanderMeulen [:RyanVM]]

v2.0: https://github.com/mozilla-b2g/gaia/commit/aeb80a140fd2c29a62298658071d0ccb20063307

Comment 39

[Anshul]

Bhavna, this is a dolphin specific feature and should only be present in the dolphin only branch. This change is causing a regression on FFOS 2.0 and FFOS 2.1 in our branches caught by our unit test framework. Request you to be please back this out from 2.0, 2.1 and moz central.

Comment 40

[shawn ku [:sku]]

Hi Anshul:
 I don't think this is Dolphin only patch.

We will need this once rild crash and restart (AOSP did this in GsmServiceStateTracker.java),
besides, we may need this for SIM_REFRESH RESET case (AOSP did this in SIMRecord.java).

Please correct me if anything is wrong.
Shawn

Comment 41

[Anshul]

(In reply to shawn ku [:sku]  (OOO 9/15 ~ 9/19) from comment #40)
> Hi Anshul:
>  I don't think this is Dolphin only patch.
> 
> We will need this once rild crash and restart (AOSP did this in
> GsmServiceStateTracker.java),
> besides, we may need this for SIM_REFRESH RESET case (AOSP did this in
> SIMRecord.java).
> 
> Please correct me if anything is wrong.
> Shawn
Shawn, please see bug 1067629 for details.

Comment 42

[Ed Morley [:emorley]]

At Fabrice's request, I've reverted this on:

2.1: https://github.com/mozilla-b2g/gaia/commit/47939f4c41d0c941e5047e5d1af74a79b7d8e0d5
2.0: https://github.com/mozilla-b2g/gaia/commit/95314e27da38b8c4edf965a77eded16cc5729ea8
2.0m: https://github.com/mozilla-b2g/gaia/commit/4d5a460d4ff71aecc035bd7c4a58619506ad822c

However it is still landed on master.

Comment 43

[shawn ku [:sku]]

(In reply to Anshul from comment #41)
> (In reply to shawn ku [:sku]  (OOO 9/15 ~ 9/19) from comment #40)
> > Hi Anshul:
> >  I don't think this is Dolphin only patch.
> > 
> > We will need this once rild crash and restart (AOSP did this in
> > GsmServiceStateTracker.java),
> > besides, we may need this for SIM_REFRESH RESET case (AOSP did this in
> > SIMRecord.java).
> > 
> > Please correct me if anything is wrong.
> > Shawn
> Shawn, please see bug 1067629 for details.

Anshul, thanks for the information.

Comment 44

[Ryan VanderMeulen [:RyanVM]]

Why was this backed out from v2.1? It was landed on master during the 2.1 development cycle. Also, QC cares about 2.0m now?

Comment 45

[EJ Chen [:eragonj][:小龍哥][Contributor mode on](inactive)]

@Morley

Why this patch got backed out so accidentally on these branches ? I can't see any discussion about backing it out (even on bug 1067629) and can you share more information here ? 

Without this patch, we are losing the ability to recover Rild, so do we already have another way to do this without Gaia's help already ? If so, why we still keep this patch intact on master ?

Any information would help.

Thanks !

Comment 46

[bhavana bajaj [:bajaj]]

(In reply to EJ Chen [:eragonj][:小龍哥] from comment #45)
> @Morley
> 
> Why this patch got backed out so accidentally on these branches ? I can't
> see any discussion about backing it out (even on bug 1067629) and can you
> share more information here ? 
> 
> Without this patch, we are losing the ability to recover Rild, so do we
> already have another way to do this without Gaia's help already ? If so, why
> we still keep this patch intact on master ?
> 
> Any information would help.
> 
> Thanks !

Ej, I think Ed just lived upto a request that was asked out of him and will not know the answers here.

We decided to backout from branches based on https://bugzilla.mozilla.org/show_bug.cgi?id=1067629#c2 here. The agreement was to do a straight backout on 2.0 and see what it took to resolve this in 2.1 after talking to CAF given its post FL for 2.1 and they rely on changes like which is breaking testing.

But since we could not do the backouts in-time taipei time, Ed did all the necessary backouts across all branches given we cause a regression(irrespective of who it impacts or not). This may have to re-land on 2.0M and 2.1 and Ken's input was we will forward fix once we have the full solution and more clarity here.

Feel free to reach-out to Ken directly here as needed.

Comment 47

[EJ Chen [:eragonj][:小龍哥][Contributor mode on](inactive)]

Thanks :bajaj to clarify my questions in person !

Comment 49

[Ken Chang[:kenkai]]

Change the summary. 
After discussing with teams, we think this is a common issue. And based on our design, this is a right fix and should be landed. Let's discuss it more in bug 1067629.

Comment 50

[Josh Cheng [:josh]]

Hi Kai-Zhen,
2.0M+ please help to land on 2.0M branch. Thanks!

Comment 51

[Kai-Zhen Li [:seinlin][:kli]]

https://github.com/mozilla-b2g/gaia/commit/ff24db33f8ebb275b200030035b851701b617b80