1042866 - MSISDN need to use a virtual long number to validate US/Canadian MSISDN numbers.

Status: VERIFIED FIXED

(Cloud Services Graveyard :: MobileID, defect, P1)

Description

[Edwin Wong [:edwong]]

Attachment: msisdn_log.txt

* flash b2g 2.0 branch to Flame device with SIM card
* use app-manager to install this app
* launch Loop, swipe to the right till "Let's start"
* tap to share
* tap 'use phone number'
* click share on phone number

actual: 'connecting' just spins, if you click 'X' to close, you see authenticating spinner

see attached log with debug on

Comment 1

[Alexis Metaireau (:alexis)]

Just a note: this script is using staging msisdn: https://github.com/ferjm/msisdn-verifier-client/blob/master/js/client_request_helper.js#L8

Comment 2

[Edwin Wong [:edwong]]

https://github.com/ferjm/msisdn-verifier-client isn't working for me either, on device or via web

I used these values: "mcc":"310","mnc":"410","msisdn":"MY_NUMBER"

Comment 3

[Rémy Hubscher (:natim)]

We need to investigate this then.
Do you have any useful information in the console when using the verifier?

Comment 4

[Edwin Wong [:edwong]]

[Blocking Requested - why for this release]:tchung asked me to mark this as a blocker if there's any gecko/gaia code

Comment 5

[Edwin Wong [:edwong]]

this is what i'm getting when i do it via web - which is using dev.  I'm not sure if the moVerifier is hard coded on the web app:
http://msisdn.dev.mozaws.net/app/
i used:
mcc: 310
mcn: 070 for ATT
my number: 1650xxxxxxx
"Discover: {"verificationMethods":["sms/mt","sms/momt"],"verificationDetails":{"sms/mt":{"mtSender":"Mozilla@","url":"https://msisdn-dev.stage.mozaws.net/sms/mt/verify"},"sms/momt":{"mtSender":"Mozilla@","moVerifier":"[phone]"}}}" app.js:103
"Register {"msisdnSessionToken":"0f9ddecdf188e7cee60435938fa2689c50669c308cb2e6bbd9f9694ae6526170"}" app.js:154
"Verify {}"

no sms was recieved by phone

Comment 6

[Rémy Hubscher (:natim)]

Your number is not well formated. It should be +1...

Comment 7

[Edwin Wong [:edwong]]

let me try that

Comment 8

[Edwin Wong [:edwong]]

hmm still not working. In the USA we don't use the '+' in front of the number.

this is my network requests:
 00:55:04.415	0.062	485	254	OPTIONS	204	text/plain	https://msisdn-dev.stage.mozaws.net/discover
 00:55:04.515	0.079	575	556	POST	200	application/json	https://msisdn-dev.stage.mozaws.net/discover
 00:55:38.541	0.067	485	254	OPTIONS	204	text/plain	https://msisdn-dev.stage.mozaws.net/register
 00:55:38.645	0.077	494	331	POST	200	application/json	https://msisdn-dev.stage.mozaws.net/register
 00:55:38.772	0.057	504	254	OPTIONS	204	text/plain	https://msisdn-dev.stage.mozaws.net/sms/mt/verify
 00:55:38.867	0.296	745	288	POST	200	application/json https://msisdn-dev.stage.mozaws.net/sms/mt/verify


i see this in the last post:
{"msisdn":"+1650946xxxx","shortVerificationCode":true}

Comment 9

[Edwin Wong [:edwong]]

can you look at the logs on https://msisdn-dev.stage.mozaws.net to see if a SMS was attempted to send

Comment 10

[Rémy Hubscher (:natim)]

I think you are missing a +1 in front of your number then, this is MSISDN it is not your number, it is the international form of your number.

    [phone]xxxx

Comment 11

[Rémy Hubscher (:natim)]

I have even better the SMS Gateway logs which are telling: https://docs.nexmo.com/index.php/sms-api/send-message#response_code

    15 	Invalid sender address 	The sender address (from parameter) was not allowed for this message. Restrictions may apply depending on the destination see our FAQs

Comment 12

[Edwin Wong [:edwong]]

so that sounds like the sender address, which is built into the app:
{"verificationMethods":["sms/mt","sms/momt"],"verificationDetails":{"sms/mt":{"mtSender":"Mozilla@","url":"https://msisdn.services.mozilla.com/sms/mt/verify"},"sms/momt":{"mtSender":"Mozilla@","moVerifier":"[phone]"}}}

But the big issue here, is that the firefoxos-loop-app on my device isn't getting authenticated via SMS. From the log, there doesn't appear to be any client errors, so we should check the nexmo or server logs.  I can give you my phone number via IRC or email

Comment 13

[Rémy Hubscher (:natim)]

   For messages to the US or Canada, the sender should be a virtual long number

Then you are right, we need to fix that.

Comment 14

[Rémy Hubscher (:natim)]

:bobm can you configure the mtSender for msisdn.services.mozilla.com to 12182967993

Comment 15

[Rémy Hubscher (:natim)]

:edwong, I have configured MSISDN-DEV for that can you try again here with your number?

   https://msisdn-dev.stage.mozaws.net/app/

Comment 16

[Rémy Hubscher (:natim)]

:bobm forget my request, because in that case it will breaks other countries...


To implement the mtSender selection wrt MNC/MCC we've got a problems.

On the /discover endpoint we are giving a mtSender number based on MCC/MNC codes.

But

1. /discover doesn't requires hawkAuth
2. /sms/mt/verify don't ask for MCC/MNC codes

So far it wasn't a big deal because the mtSender was the same for all countries but there is some restrictions for US and Canada that only accepts a virtual US number.

I even think we may need to use short codes for sending SMS in the US.

Anyway we need to change the API to:

1. either ask for hawk auth on /discover to remember what is the last mtSender we gave to that user. (then it will be mandatory to call /discover after /register)
2. either ask for MCC/MNC on POST /sms/mt/verify

Luckily SMS /sms/momt/verify is already returning us the network-code (which is the MCCMNC unique id)

I think the best path here is to ask for MCC/MNC code on POST /sms/mt/verify and default to the default mtSender in case they are missed to not breaks current clients.

Also note that if they are missed (as for current implementation), the MSISDN will not works for US and Canada.

Comment 17

[Rémy Hubscher (:natim)]

Well for nexmo we can receive the network-code it: https://docs.nexmo.com/index.php/sms-api/handle-inbound-message
But not for beepsend: http://api.beepsend.com/docs.html#callback-mo

Comment 18

[Fernando Jiménez Moreno [:ferjm]]

(In reply to Rémy Hubscher (:natim) from comment #16)
> 
> I think the best path here is to ask for MCC/MNC code on POST /sms/mt/verify
> and default to the default mtSender in case they are missed to not breaks
> current clients.
> 

FWIW this is the easiest change for the client.

Requiring the hawk id in /discover means that we need to call to /register before and that's quite a big change in the client logic.

Comment 19

[Edwin Wong [:edwong]]

:natim
i tried this with my AT&T flame:
1. on desktop fx goto: https://msisdn-dev.stage.mozaws.net/app/
2. enter the following
msisdn: {165094671xx, [phone]xx, [phone]xx} (I tried all three numbers)
mcc: 310
mnc: 070

no SMS was recieved by phone.  I setup a trial nexmo acct and was able to receive a sms from them to setup an acct.

Comment 20

[Tony Chung [:tchung]]

triage discussed that using the blocking-b2g flag may not be the best way to track server-side owned bugs.  Will take the triage portion of this offline between Jason and Edwin to discuss steps.  meanwhile, who should this bug be assigned to to get a fix?

Comment 22

[Jason Smith [:jsmith]]

From what I understand, this bug is a test blocker for QA as it prevents us from testing MSISDN in US & Canada. On that regard, I would block on this as a NPOTB issue needing to be fixed to unblock QA testing.

Comment 23

[James Bonacci [:jbonacci]]

:tchung the loop-server dev team will have to fix it, so probably :natim
:jsmith I agree

Comment 24

[Bob Micheletto [:bobm]]

(In reply to Rémy Hubscher (:natim) from comment #14)
> :bobm can you configure the mtSender for msisdn.services.mozilla.com to
> 12182967993

Leaving alone per comment 16.

Comment 25

[Chris Karlof [:ckarlof]]

> I think the best path here is to ask for MCC/MNC code on POST /sms/mt/verify and default to the default mtSender in case they are missed to not breaks current clients.

Fine with me as well.

Comment 26

[Rémy Hubscher (:natim)]

Ok thank you guys, I will implement this today.

    Ask for MCC/MNC code on POST /sms/mt/verify

Comment 27

[Fernando Jiménez Moreno [:ferjm]]

(In reply to Jason Smith [:jsmith] from comment #22)
> From what I understand, this bug is a test blocker for QA as it prevents us
> from testing MSISDN in US & Canada. On that regard, I would block on this as
> a NPOTB issue needing to be fixed to unblock QA testing.

This requires also a client side change that needs to land on 2.0, otherwise, we won't have Mobile ID for US or Canada.

Comment 28

[Fernando Jiménez Moreno [:ferjm]]

I filed Bug 1044792 for the client change.

Comment 29

[Rémy Hubscher (:natim)]

Attachment: Link to github PR

Comment 30

[Fernando Jiménez Moreno [:ferjm]]

Comment on attachment 8463289 [details] [review]
Link to github PR

Thanks Rémy!

Comment 31

[Alexis Metaireau (:alexis)]

Pushed at https://github.com/mozilla-services/msisdn-gateway/commit/14fefe1b8b298721aaebd5338344db5768097da2

Comment 32

[Ryan VanderMeulen [:RyanVM]]

Is this fixed for v2.0 as well? It's not clear to me what branches serve which versions in that repo.

Comment 33

[Rémy Hubscher (:natim)]

I think so yes.

Comment 34

[Fernando Jiménez Moreno [:ferjm]]

Yes, this is fixed on 2.0

Comment 35

[William Hsu [:whsu]]

Can it be tested in Asia?

There is no response while I tap "Use phone number" button or "Use Firefox Accounts" button.
Demo video: LOOP_Client_test.mp4

* Build Information:
 - Gaia        7edd3b0b9f65c3dde235c732d270e43e055a1254
 - Gecko       https://hg.mozilla.org/releases/mozilla-b2g32_v2_0/rev/13e04ab68621
 - BuildID     20140914162307
 - Version     32.0
 - Base image  Flame KK
 - ro.build.version.incremental=27
 - ro.build.date=Thu Sep  4 14:59:02 CST 2014

--- -- - --- -- - --- ADB logcat-- - --- -- - --- -- - --- -- -
V/WLAN_PSA(  215): NL MSG, len[048], NL type[0x11] WNI type[0x5050] len[028]
I/Gecko   (  211): 1410782789754	MobileId	ERROR	MobileIdentityClient -> Error {"code":400,"errno":108,"error":"Missing mcc"}
I/Gecko   (  211): 1410782789772	MobileId	ERROR	getMobileIdAssertion rejected with INVALID_BODY_MISSING_PARAMS
I/Gecko   (  211): 1410782789772	MobileId	ERROR	UI error INVALID_BODY_MISSING_PARAMS
E/GeckoConsole( 2372): Content JS LOG at app://loop.services.mozilla.com/js/helpers/account_helper.js:151 in _onloginerror: onloginerror [object DOMError]
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
D/QSEECOMAPI: ( 2473): QSEECom_get_handle sb_length = 0x2000
D/QSEECOMAPI: ( 2473): App is not loaded in QSEE
E/QSEECOMAPI: ( 2473): Error::Cannot open the file /vendor/firmware/keymaster/keymaster.mdt
E/QSEECOMAPI: ( 2473): Error::Loading image failed with ret = -1
D/QSEECOMAPI: ( 2473): QSEECom_get_handle sb_length = 0x2000
D/QSEECOMAPI: ( 2473): App is not loaded in QSEE
I/cat     (  261): <4>[  801.528830] QSEECOM: qseecom_release: data: released = false, type = 1, data = 0xec3a8480
E/QSEECOMAPI: ( 2473): Error::Cannot open the file /firmware/image/keymaste.mdt
E/QSEECOMAPI: ( 2473): Error::Loading image failed with ret = -1
I/cat     (  261): <4>[  801.536455] QSEECOM: qseecom_release: data: released = false, type = 1, data = 0xec3a8480
E/QCOMKeyMaster( 2473): Loading keymaster app failied
E/keystore( 2473): could not open keymaster device in keystore (Operation not permitted)
E/keystore( 2473): keystore keymaster could not be initialized; exiting
D/QLOC-OSAgent(  211): Subscriber already registerd XTWiFi-PE
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
I/qdhwcomposer(  211): prepare: Number of App layers exceeded the limit 
D/wpa_supplicant( 2090): RX ctrl_iface - hexdump(len=11): 53 49 47 4e 41 4c 5f 50 4f 4c 4c
D/wpa_supplicant( 2090): wlan0: Control interface command 'SIGNAL_POLL'
D/wpa_supplicant( 2090): nl80211: survey data missing!

Comment 36

[William Hsu [:whsu]]

Attachment: LOOP_Client_test.mp4

Comment 37

[Rémy Hubscher (:natim)]

Can you test using msisdn-cli[0] like so:

    msisdn-cli -H https://msisdn.services.mozilla.com -c 460

[0] https://github.com/mozilla-services/msisdn-cli

Comment 38

[Fernando Jiménez Moreno [:ferjm]]

William, it would also be great if you could retry enabling the Mobile ID debug prefs:

user_pref("services.mobileid.loglevel", "Debug");
user_pref("services.hawk.loglevel", "Debug");
user_pref("services.hawk.log.sensitive", true); 

This should add enough information to the logcat to let us understand why the requests are failing.

Also, you can test the mobile ID flow with the "UI Test - privileged" app to discard that this isn't a Loop specific issue.

Comment 39

[William Hsu [:whsu]]

(In reply to Rémy Hubscher (:natim) from comment #37)
> Can you test using msisdn-cli[0] like so:
> 
>     msisdn-cli -H https://msisdn.services.mozilla.com -c 460
> 
> [0] https://github.com/mozilla-services/msisdn-cli

Hi, Rémy,

WOW. Cool!
Thanks for sharing this tool
There seems to have something wrong on the Mozilla loop client I installed.
Because I can get the correct response by using tool, log is shown as below.

Have a nice weekend.

--- -- - --- -- - --- -- - --- -- - --- -- - --- -- - --- -- -
(.venv)william@tpemozilla:~/Workspace_B2G/TestApp/msisdn-cli$ msisdn-cli -H https://msisdn.services.mozilla.com -c 460
Please enter your MSISDN number (ie [phone]): [phone]
MT Flow for [phone]
Please enter the code that you will get by SMS: 131207
Verified: {
  "exp": 1411207656, 
  "iat": 1411121246, 
  "iss": "msisdn.services.mozilla.com", 
  "lastAuthAt": 1411121256315, 
  "principal": {
    "email": "[e-mail]"
  }, 
  "pubkey": "<stripped>", 
  "publicKey": "<stripped>", 
  "verifiedMSISDN": "[phone]"
}
--- -- - --- -- - --- -- - --- -- - --- -- - --- -- - --- -- -

Comment 40

[Rémy Hubscher (:natim)]

Can you tell us what was the sender of the SMS you received the code from?

Comment 41

[William Hsu [:whsu]]

(In reply to Fernando Jiménez Moreno [:ferjm] from comment #38)
> William, it would also be great if you could retry enabling the Mobile ID
> debug prefs:
> 
> user_pref("services.mobileid.loglevel", "Debug");
> user_pref("services.hawk.loglevel", "Debug");
> user_pref("services.hawk.log.sensitive", true); 
> 
> This should add enough information to the logcat to let us understand why
> the requests are failing.
> 
> Also, you can test the mobile ID flow with the "UI Test - privileged" app to
> discard that this isn't a Loop specific issue.

Hi, Fernando,

Thanks for providing this tip.
I have collected the log. Please find the attachment.

There seems to have something wrong on Mozilla loop client I installed.
I cloned it from following repository. Is it correct one?
- https://github.com/mozilla-b2g/firefoxos-loop-client.git

I saw the following error message while I tap "Use phone number" button.
Do you think that we need to file the other bug to trace this problem?

--- -- - --- -- - --- -- - --- -- - --- -- - --- -- - --- -- - --- -- - 
(.venv)william@tpemozilla:~/Workspace_B2G/TestApp/firefoxos-loop-client$ adb logcat | grep error
I/Gecko   (  207): [Parent 207] WARNING: pipe error (190): Connection reset by peer: file ../../../gecko/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 450
I/Gecko   (  207): 1411122189028	MobileId	ERROR	MobileIdentityClient -> Error {"code":400,"errno":108,"error":"Missing mcc"}
I/Gecko   (  207): 1411122189030	MobileId	ERROR	UI error INVALID_BODY_MISSING_PARAMS
E/GeckoConsole( 2880): Content JS LOG at app://loop.services.mozilla.com/js/helpers/account_helper.js:151 in _onloginerror: onloginerror [object DOMError]
I/Gecko   (  207): 1411122197160	MobileId	ERROR	MobileIdentityClient -> Error {"code":400,"errno":108,"error":"Missing mcc"}
I/Gecko   (  207): 1411122197162	MobileId	ERROR	UI error INVALID_BODY_MISSING_PARAMS
E/GeckoConsole( 2880): Content JS LOG at app://loop.services.mozilla.com/js/helpers/account_helper.js:151 in _onloginerror: onloginerror [object DOMError]
--- -- - --- -- - --- -- - --- -- - --- -- - --- -- - --- -- - --- -- -

Comment 42

[William Hsu [:whsu]]

(In reply to Rémy Hubscher (:natim) from comment #40)
> Can you tell us what was the sender of the SMS you received the code from?

I got it from "[phone]".
Thanks. :D

Comment 43

[William Hsu [:whsu]]

Attachment: Mozilla_LOOP_20140919.log

Comment 44

[Fernando Jiménez Moreno [:ferjm]]

(In reply to William Hsu [:whsu] from comment #41)
> 
> Hi, Fernando,
> 
> Thanks for providing this tip.
> I have collected the log. Please find the attachment.
> 
> There seems to have something wrong on Mozilla loop client I installed.
> I cloned it from following repository. Is it correct one?
> - https://github.com/mozilla-b2g/firefoxos-loop-client.git
> 
> I saw the following error message while I tap "Use phone number" button.
> Do you think that we need to file the other bug to trace this problem?
> 
> --- -- - --- -- - --- -- - --- -- - --- -- - --- -- - --- -- - --- -- - 
> (.venv)william@tpemozilla:~/Workspace_B2G/TestApp/firefoxos-loop-client$ adb
> logcat | grep error
> I/Gecko   (  207): [Parent 207] WARNING: pipe error (190): Connection reset
> by peer: file
> ../../../gecko/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 450
> I/Gecko   (  207): 1411122189028	MobileId	ERROR	MobileIdentityClient ->
> Error {"code":400,"errno":108,"error":"Missing mcc"}
> I/Gecko   (  207): 1411122189030	MobileId	ERROR	UI error
> INVALID_BODY_MISSING_PARAMS
> E/GeckoConsole( 2880): Content JS LOG at
> app://loop.services.mozilla.com/js/helpers/account_helper.js:151 in
> _onloginerror: onloginerror [object DOMError]
> I/Gecko   (  207): 1411122197160	MobileId	ERROR	MobileIdentityClient ->
> Error {"code":400,"errno":108,"error":"Missing mcc"}
> I/Gecko   (  207): 1411122197162	MobileId	ERROR	UI error
> INVALID_BODY_MISSING_PARAMS
> E/GeckoConsole( 2880): Content JS LOG at
> app://loop.services.mozilla.com/js/helpers/account_helper.js:151 in
> _onloginerror: onloginerror [object DOMError]
> --- -- - --- -- - --- -- - --- -- - --- -- - --- -- - --- -- - --- -- -

Thanks William, this is a Loop specific issue that was fixed in bug 1059251. Could you try with the latest version of https://github.com/mozilla-b2g/firefoxos-loop-client please? It should be fixed already.

Comment 45

[William Hsu [:whsu]]

Hi, Fernando,

Thanks so much.
After I pull the latest code, the Mozilla Loop Client works as expected.
Also, this bug should be solved. Verified it.

My test number (Taiwan): "09787251xx -> [phone]"

* Build information:
 - Gaia          31434a3949556171f3565ca47ac2b44e810e95e6
 - Gecko         https://hg.mozilla.org/releases/mozilla-b2g32_v2_0/rev/a10034c1964f
 - BuildID       20140919000204
 - Version       32.0
 - Mozilla Loop: 0d03a08

Have a nice weekend! :D