Last Comment Bug 828861 - B2G PhoneNumberJS: Applying network mcc to all phone numbers is not working properly
: B2G PhoneNumberJS: Applying network mcc to all phone numbers is not working p...
Status: RESOLVED FIXED
[triage:1/16]
:
Product: Core
Classification: Components
Component: DOM: Device Interfaces (show other bugs)
: unspecified
: ARM Gonk (Firefox OS)
: -- normal (vote)
: mozilla21
Assigned To: Borja Salguero [:borjasalguero]
:
Mentors:
Depends on:
Blocks: b2g-phonenumberjs b2g-v1-certification
  Show dependency treegraph
 
Reported: 2013-01-10 01:22 PST by Borja Salguero [:borjasalguero]
Modified: 2013-02-14 07:38 PST (History)
29 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
tef+
wontfix
wontfix
fixed
fixed
fixed
fixed


Attachments
Patch to B2G SMS (2.34 KB, patch)
2013-01-28 10:00 PST, Borja Salguero [:borjasalguero]
vicamo: review+
Details | Diff | Review

Description Borja Salguero [:borjasalguero] 2013-01-10 01:22:33 PST
There are a lot of scenarios where reproduce it, and it's an important problem because it's making SMS, Calls & Contacts not working properly.

Scenario 1:
- In Berlin, with a Spanish SIM Card, type a SMS to a Spanish phone number without country code (i.e. 612123123)

Expected:
- Message is sent to +34612123123

Currently:
- Message is sent to +49612123123! The problem is that, for getting the internationalized version of the phone number, we are taking into account ONLY the MCC of the network, and we are neglecting the local MCC of the SIM Card.

Scenario 2:
- Create a contact during my visit to Berlin with my Spanish SIM Card with the following data:
 + Manolo Johanson
 + 612123123
- Go to Dialer and call the internationalized phone number of our Contact '+34612123123'

EXPECTED:
- Contacts API will retrieve 'Manolo Johanson' as the contact requested

CURRENTLY:
- The contact it's created with '+49612123123', so we are not retrieving any contact!


This one should be blocking because our device it's not working properly in Roaming!
Comment 1 Gregor Wagner [:gwagner] 2013-01-10 01:41:50 PST
I don't think we should try to be smart in gecko here. We can always can come up with an example where it doesn't work.

If we really want to solve this problem we should give the user the choice to choose a country when he enters the contact.

We need some UX input here if we should add this feature to the contacts app in one of our next versions.
Comment 2 Gregor Wagner [:gwagner] 2013-01-10 01:45:30 PST
Just to clarify: The user can save the international version of a number to avoid the problem.

We could make this more explicit (like Skype does) by adding a UI where the user has to choose the country with a flag when he enters a number.
Comment 3 Borja Salguero [:borjasalguero] 2013-01-10 01:55:59 PST
But this it's only 'fixing' the second scenario, but in the first one (When you try to call/send SMS directly to a Spanish SIM Card from Berlin with a Spanish SIM Card) it's not working :S, and we have tested with Android and it is! So probably ONLY for Roaming we should use the SIM MCC instead of Network MCC... Wdyt?
Comment 4 Daniel Coloma:dcoloma 2013-01-10 02:10:39 PST
(In reply to Borja Salguero [:borjasalguero] from comment #3)
> But this it's only 'fixing' the second scenario, but in the first one (When
> you try to call/send SMS directly to a Spanish SIM Card from Berlin with a
> Spanish SIM Card) it's not working :S, and we have tested with Android and
> it is! So probably ONLY for Roaming we should use the SIM MCC instead of
> Network MCC... Wdyt?

First scenario is pretty bad, it is a user expectation and likely to be a certification blocker. Any chance that one can be fixed?
Comment 5 Julien Wajsberg [:julienw] 2013-01-10 06:04:47 PST
About Scenario 1: 
I know that when I'm in roaming, my operator always tells me (with a SMS) that I have to use international numbers to join either french numbers or (if I am in Germany) german numbers. So I suspect that these scenarios don't work on some phones anyway.

So I don't think this is a certification blocker.

However, this works on my Android (tried both sending a SMS or dialing a french number with my french SIM in germany) so it might be nice to make it work too.

About Scenario 2:
This is really not blocking. Should we open a new bug for that ?
Comment 6 David Palomino [:dpalomino] 2013-01-11 03:23:36 PST
Hi, 

About scenario 1, I think this is an important issue. At least with movistar Spain SIM cards, in pretty most of handsets, it's possible to call or send SMS to spanish recipients without adding +34 prefix. Indeed (just as an example), carrier usually sends a SMS to roaming users telling that it is possible to use MSISDNs without country prefix. 

About scenario 2, I think it's also blocker. 

Adding certification blocker dep, and bb? for retriaging... 

Thanks!
David
Comment 7 Andrew Overholt [:overholt] 2013-01-11 03:38:18 PST
This is indeed an important issue.  Further testing is required to see how this case is handled elsewhere.  It is definitely something for v1 but not something that can be fixed for basecamp.
Comment 8 Julien Wajsberg [:julienw] 2013-01-11 03:51:32 PST
(In reply to David Palomino [:dpv] from comment #6)
> About scenario 1, I think this is an important issue. At least with movistar
> Spain SIM cards, in pretty most of handsets, it's possible to call or send
> SMS to spanish recipients without adding +34 prefix.

> Indeed (just as an
> example), carrier usually sends a SMS to roaming users telling that it is
> possible to use MSISDNs without country prefix. 

Actually not mine, he's actually telling me the opposite.

But I can buy that scenario 1 is a blocker.

> 
> About scenario 2, I think it's also blocker. 

Could you explain why ?

The problem is : a number added in germany without international prefix is not found when we use the number with the spanish international prefix. Really, I don't see why this should  block.

Whay _could_ be a blocker is : add a national-format number while in Germany, go back to Spain, try to call this number; if the number called is not correct (ie, not spanish) then it is a blocker. Could you try this ?

But really I think these are different bugs and so you should file different bugs...
Comment 9 David Palomino [:dpalomino] 2013-01-11 05:20:02 PST
Hi Julien, 

> 
> > 
> > About scenario 2, I think it's also blocker. 
> 
> Could you explain why ?
> 
> The problem is : a number added in germany without international prefix is
> not found when we use the number with the spanish international prefix.
> Really, I don't see why this should  block.
> 
> Whay _could_ be a blocker is : add a national-format number while in
> Germany, go back to Spain, try to call this number; if the number called is
> not correct (ie, not spanish) then it is a blocker. Could you try this ?
> 
> But really I think these are different bugs and so you should file different
> bugs...

The new contact should be stored assuming that, there is no internatinal prefix, or, it's the one specified by SIM card (in the example, 214 for Spain (+34), not Germany (+49)). 

If the contact is created as '+49612123123' (following the same example), then when coming back to Spain, trying to call to this contact will fail. 

I cannot test this, as I haven't traveled to Berlin. But if the spanish contact has been stored with +49... it won't work.
Comment 10 Julien Wajsberg [:julienw] 2013-01-11 05:49:54 PST
(In reply to David Palomino [:dpv] from comment #9)
> Hi Julien, 
> 
> The new contact should be stored assuming that, there is no internatinal
> prefix, or, it's the one specified by SIM card (in the example, 214 for
> Spain (+34), not Germany (+49)). 
> 
> If the contact is created as '+49612123123' (following the same example),
> then when coming back to Spain, trying to call to this contact will fail. 

> 
> I cannot test this, as I haven't traveled to Berlin. But if the spanish
> contact has been stored with +49... it won't work.

I think the spanish contact is stored with both numbers. The one with +49 would only be for lookup. But I don't know well this so this must be verified.

And I still don't understand why this should block basecamp.
Comment 11 Gregor Wagner [:gwagner] 2013-01-14 13:18:59 PST
What should be the behavior for the scenarios described in the description?
Basically it comes down to should we use the country information from the network or SIM card for the default?

For example if I am in Germany with my American SIM card and send a SMS to 650-123-4567.
Should it be delivered to +1-650-123-4567 because we have an american SIM card or to +49-650-123-4567 because we are in the German network assuming that 650-123-4567 is a valid local format in the US and Germany.
Comment 12 Julien Wajsberg [:julienw] 2013-01-14 13:37:48 PST
I'd also say that we'd have to save both in contact when creating a contact, so that we'd match both when searching. But I'm not sure for this.
Comment 13 Josh Carpenter [:jcarpenter] 2013-01-15 14:37:54 PST
Ayman is the right man for UX input here. Adding a needsinfo for him.
Comment 14 Kevin Hu [:khu] 2013-01-16 01:28:09 PST
Swu will discuss with Ken to see who could own this case.
Comment 15 Gregor Wagner [:gwagner] 2013-01-16 04:00:33 PST
(In reply to khu from comment #14)
> Swu will discuss with Ken to see who could own this case.

We still have to figure out if there is actually a bug here.
In my opinion, scenario 1 and scenario 2 behave correctly.
If I am in Germany with my US phone and dial a local German number, I expect to reach the German number and not an equivalent US number.
We have to wait for some UX input first.
Comment 16 David Palomino [:dpalomino] 2013-01-17 05:41:10 PST
Hi, 

I think basically the point here is if we assume home country as international prefix, or visited country instead. 

Being in Germany with Spanish SIM card in roaming (for instance) and calling to "612 345 678" really means call to "+34 612 345 678" or to "+49 612 345 678". 

IMO it should be always home country (in the example, "+34 612 345 678"). 

Cheers, 
David
Comment 17 Milan Sreckovic [:milan] 2013-01-18 10:28:00 PST
I'm with Gregor in comment 15.  We should assume the visited country.  If I'm walking around the town and see the number on the billboard and want to call it, I should be able to dial what's there, not try and figure out what the country code is, and how to convert the number I'm seeing, to the international number.
Comment 18 Daniel Coloma:dcoloma 2013-01-18 10:48:42 PST
(In reply to Gregor Wagner [:gwagner] from comment #15)
> (In reply to khu from comment #14)
> > Swu will discuss with Ken to see who could own this case.
> 
> We still have to figure out if there is actually a bug here.
> In my opinion, scenario 1 and scenario 2 behave correctly.
> If I am in Germany with my US phone and dial a local German number, I expect
> to reach the German number and not an equivalent US number.
> We have to wait for some UX input first.

that is not the expectaion we set to Movistar customers when travelling abroad. In most of the countries, Telefónica allows calling home country numbers without entering any special prefix, indeed, most of the times first time connecting to the roaming network, Telefónica sends an SMS to customers letting them know about this feature. This works well with Android, iPhone and any feature phone, don't understand why we should do something different here. At least scenario 1 is clearly a bug.
Comment 19 Milan Sreckovic [:milan] 2013-01-18 11:26:10 PST
That's good info.  Is this carrier specific?  Something that we can read as a setting from SIM or something else?  Or is it a property of a network you're currently on?  I've never actually experienced this behaviour with the Canadian carriers I've used, but things may have changed.  If there is a way to get the setting from SIM or such that says "local numbers means home, not current" or vice versa, we can figure out how to behave.
Comment 20 Julien Wajsberg [:julienw] 2013-01-18 11:38:31 PST
Let's ask some people that are currently outside of their country.

Dale, could you try to call a unprefixed number in UK from USA with a phone that does not run Firefox OS and that have your UK SIM ?
Comment 21 Joe Cheng [:jcheng] 2013-01-21 03:31:35 PST
Taipei team spent some time discussing this issue (QA / PM/ RIL) and it is believed that this is a feature instead of a bug. Therefore TEF?

It seems like currently for Gaia/RIL, no prefix is added to the phone number you are dialing and therefore whatever you are dialing is sent to the network operator. As in comment 0 scenario 1, since the network operator receive a number without any prefix, so the network operator will consider the number as local number.

This might be the correct behavior for V1 since nothing smart is being done here.

In order to behave as other phones in the market (iPhone/Android), it seems Gaia needs to consider roaming scenarios and add home country mcc prefix accordingly. (RIL should not really be adding any prefix automatically as there will be use cases where mcc should be added and some cases that mcc should not be added). Examples that came up in the discussion were:
When roaming,
- whatever number you input in the dialer manually/directly should just dial out as is (since you might be looking at some local ads and want to dial that number directly)
- numbers dialing out from a saved contact should probably add the home country mcc prefix so that the experience would be similar as if you are still in your home country (but exception is what if you add the contact with a local number without prefix when you are roaming already. So there might be need to have some kind of setting in the contacts app or dialer app to see if the user wants to automatically add home country prefix while roaming or simply ask the user if home country prefix is needed every time calling a saved contact)
- similarly, Message app should probably do something similar to handle the roaming case as dialer app... the scenarios can be more completed requiring UX input

This seems to require further Product/UX input as described. (i agree that current behavior is not optimal but given the complicated user scenarios it can be, might not be the best time implement this given the short time we have)
Comment 22 Daniel Coloma:dcoloma 2013-01-21 03:47:18 PST
(In reply to Joe Cheng [:jcheng] from comment #21)
> Taipei team spent some time discussing this issue (QA / PM/ RIL) and it is
> believed that this is a feature instead of a bug. Therefore TEF?
> 
> It seems like currently for Gaia/RIL, no prefix is added to the phone number
> you are dialing and therefore whatever you are dialing is sent to the
> network operator. As in comment 0 scenario 1, since the network operator
> receive a number without any prefix, so the network operator will consider
> the number as local number.
> 
> This might be the correct behavior for V1 since nothing smart is being done
> here.
> 
> In order to behave as other phones in the market (iPhone/Android), it seems
> Gaia needs to consider roaming scenarios and add home country mcc prefix
> accordingly. (RIL should not really be adding any prefix automatically as
> there will be use cases where mcc should be added and some cases that mcc
> should not be added). Examples that came up in the discussion were:
> When roaming,
> - whatever number you input in the dialer manually/directly should just dial
> out as is (since you might be looking at some local ads and want to dial
> that number directly)
> - numbers dialing out from a saved contact should probably add the home
> country mcc prefix so that the experience would be similar as if you are
> still in your home country (but exception is what if you add the contact
> with a local number without prefix when you are roaming already. So there
> might be need to have some kind of setting in the contacts app or dialer app
> to see if the user wants to automatically add home country prefix while
> roaming or simply ask the user if home country prefix is needed every time
> calling a saved contact)
> - similarly, Message app should probably do something similar to handle the
> roaming case as dialer app... the scenarios can be more completed requiring
> UX input
> 
> This seems to require further Product/UX input as described. (i agree that
> current behavior is not optimal but given the complicated user scenarios it
> can be, might not be the best time implement this given the short time we
> have)

Can we go a step back, the behaviour you are describing is not the same than the bug reported in scenario 1:

"Scenario 1:
- In Berlin, with a Spanish SIM Card, type a SMS to a Spanish phone number without country code (i.e. 612123123)

Expected:
- Message is sent to +34612123123

Currently:
- Message is sent to +49612123123! The problem is that, for getting the internationalized version of the phone number, we are taking into account ONLY the MCC of the network, and we are neglecting the local MCC of the SIM Card."

i.e. a prefix was being injected, that is not good and is a certification blocker. Can you confirm that this behaviour has been fixed and in case it is not mark it again as a blocker?
Comment 23 Vicamo Yang [:vicamo][:vyang] 2013-01-21 20:59:01 PST
(In reply to Daniel Coloma:dcoloma from comment #22)
> Can we go a step back, the behaviour you are describing is not
> the same than the bug reported in scenario 1:
> 
> "Scenario 1:
> - In Berlin, with a Spanish SIM Card, type a SMS to a Spanish
> phone number without country code (i.e. 612123123)
> 
> Expected:
> - Message is sent to +34612123123

Actually, your expectation is WRONG. When you message somebody with a non-international number in Berlin, the destination SHOULD always be someone in Germany, not Spain. Only by following such behaviour can users have unified experience in sending messages anywhere to anyone.

See following result matrix: (sending non-international number)

        Destination | A in Berlin | B in Madrid |
-------+------------+-------------+-------------+
       | German SIM |      A      |   Unknown   |
Berlin +------------+-------------+-------------+
       | Spain SIM  |      A      |   Unknown   |
-------+------------+-------------+-------------+
       | German SIM |   Unknown   |      B      |
Madrid +------------+-------------+-------------+
       | Spain SIM  |   Unknown   |      B      |
-------+------------+-------------+-------------+

But in your expectation:

        Destination | A in Berlin | B in Madrid |
-------+------------+-------------+-------------+
       | German SIM |      A      |   Unknown   |
Berlin +------------+-------------+-------------+
       | Spain SIM  |   Unknown(1)|      B(2)   |
-------+------------+-------------+-------------+
       | German SIM |      A      |   Unknown   |
Madrid +------------+-------------+-------------+
       | Spain SIM  |   Unknown   |      B      |
-------+------------+-------------+-------------+

In this case, when you borrow a SIM card from a foreign friend or roaming as (1), and try to contact your local friends, you find everything goes wrong. You just can't reach any local numbers easily.

In contrast, B's number may also be a valid subscription number in Berlin. There's no way to reach that person under your expectation because it'll always be sent to B like (2) shows.

The same thing applies to the Scenario 2 but IMHO contact database should never store normalized numbers automatically. Automatically picking a correct country code is a feature that should be done in Gaia, not Gecko. +49 or +34, or the routing of a message, a call, is actually decided by service providers, not us.
Comment 24 Peter Dolanjski [:pdol] 2013-01-22 00:09:32 PST
From GSMA:
"When you are roaming and you make a call, the operator in the visited country analyses the dialled number, and decides how best to route the call... Remember, when you call home or any other country, you have to type in the international access code and the correct country code along with the telephone number, omitting the leading zero. For example, to dial the UK mobile number 07903 XXX XXX from another country, you dial +44 7903 XXX XXX. If you are calling a landline, you may need to include an area code. If you are calling a local number in the visited country, the visited operator will usually connect the call directly to the party within the country you are in."

This supports Vicamo's description.  The only exception might be when a contact is called, if an assumption is made that all contacts are based in the SIM home country, unless a country code is included with the contact.

Daniel, if you think this truly blocks certification, please let us know.
Comment 25 Vicamo Yang [:vicamo][:vyang] 2013-01-22 01:21:03 PST
(In reply to Peter Dolanjski from comment #24)
> From GSMA:

Thank you. Just add the origin of the cited: http://www.gsma.com/aboutus/gsm-technology/roaming/how-roaming-works
Comment 26 Joe Cheng [:jcheng] 2013-01-22 02:31:11 PST
a(In reply to Daniel Coloma:dcoloma from comment #22)
> 
> Can we go a step back, the behaviour you are describing is not the same than
> the bug reported in scenario 1:
> 
> "Scenario 1:
> - In Berlin, with a Spanish SIM Card, type a SMS to a Spanish phone number
> without country code (i.e. 612123123)
> 
> Expected:
> - Message is sent to +34612123123
> 
> Currently:
> - Message is sent to +49612123123! The problem is that, for getting the
> internationalized version of the phone number, we are taking into account
> ONLY the MCC of the network, and we are neglecting the local MCC of the SIM
> Card."
> 
> i.e. a prefix was being injected, that is not good and is a certification
> blocker. Can you confirm that this behaviour has been fixed and in case it
> is not mark it again as a blocker?

the earlier comment 21 was more on the dialer part and i think for dialing, the current behavior is acceptable and correct (as it is the same as Nexus One that i tested)

and let's focus more back on the SMS experience. i was trying roaming with AT&T and China SIMs today and here are what I think is happening between the Message App and RIL. (let's not focus on the part involving network operator as per comment 24, network operator applies some logic that seems to be different with different SIMs you use)

MT (mobile-terminated) SMS:
RIL: pass the phone number to the message app as is from the network.
Message app: create the message thread base on the phone number from the RIL.
i think MT SMS is fine.

MO (mobile-originated) SMS:
Message app: Whatever number you input in the TO field, it sends that exact number (e.g. 0912345678) to RIL when you hit SEND in the Message Composition page. Here is what's interesting. Then I saw an empty message content page titled 0912345678. I have to press the BACK key to go back to All Message Threads view to find that there is a message thread +886912345678. Inside the +886912345678 thread, I found that the message i was trying to send did not go through. However, if i tap the message and resend it in the +886912345678 thread, the message will go through.
RIL: RIL takes whatever number message app sent to RIL and send it to the network.

So, here's what i think happened, when you send a message from the message composition page, the message is sent to the number as you specified. So i think it is working fine for v1. However, in the next step when the message is created in the message threads, there is some module that adds the prefix to the number and then creates the message thread and i think this is where the problem is. 

I heard there is this PhoneNumberUtility module in Gecko? so I am taking the assignee off from RIL and let's find a better assignee.

and thanks for reading my long paragraphs.
Comment 27 Tim Guan-tin Chien [:timdream] (please needinfo) 2013-01-22 02:34:22 PST
(In reply to Joe Cheng [:jcheng] from comment #26)
> I heard there is this PhoneNumberUtility module in Gecko? so I am taking the
> assignee off from RIL and let's find a better assignee.

Yes, someone should fix this on the Gecko side, from the upstream project

https://github.com/andreasgal/PhoneNumber.js
Comment 28 Daniel Coloma:dcoloma 2013-01-22 02:39:31 PST
With respect to comment 24: 

Roaming behaviour depends on the CAMEL agreements every company has. 

Whenever you are roaming in a country with a CAMEL agreement (very usual case for Telefónica roaming users) customers receive an SMS saying something like:

"Welcome to XXXXX Network, please remember that you can make calls and send SMS to Spain without entering any prefix"

If the device adds to the destination number any international prefix when roaming, CAMEL is not going to work and hence the device is not honouring what the customer has been told in the SMS. We need to honour what we tell the customers and not what a GSMA page says (please note that it is not a technical spec or a place where operator customers are going to look for help). 

For comment 23, Vicamo, do not understand your table, all I am saying is:
 
1/ If the device is in the local network, and no prefix is inserted by the user, the local prefix could be inserted. 
2/ If the device is in the local network and the user manually types an international prefix, the SMS is sent to the destination as typed by the customer.
3/ If the device is in a roaming network no prefix should be inserted by the device, the device should be sent as typed by the customer. The network should decide, based on the roaming agreements you have, how to route that message as it is the only one that knows how to do it based on the roaming agreements.

How can this behaviour lead to any problem at all?

This is a certification blocker, please mark it as tef+.
Comment 29 Gregor Wagner [:gwagner] 2013-01-22 07:35:32 PST
(In reply to Daniel Coloma:dcoloma from comment #28)
> With respect to comment 24: 

> 3/ If the device is in a roaming network no prefix should be inserted by the
> device, the device should be sent as typed by the customer. The network
> should decide, based on the roaming agreements you have, how to route that
> message as it is the only one that knows how to do it based on the roaming
> agreements.
> 

Ok now it starts to make sense. So you are saying we still send the SMS to the local number and the network figures out the rest. I assumed you wanted gecko to insert the spanish prefix in Scenario 1.
We are not modifying the number when we send the SMS so the bug here is the international number we display for the SMS thread.

Do we get any feedback from the network that tells us to what number the SMS was actually delivered? If this depends on the roaming agreements we need a setting for it.
Comment 30 Borja Salguero [:borjasalguero] 2013-01-22 11:54:19 PST
Hi there. A lot of comments here about the bug that I reported, and Im going to try to explain a little bit more because probably the solution is easier than it looks (or at least for the majority of the scenarios).

As the MAJORITY of our contacts are created in our home country, with or WITHOUT prefix, we have to ensure that send a SMS/placing a call from our Agenda it's gonna work as expected in Roaming.
During our working week in Berlin I was calling to my contacts as usual, without any problem (with my Android), using my Spanish SIM Card in roaming.

How could we ensure that these scenarios are going to work properly?

Using the MCC retrieved from the SIM Card (not the network one) for getting the internationalized version of the phone number.
Being abroad with my Movistar SIM Card should work with my agenda, so typing a SMS to 'Manolo 612123123' must work as in Spain (so we are going to send the SMS to +34612123123).

What is happening if I want to try to place a call to a local UK phone number using my Spanish SIM Card in London?

Simply you should call to the internationalized phone number. This could sound straightforward, even so simple, but at the end it's the common way of placing a call abroad.
Example. Imagine that during your holidays in UK, you create some contacts with UK local phone numbers with your Spanish SIM Card. If you want to keep in contact with them in the future, you should specify EXPLICITLY the country code! Because once you come back to your home country It's gonna be impossible to place a call or send a SMS to that contact without adding the UK country code!.

How could we solve these scenarios with a small change?

IMHO the solution it's quite simple. USING MCC FROM SIM Card instead of the network one for storing SMSs & Contacts in Gecko DBs & APIs. This solution is gonna work in the majority of the scenarios (the scenarios explained in this bug would be solved), and even if we consider that we need to review it deeply, we could open a follow-up in order to review the edge cases where this solution could fail.

[:vicamo] Wdyt about this solution? Would be feasible to have this surgical patch (changing from network MCC to SIM Card MCC) and opening the follow-up if needed?  Thanks a lot!
Comment 31 Gregor Wagner [:gwagner] 2013-01-22 14:09:29 PST
(In reply to Borja Salguero [:borjasalguero] from comment #30)
> 
> IMHO the solution it's quite simple. USING MCC FROM SIM Card instead of the
> network one for storing SMSs & Contacts in Gecko DBs & APIs. This solution
> is gonna work in the majority of the scenarios (the scenarios explained in
> this bug would be solved), and even if we consider that we need to review it
> deeply, we could open a follow-up in order to review the edge cases where
> this solution could fail.
> 
> [:vicamo] Wdyt about this solution? Would be feasible to have this surgical
> patch (changing from network MCC to SIM Card MCC) and opening the follow-up
> if needed?  Thanks a lot!

I think we would only shift the problem but we wouldn't solve it. The current implementation works well with my austrian SIM card. If we switch from the network mcc to the SIM mcc, all roaming SIM cards that don't support this carrier agreement would see the reversed problem.

From the other comments here I see that not many other countries support such a feature and it seems that this carrier agreement is the corner case. (Correct me if I am wrong here)

Somehow we have to get the information what MCC the network is using for local numbers. Is this info stored on the SIM card or can we ask the network for it? If this information is not available we need a setting.
My favorite solution would be UI based where we expose the country code to the user or the user has to choose the right country code but I guess that's too late to change now.
Comment 32 Gregor Wagner [:gwagner] 2013-01-22 14:11:02 PST
Ahhh I didn't want to reset all the flags! ***** bugzilla
Comment 33 Daniel Coloma:dcoloma 2013-01-22 14:14:40 PST
(In reply to Gregor Wagner [:gwagner] from comment #31)
> (In reply to Borja Salguero [:borjasalguero] from comment #30)
> > 
> > IMHO the solution it's quite simple. USING MCC FROM SIM Card instead of the
> > network one for storing SMSs & Contacts in Gecko DBs & APIs. This solution
> > is gonna work in the majority of the scenarios (the scenarios explained in
> > this bug would be solved), and even if we consider that we need to review it
> > deeply, we could open a follow-up in order to review the edge cases where
> > this solution could fail.
> > 
> > [:vicamo] Wdyt about this solution? Would be feasible to have this surgical
> > patch (changing from network MCC to SIM Card MCC) and opening the follow-up
> > if needed?  Thanks a lot!
> 
> I think we would only shift the problem but we wouldn't solve it. The
> current implementation works well with my austrian SIM card. If we switch
> from the network mcc to the SIM mcc, all roaming SIM cards that don't
> support this carrier agreement would see the reversed problem.
> 
> From the other comments here I see that not many other countries support
> such a feature and it seems that this carrier agreement is the corner case.
> (Correct me if I am wrong here)

This is not a cornercase. I can even call Spain from the US without prefix when roaming with my Movistar SIM Card. We really need to fix it.

> 
> Somehow we have to get the information what MCC the network is using for
> local numbers. Is this info stored on the SIM card or can we ask the network
> for it? If this information is not available we need a setting.

The information is not available in the SIM Card as the roaming agreements are subject to change and SIM Cards cannot be reflashed. I am afraid the network does not return that information either.  

> My favorite solution would be UI based where we expose the country code to
> the user or the user has to choose the right country code but I guess that's
> too late to change now.
Comment 34 Gregor Wagner [:gwagner] 2013-01-22 17:11:35 PST
(In reply to Daniel Coloma:dcoloma from comment #33)
> (In reply to Gregor Wagner [:gwagner] from comment #31)
> > 
> > I think we would only shift the problem but we wouldn't solve it. The
> > current implementation works well with my austrian SIM card. If we switch
> > from the network mcc to the SIM mcc, all roaming SIM cards that don't
> > support this carrier agreement would see the reversed problem.
> > 
> > From the other comments here I see that not many other countries support
> > such a feature and it seems that this carrier agreement is the corner case.
> > (Correct me if I am wrong here)
> 
> This is not a cornercase. I can even call Spain from the US without prefix
> when roaming with my Movistar SIM Card. We really need to fix it.
> 

Where can I find more information about roaming agreements?
Is this roaming agreement at least consistent for all countries of the world or does this depend on the country or maybe even the roaming network within the country?
Comment 35 Vicamo Yang [:vicamo][:vyang] 2013-01-22 19:03:20 PST
(In reply to Gregor Wagner [:gwagner] from comment #34)
> Where can I find more information about roaming agreements?
> Is this roaming agreement at least consistent for all countries
> of the world or does this depend on the country or maybe even
> the roaming network within the country?

http://en.wikipedia.org/wiki/CAMEL_Application_Part
Comment 36 Vicamo Yang [:vicamo][:vyang] 2013-01-22 19:23:44 PST
(In reply to Tim Guan-tin Chien [:timdream] (MoCo-TPE) from comment #27)
> (In reply to Joe Cheng [:jcheng] from comment #26)
> > I heard there is this PhoneNumberUtility module in Gecko? so
> > I am taking the assignee off from RIL and let's find a
> > better assignee.
> 
> Yes, someone should fix this on the Gecko side, from the
> upstream project
> 
> https://github.com/andreasgal/PhoneNumber.js

That's already filed as bug 830335, which was then resolved as a side effect in bug 823010.
Comment 37 Thomas Zimmermann [:tzimmermann] [:tdz] 2013-01-23 02:38:47 PST
I got this bug assigned, but I'm really not sure what to do here.

- The problem described in comment #26 apparently has been fixed as part of bug 823010.
- There is no agreement on which prefix to use when roaming, if any.

Personally, I would just use the phone number as entered by the user or stored in the contacts, without adding any prefix at all. The network operator will tell the user how the routing behaves. Additionally, we should give the user an option in the Settings app, where he/she can select the default prefix (none, SIM or network) for non-prefixed numbers.
Comment 38 Thomas Zimmermann [:tzimmermann] [:tdz] 2013-01-23 03:06:15 PST
Or maybe not.

Thinking about it, I don't think we should ever use the prefix from the SIM card by default. Suppose you're in a roaming network and need to call the police or ambulance. The call must go to the local authorities and not the ones on your home country. I wouldn't expect this to work correctly with a foreign prefix applied.
Comment 39 Maria Angeles Oteo (:oteo) 2013-01-23 07:32:00 PST
Hi David, Lucas,
could you please add the tef+ flag as it was agreed yesterday in the triage?
It seems that Gregor removed it by mistake, thanks!!
Comment 40 David Scravaglieri [:scravag] 2013-01-23 17:57:56 PST
Daniel, could you provide a solution that will not need to be hardcoded and will work in generic cases even if the roaming agreement is canceled ? We need a consistent solution there.
Comment 41 Daniel Coloma:dcoloma 2013-01-23 18:02:39 PST
(In reply to David Scravaglieri [:scravag] from comment #40)
> Daniel, could you provide a solution that will not need to be hardcoded and
> will work in generic cases even if the roaming agreement is canceled ? We
> need a consistent solution there.

Been talking to Borja today, I think we have a solution that works in nearly all the scenarios (at least in the common ones) but I'd like to review it tomorrow with him in detail.
Comment 42 Andrew Overholt [:overholt] 2013-01-24 10:28:43 PST
(In reply to Daniel Coloma:dcoloma from comment #41)
> (In reply to David Scravaglieri [:scravag] from comment #40)
> > Daniel, could you provide a solution that will not need to be hardcoded and
> > will work in generic cases even if the roaming agreement is canceled ? We
> > need a consistent solution there.
> 
> Been talking to Borja today, I think we have a solution that works in nearly
> all the scenarios (at least in the common ones) but I'd like to review it
> tomorrow with him in detail.

I'm going to assign to Borja since he's working on this.
Comment 43 Borja Salguero [:borjasalguero] 2013-01-24 16:55:56 PST
Im gonna un-assign because we were thinking how to solve this, but I would need help of some Gecko developers for implementing it due to the complexity of the patch, and I would like to ensure that the other parts of SMS are not affected with it. Probably [:anygregor] or [:vicamo] could help us with it.
Comment 44 Johnny Stenback (:jst, jst@mozilla.com) 2013-01-24 23:16:16 PST
Vicamo, any chance you can jump on this one now, and work with Borja here. Borja, where's this plan that's been discussed and needs implemented? A link, or a description here in the bug would be much appreciated. Thanks!
Comment 45 Andrew Overholt [:overholt] 2013-01-25 13:11:56 PST
Is bug 834755 a dupe of this bug?
Comment 46 Vicamo Yang [:vicamo][:vyang] 2013-01-27 18:44:06 PST
Whenever this bug assigned back to me, my response is WONTFIX. Hope this is the last time I have to be so rude. Sorry.
Comment 47 Vicamo Yang [:vicamo][:vyang] 2013-01-28 08:19:19 PST
*** Bug 834755 has been marked as a duplicate of this bug. ***
Comment 48 Borja Salguero [:borjasalguero] 2013-01-28 10:00:54 PST
Created attachment 707156 [details] [diff] [review]
Patch to B2G SMS
Comment 49 Borja Salguero [:borjasalguero] 2013-01-28 10:01:28 PST
Hi there,
After reviewing the issue and analyzing how Android and other platforms works, we have found the following.

In Gecko, at https://mxr.mozilla.org/mozilla-central/source/dom/system/gonk/RadioInterfaceLayer.js#2512 we store the SMS in Gecko, and then (in the callback) we deliver the SMS to the network.

What is the problem here?
The problem is that, when Roaming, we are internationalizing the number https://mxr.mozilla.org/mozilla-central/source/dom/sms/src/ril/SmsDatabaseService.js#765 , and we are using the MCC of the roaming network, which is not always appropriate, e.g. when we were in Berlin with a Spanish SIM Card, when sending a SMS to a Spanish phone number it was internationalized as +49612.... And that did not meet the expectations of TEF customers, as they can call and text home without entering any prefix in any other device.

How to solve it?
After testing a lot with other SIM Cards in Roaming and different platforms, we have concluded that in Roaming the task of routing a call or sending a SMS it's delegated to the network. This means that there is no need for adding a 'country-code' during Roaming, because the network will be in charge of routing or sending our call/SMS, and the behaviour of the network depends on dynamic conditions such as roaming agreements. 

In Android (tested with Movistar SIM Card in UK, Samsung Galaxy ACE):
- SIM Movistar Spain in Roaming ---> Spanish Phone number without Country Code   ----> OK
- SIM Movistar Spain in Roaming ---> Spanish Phone number with Country Code      ----> OK
- SIM Movistar Spain in Roaming ---> UK Phone number without Country Code      ----> Error. Network send us an error due to the destination it's unreacheable.
- SIM Movistar Spain in Roaming ---> UK Phone number with Country Code      ----> OK

With our patch the results are the following:
In Firefox OS (tested with Movistar SIM Card in UK, Unagi):
- SIM Movistar Spain in Roaming ---> Spanish Phone Phone number without Country Code   ----> OK
- SIM Movistar Spain in Roaming ---> Spanish Phone number with Country Code      ----> OK
- SIM Movistar Spain in Roaming ---> UK Phone number without Country Code      ----> Error. Same as Android.
- SIM Movistar Spain in Roaming ---> UK Phone number with Country Code      ----> OK

However, if we are in our local network, we will apply the Country Code as usual, getting the expected threading.

Surgical patch for a certification bug, same behaviour as Android.
Comment 50 Vicamo Yang [:vicamo][:vyang] 2013-01-28 10:27:20 PST
(In reply to Borja Salguero [:borjasalguero] from comment #49)
> What is the problem here?

There is no problem. The root of all problems is transform numbers into a specific format.

> The problem is that, when Roaming, we are internationalizing
> the number
> https://mxr.mozilla.org/mozilla-central/source/dom/sms/src/ril/
> SmsDatabaseService.js#765 , and we are using the MCC of the
> roaming network

If you read the code more carefully, you'll find the number is actually assigned in http://mxr.mozilla.org/mozilla-central/source/dom/system/gonk/RadioInterfaceLayer.js#2522 .  That is, the number sent to the network is the unmodified one. No matter you're roaming or not, Gecko sends the unmodified number.

Please!
Comment 51 Jason Smith [:jsmith] 2013-01-28 10:29:58 PST
Triage has determined that this is a blocker and must be fixed. Please don't close this again. Thanks.
Comment 52 Gregor Wagner [:gwagner] 2013-01-28 10:32:13 PST
(In reply to Vicamo Yang [:vicamo][:vyang] from comment #50)
> (In reply to Borja Salguero [:borjasalguero] from comment #49)
> > What is the problem here?
> 
> There is no problem. The root of all problems is transform numbers into a
> specific format.
> 
> > The problem is that, when Roaming, we are internationalizing
> > the number
> > https://mxr.mozilla.org/mozilla-central/source/dom/sms/src/ril/
> > SmsDatabaseService.js#765 , and we are using the MCC of the
> > roaming network
> 
> If you read the code more carefully, you'll find the number is actually
> assigned in
> http://mxr.mozilla.org/mozilla-central/source/dom/system/gonk/
> RadioInterfaceLayer.js#2522 .  That is, the number sent to the network is
> the unmodified one. No matter you're roaming or not, Gecko sends the
> unmodified number.

I think the first time it works well. But once we store it in the SMS DB, we change it to the international version because we have to match it with the incoming number. So the 2nd time you send from the thread, you will use the international number and then it doesn't work any more.
Comment 53 Fernando Jiménez Moreno [:ferjm] 2013-01-28 10:37:04 PST
(In reply to Vicamo Yang [:vicamo][:vyang] from comment #50)
> No matter you're roaming or not, Gecko sends the
> unmodified number.
>

Indeed. That's why Gecko should also store the unmodified number, right?
Comment 54 Borja Salguero [:borjasalguero] 2013-01-28 10:39:24 PST
(In reply to Gregor Wagner [:gwagner] from comment #52)
> I think the first time it works well. But once we store it in the SMS DB, we
> change it to the international version because we have to match it with the
> incoming number. So the 2nd time you send from the thread, you will use the
> international number and then it doesn't work any more.

Yep, this is the thing that we are trying to fix with this patch. As Fernando suggested, as sometimes we cant make any assumption about if applyting MCC is right or not (case of Roaming), we should store as we sent.
Comment 55 Vicamo Yang [:vicamo][:vyang] 2013-01-28 10:41:39 PST
(In reply to Jason Smith [:jsmith] from comment #51)
> Triage has determined that this is a blocker and must be fixed.
> Please don't close this again. Thanks.

You can fix it in Gaia. That's not a Gecko bug.
Comment 56 Borja Salguero [:borjasalguero] 2013-01-28 10:46:37 PST
(In reply to Vicamo Yang [:vicamo][:vyang] from comment #55)
> You can fix it in Gaia. That's not a Gecko bug.

Vicamo, as we have talked several times in other bugs, PhoneNumberJS is not available in Gaia layer since some months ago. That's why it's not a bug in Gaia, because phone number management is handled by Gecko.
Comment 57 Vicamo Yang [:vicamo][:vyang] 2013-01-28 10:50:22 PST
Ok, I was mad. I've reviewed your patch and you're not going to add +34 unconditionally. That's acceptable.
Comment 58 Daniel Coloma:dcoloma 2013-01-28 10:52:00 PST
(In reply to Vicamo Yang [:vicamo][:vyang] from comment #57)
> Ok, I was mad. I've reviewed your patch and you're not going to add +34
> unconditionally. That's acceptable.

Great! What should not be acceptable is marking patches as r- without even opening them.
Comment 59 Gregor Wagner [:gwagner] 2013-01-28 10:58:40 PST
So just to make sure, with this patch the thread-view doesn't work any more if we send to a local number and receive from the international number afterwards in a roaming scenario. Right?
Comment 60 Vicamo Yang [:vicamo][:vyang] 2013-01-28 11:04:33 PST
(In reply to Gregor Wagner [:gwagner] from comment #59)
> So just to make sure, with this patch the thread-view doesn't
> work any more if we send to a local number and receive from the
> international number afterwards in a roaming scenario. Right?

Correct. That's a consequence of having two or more formats in database.
Comment 61 Vicamo Yang [:vicamo][:vyang] 2013-01-28 11:18:02 PST
(In reply to Fernando Jiménez Moreno [:ferjm] from comment #53)
> (In reply to Vicamo Yang [:vicamo][:vyang] from comment #50)
> > No matter you're roaming or not, Gecko sends the
> > unmodified number.
> 
> Indeed. That's why Gecko should also store the unmodified
> number, right?

But you can't store every valid format. In the end the purpose of all these things is for assigning a thread id to each message. Relying on one universal format is the way we made it. However this method is now proven error prone, and we just can't fix everything with it. After all, even Android's super complicated way has exceptions.
Comment 62 Julien Wajsberg [:julienw] 2013-01-28 11:31:36 PST
(In reply to Gregor Wagner [:gwagner] from comment #59)
> So just to make sure, with this patch the thread-view doesn't work any more
> if we send to a local number and receive from the international number
> afterwards in a roaming scenario. Right?


I don't get it: we always receive from an international number, even in the origin country...
Comment 63 Vicamo Yang [:vicamo][:vyang] 2013-01-28 11:43:42 PST
(In reply to Julien Wajsberg [:julienw] from comment #62)
> I don't get it: we always receive from an international
> number, even in the origin country...

This patch skips normalization of the outgoing number when roaming. So the saved message will have a unmodified recipient number and breaks the assumption "every phone number stored in database is in international format".

Actually this patch has nothing to do with the bug description. Can we say it "resolves" this bug? Whatever, you can decide it in triage meeting.
Comment 65 Vicamo Yang [:vicamo][:vyang] 2013-01-28 20:37:01 PST
(In reply to Gregor Wagner [:gwagner] from comment #64)
> https://hg.mozilla.org/integration/mozilla-inbound/rev/b2b61d8b2735

Now you can file another bug: messages sent during roaming aren't attached to the previous conversation with the same recipient.
Comment 66 Ryan VanderMeulen [:RyanVM] 2013-01-29 06:51:45 PST
https://hg.mozilla.org/mozilla-central/rev/b2b61d8b2735

Note You need to log in before you can comment on or make changes to this bug.