1080008 - [Contacts] Importing contacts with special characters from Outlook do not appear correctly in contacts list.

Status: VERIFIED FIXED

(Firefox OS Graveyard :: Gaia::Contacts, defect)

Description

[John Thomas [:Johnt]]

Attachment: logcat_20141008_1008.txt

Description:

Repro Steps:
1) Update a Flame to 20141008000201
2) Have at least 1 contact with foreign characters in Outlook. Specific examples below under Notes.
3) Open Contacts app.
4) Select Gear / Settings option.
5) Select Import Contacts
6) Select Outlook
7) Attempt to select contacts with special characters in name. 

Actual: Contacts with foreign symbols appear blank in the import list, as well as the contacts list once they are import into the device.

Expected: It is expected that the contacts with foreign symbols will appear in the import list and the contacts list. 

Environmental Variables:
Device: Flame 2.1 KK (319mb)
Build ID: 20141008000201
Gaia: d71f8804d7229f4b354259d5d8543c25b4796064
Gecko: 7fa82c9acdf2
Version: 34.0a2 Flame 2.1 KK (319mb)
Firmware Version: L1TC10011800
User Agent: Mozilla/5.0 (Mobile; rv:34.0) Gecko/34.0 Firefox/34.0

Notes: The emails I specifically used to import from my outlook account were áéáéáé@gmail.com, áéáé@gmail.com and ईमेलपता@wtf.com

Repro frequency: 100%
Link to failed test case: https://moztrap.mozilla.org/manage/case/7210/
See attached: Logcat, Screenshot

Comment 1

[John Thomas [:Johnt]]

Attachment: NamelessContacts.png

Comment 2

[John Thomas [:Johnt]]

This issue DOES occur on the Flame 2.2 Master KK (319mb) (Full Flash)

"Device: Flame 2.2 Master KK (319mb) (Full Flash)
BuildID: 20141008040203
Gaia: 0bc74ce502672cf0265b24cf3a25d117c3de5e71
Gecko: e4cfacb76830
Gonk: 52c909e821d107d414f851e267dedcd7aae2cebf
Version: 35.0a1 (2.2 Master)
Firmware: V180
User Agent: Mozilla/5.0 (Mobile; rv:35.0) Gecko/35.0 Firefox/35.0"

--------------------------------------------------------------------

This issue does NOT reproduce on the 2.0 KK (319mb) (Full Flash)

Result: The email address for áéáéáé@gmail.com and áéáé@gmail.com appear correctly. However, the email for the ईमेलपता@wtf.com address appears as question marks, such as as ??????@wtf.com. I assume this may be because these characters are not implemented by default on the Flame 2.0 device.

"Device: Flame 2.0 KK (319mb) (Full Flash)
BuildID: 20141008000202
Gaia: 31a49c7932c7085961760a6bef9ed381ea38d7e3
Gecko: a2d707e79061
Gonk: 52c909e821d107d414f851e267dedcd7aae2cebf
Version: 32.0 (2.0)
Firmware: V180
User Agent: Mozilla/5.0 (Mobile; rv:32.0) Gecko/32.0 Firefox/32.0"

Comment 3

[Johan Lorenzo [:jlorenzo]]

[Blocking Requested - why for this release]: Regression from 2.0 with a high user impact in non-english speaking countries.

Comment 4

[Francisco Jordano [:arcturus] [:francisco]]

I've been trying to reproduce this several times and I see the following.

When I setup a contact with the email: ईमेलपता@wtf.com, in Hotmail itself it shows to me as said in comment 2: ??????@wtf.com

Also if I save the contact with name (not email): ईमेलपता@wtf.com, I can see it correctly on the list (will attach a patch).

So, IMHO, this is not a bug, just the data coming from Hotmail.

Will wait till Johan can confirm.

Comment 5

[Francisco Jordano [:arcturus] [:francisco]]

Attachment: 2014-10-09-15-39-31.png

As you can see in the bottom, when the name is ईमेलपता@wtf.com, it appears correctly, when that info is in the email, we receive from the hotmail api ???????@wtf.com

Comment 6

[Johan Lorenzo [:jlorenzo]]

I agree, we receive "???????@wtf.com" from hotmail:
> GET https://apis.live.net/v5.0/me/contacts?access_token=ACCESS_TOKEN
returns for the contact:
> {
>    "id": "contact.825ba182000000000000000000000000", 
>    "first_name": "ईमेलपता", 
>    "last_name": "ईमेलपता", 
>    "name": "ईमेलपता ईमेलपता", 
>    "is_friend": false, 
>    "is_favorite": false, 
>    "user_id": null, 
>    "email_hashes": [
>       "6ebe89605b0bea4e30f94cda8dca1119d6c9eddb074fd9794ed46be1a64b6165"
>    ], 
>    "updated_time": "2014-10-09T16:10:07+0000", 
>    "birth_day": null, 
>    "birth_month": null, 
>    "emails": {
>       "preferred": "???????@wtf.com", 
>       "account": null, 
>       "personal": "???????@wtf.com", 
>       "business": null, 
>       "other": null
>    }, 
>    "addresses": {
>       "personal": {
>          "street": null, 
>          "street_2": null, 
>          "city": null, 
>          "state": null, 
>          "postal_code": null, 
>          "region": null
>       }, 
>       "business": {
>          "street": null, 
>          "street_2": null, 
>          "city": null, 
>          "state": null, 
>          "postal_code": null, 
>          "region": null
>       }
>    }, 
>    "phones": {
>       "personal": null, 
>       "business": null, 
>       "mobile": null
>    }
> }

Replace ACCESS_TOKEN by yours (you can get it by using the plug your FxOS phone to the WebIDE, imports the contacts on the phone and then have a look at the Network tab).

By the way, I tried to repro on 2.1 and the accented characters appears correctly.
Gaia-Rev        7e2ef41d3ac98757acaf490b5413fb42061ad3e6
Gecko-Rev       https://hg.mozilla.org/releases/mozilla-aurora/rev/75ebb70f8b38
Build-ID        20141009000203
Version         34.0a2
Device-Name     flame-kk
FW-Release      4.4.2
FW-Incremental  34
FW-Date         Tue Sep 30 14:06:36 CST 2014
Bootloader      L1TC00011840


John, can you provide a screenshot of the issue with the accented characters. I'm removing the nom while we confirm it.

Comment 7

[John Thomas [:Johnt]]

Screenshot is already attached as "Nameless characters." The emails I specifically used to import from my outlook account were áéáéáé@gmail.com, áéáé@gmail.com and ईमेलपता@wtf.com

Comment 8

[Johan Lorenzo [:jlorenzo]]

Weird, áéáéáé@gmail.com appears under the letter "A" (not "#") on my 2.1 device. Can you check it's not the case for you? Thank you very much.

Comment 9

[Johan Lorenzo [:jlorenzo]]

Sorry, I understood why áéáéáé@gmail.com appears under A. I put a name to my contact. 

Francisco, you can repro the issue if you delete the names of your contact, and you'll see what we see on the screenshot.

Comment 10

[Johan Lorenzo [:jlorenzo]]

(In reply to Johan Lorenzo [:jlorenzo] from comment #9)
> Francisco, you can repro the issue if you delete the names of your contact,
> and you'll see what we see on the screenshot.

Francisco, you can repro the issue if you delete the names of your contacts in Outlook, try to reimport then you'll see what we see on the screenshot.

Comment 11

[John Thomas [:Johnt]]

Attachment: áAppearingUnderLetterA

Comment 12

[John Thomas [:Johnt]]

This is the same case for me as well. Attached a screen show showing this for reference. 

Environmental Variables:
Device: Flame 2.1 KK (319mb) (Full Flash)
Build ID: 20141008000201
Gaia: d71f8804d7229f4b354259d5d8543c25b4796064
Gecko: 7fa82c9acdf2
Version: 34.0a2 Flame 2.1 KK (319mb)
Firmware Version: v180
User Agent: Mozilla/5.0 (Mobile; rv:34.0) Gecko/34.0 Firefox/34.0

Comment 13

[Johan Lorenzo [:jlorenzo]]

[Blocking Requested - why for this release]: Regression from 2.0 with a user impact in non-english speaking countries.

Comment 14

[Maria Angeles Oteo (:oteo)]

Comms triage: Regression, we need to fix it.

Comment 15

[Francisco Jordano [:arcturus] [:francisco]]

Finally got what you mean guys and I'm happy to announce that's done by design, actually we fixed in 2.1

According to the contacts list specification (not the list for import contacts), we normalise the contacts, so:
áá@aa.com will be under the group A cause in the search is normalised as [e-mail]

This is happening already in the normal contacts list.

So again, this was a fix for the import list to make the search work properly that we added.

Comment 16

[Francisco Jordano [:arcturus] [:francisco]]

Just giving a bit more of context, we do this normalisation cause, product asked us if you want to search or find one of this special characters é, when we search we can type 'e' and still find it.

Same reason why ?????@wtf.com is appearing under #, after normalisation it doesnt start by a character from the alphabet that we use as groups.

Comment 17

[Francisco Jordano [:arcturus] [:francisco]]

Reopening until we check as well in 2.1

Comment 18

[Francisco Jordano [:arcturus] [:francisco]]

Attachment: Pointer to PR 25024

I would say this will be failing even in 1.3

The reason, a change in Live api, now trying to get the email from the api response from different fields.

Comment 19

[Sergi Mansilla [:sergi] (Telenor) [PTO]]

Comment on attachment 8503117 [details] [review]
Pointer to PR 25024

r=me after Johan confirms it works properly. Good job Francisco!

Comment 20

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8503117 [details] [review]
Pointer to PR 25024

Tested and works for me. I now see áéáéáé@gmail.com and ??????@wtf.com (for ईमेलपता@wtf.com)

Comment 21

[Francisco Jordano [:arcturus] [:francisco]]

Landed in master:

https://github.com/mozilla-b2g/gaia/commit/a95b616ef299eabeb7674d24ecd3b12e3cdd2cc4

Comment 22

[Francisco Jordano [:arcturus] [:francisco]]

Comment on attachment 8503117 [details] [review]
Pointer to PR 25024

[Approval Request Comment]
[Bug caused by] (feature/regressing bug #):
Change in Outlook api
[User impact] if declined:
User could not see her/his contacts when importing
[Testing completed]:
QA test the patch
[Risk to taking this patch] (and alternatives if risky):
Pretty low, we are just looking for the new places where to take the email information from the api response
[String changes made]:
None

Comment 23

[bhavana bajaj [:bajaj]]

Comment on attachment 8503117 [details] [review]
Pointer to PR 25024

lets verify this once it lands on 2.1

Comment 24

[Ryan VanderMeulen [:RyanVM]]

v2.1: https://github.com/mozilla-b2g/gaia/commit/a3183cebc140c026b0fc28da161508ec9111fc03

Comment 25

[[:RJ]]

Issue is verified fixed on the 2.2 flame and 2.1 flame 

Importing contacts with special characters from out look appears correctly.

Flame 2.2

Device: Flame 2.2 Master KK (319mb) (Full Flash)
BuildID: 20141017040208
Gaia: abef62c0623e5504a97b4fd411e879a67b285b52
Gecko: ae1dfa192faf
Gonk: 52c909e821d107d414f851e267dedcd7aae2cebf
Version: 36.0a1 (2.2 Master)
Firmware: V180
User Agent: Mozilla/5.0 (Mobile; rv:36.0) Gecko/36.0 Firefox/36.0

Flame 2.1

Device: Flame 2.1 KK (319mb) (Full Flash)
BuildID: 20141017001201
Gaia: 1ea74943cfe525c76a074ca1d7de8e51a70f6b98
Gecko: 2befa902ff5c
Gonk: 05aa7b98d3f891b334031dc710d48d0d6b82ec1d
Version: 34.0 (2.1)
Firmware: V180
User Agent: Mozilla/5.0 (Mobile; rv:34.0) Gecko/34.0 Firefox/34.0