804977 - [email] Inbox box is wrong when it contains accent like in french 'Boîte de réception' and can't check mails

Status: VERIFIED FIXED

(Firefox OS Graveyard :: Gaia::E-Mail, defect, P1)

Description

[David Scravaglieri [:scravag]]

Attachment: Wrong recipient box

Configure gmail account

the recipient is wrong (see attachement), can't check for new emails

Comment 1

[Andrew Sutherland [:asuth] (he/him)]

By "recipient box", do you mean the names of the mail folders are wrong due to an apparent encoding issue?  Something is most definitely wrong in that folder list, either way!

And can you elaborate on "can't check for new emails"?  Assuming the gmail account (based on the screenshot) was created as an ActiveSync acccount, the folders should be named by guid's, not their encoded string names, so you should be able to select a folder and see it populated with messages.  However, the number of days we synchronize (bring up the settings screen, click on the account, see what the value of "Synchronize" is.  We default to 3 days right now, so if the messages are older than 3 days, none will show up.)  While you are on that page, please confirm that the Account Type is "ActiveSync".

Comment 3

[Jim Porter (:squib)]

This works fine for me with Gmail-over-ActiveSync (I just made a label with the name in the bug summary).

Comment 4

[Jim Porter (:squib)]

Since I'm on desktop and a few days out of date, I'm wondering if perhaps something has broken with the TextEncoder API. I know this stuff worked in the past, since I had a commit to fix exactly this issue a while back.

Comment 5

[Andrew Sutherland [:asuth] (he/him)]

So the broken mapping is:

char, unicode, utf-8
î = \u00ee = \xc3\xae  =>  &AO4-
é = \u00e9 = \xc3\xa9  =>  &AOk-

These are base64 encodings of the unicode character values preceded by an '&' and where '-' is used for padding instead of '='.  This is not striking my memory as a standard encoding, but I could be repressing something horrible...

Comment 6

[Jim Porter (:squib)]

Attachment: A test patch for debugging this

I can't reproduce this at all. Some questions for people experiencing this:

1) What platform are you on? (Otoro, Unagi, Linux...?)
2) Can you try applying the attached patch and then post the resulting logcat? (Or email it to me if you prefer.)

Comment 7

[Alex Keybl [:akeybl]]

Naoki - are you able to repro?

Comment 8

[John Shih]

Can't reproduce anymore, I think this has been fixed already
closing

2012-11-09
gaia master : 18107c6961b8c3f559c82fc7be8ba411c4a67c28
mozilla-aurora(hg) : 6735d6d13751

Comment 9

[David Scravaglieri [:scravag]]

Nov 15 build

gecko-aurora: 0d76df6f808d
gaia: 487a7c18c8ff08db7d2e76239e85e3e5eefcc8b7

This is on gmail (must be synch with IMAP now)

The problem is that my mailbox is localized in French, so the app must synch with "Boîte de réception" (notice the accents).
But there is a UTF-8 problem and the app try to synch with "Bo&AO4-te de r&AOk-ception" which doesn't exists.

Comment 10

[Jim Porter (:squib)]

Did you delete and recreate your account today? You shouldn't be getting badly-internationalized folders from the server, since IMAP doesn't localize the inbox's name.

Comment 11

[Jim Porter (:squib)]

Ok, actually, I can confirm this for Gmail-over-IMAP, but as before Gmail-over-ActiveSync works fine (in fact, it gives me English strings even when the phone is set to French). Do you mind me asking what the domain part of your email address is?

Comment 12

[Jim Porter (:squib)]

This is an IMAP bug, so let's have asuth look at it.

Comment 13

[Andrew Sutherland [:asuth] (he/him)]

Okay, so that encoding is indeed the modified UTF-7 used for IMAP folder names.  Clearly David was not using ActiveSync when he first ran into this, which is confusing and indeed the domain would be useful.  This is a deficiency in imap.js; I'll prepare a patch for our branch and also try and upstream it.

It's possible gmail is doing this because we use XLIST which enables it to convey via meta-data that the folder is actually an INBOX.

I checked the Thunderbird code, and Thunderbird actually explicitly forces the folder name to INBOX in cases where the name is not INBOX but there is an inbox flag.  I assume this is some kind of invariant for either sanity or to avoid breakage elsewhere in the mailnews codebase.

This is an example where a fake-server for IMAP would indeed be useful and where we could then probably use the same test for both ActiveSync and IMAP.  I fear that will need to be postponed for time reasons.

Comment 14

[Andrew Sutherland [:asuth] (he/him)]

Attachment: Pointer to Github pull request: https://github.com/mozilla-b2g/gaia-email-libs-and-more/pull/80

Pointer to Github pull-request

Comment 15

[Jim Porter (:squib)]

Comment on attachment 682324 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia-email-libs-and-more/pull/80

r- for now since, as mentioned over IRC, this still has some problems.

Comment 16

[Andrew Sutherland [:asuth] (he/him)]

For general interest/subsequent fake-server test creation, here's the XLIST payload results for squib's french gmail test account.  I have no idea what's up with entries having both \HasChildren and \HasNoChildren but we don't care anyways.

(\HasNoChildren \Inbox) "/" "Bo&AO4-te de r&AOk-ception"
(\Noselect \HasChildren) "/" "[Gmail]"
(\HasChildren \HasNoChildren \Drafts) "/" "[Gmail]/Brouillons"
(\HasChildren \HasNoChildren \Trash) "/" "[Gmail]/Corbeille"
(\HasChildren \HasNoChildren \Important) "/" "[Gmail]/Important"
(\HasNoChildren \Sent) "/" "[Gmail]/Messages envoy&AOk-s"
(\HasNoChildren \Spam) "/" "[Gmail]/Spam"
(\HasNoChildren \Starred) "/" "[Gmail]/Suivis"
(\HasChildren \HasNoChildren \AllMail) "/" "[Gmail]/Tous les messages"

Comment 17

[Andrew Sutherland [:asuth] (he/him)]

Comment on attachment 682324 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia-email-libs-and-more/pull/80

Back up for review with gmail-specific INBOX fix and bonus "not appending sent messages to the sent folder" feature just for gmail.

Comment 18

[Jim Porter (:squib)]

Comment on attachment 682324 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia-email-libs-and-more/pull/80

I tested this out, and it looks good to me!

Comment 19

[Andrew Sutherland [:asuth] (he/him)]

landed:
https://github.com/mozilla-b2g/gaia-email-libs-and-more/pull/80
https://github.com/mozilla-b2g/gaia/pull/6515

Comment 20

[amiller]

Device: Unagi
Build: 20130103070201
Accented characters text message received and opened without issue.