1128883 - [email/activesync] "Junk" folder is not detected and localized due to ActiveSync protocol limitations (there is no junk/spam folder type); use browserbox name database to detect

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::E-Mail, enhancement)

Description

[Beatriz Rodríguez [:brg]]

Attachment: Screenshot Flame v2.1

Tested in Flame v2.1 (with l10n-v2.1 repository) and Flame master and l10n repositoires:
- Create hotmail account
- Check the folders of your account in the menu.
- Junk folder is not translated in Spanish (the only one not translated)

IMHO it must be something like: "Correo no deseado" (like I can see it in hotmail web site).

Comment 1

[Guillermo López :willyaranda (probably SLOW response)]

Comment on attachment 8558432 [details]
Screenshot Flame v2.1

Comes from

https://github.com/mozilla-b2g/gaia/blob/v2.1/build/test/resources/locales/en-US/apps/email/email.properties#L401

To here

http://mozilla.locamotion.org/es/firefoxos_2.1/translate/apps/email/email.properties.po#filter=all&unit=7735328

And it is pushed into the repo:

http://hg.mozilla.org/releases/gaia-l10n/v2_1/es/file/245c34d11335/apps/email/email.properties#l471

I have not tested the translation since last September, but this seems correct for me. Who is in charge of gaia-l10n now? Beatriz, please ni? him/her.

Comment 2

[Beatriz Rodríguez [:brg]]

For more information, I have checked French and Portuguese and I can see Junk label as well in hotmail. while Gmail accounts are showing the suitable translations in each language.
Moved to mail component.

Comment 3

[Andrew Sutherland [:asuth] (he/him)]

Unfortunately this is an outgrowth of a limitation of the ActiveSync protocol.  It can tell us Inbox/Drafts/Deleted/Sent/Outbox using a special enumerated type, but that's it.  The workaround is to detect the folder type based on its name.  We had some limited logic for this in IMAP, but since we've adopted email.js, it also has a multilingual dataset built-in that can potentially do even better in the event the folder name is localized.

:brg, for prioritization, can you indicate what the locale of the hotmail account was when you created it?  For my en-US created account, when I changed the language in the outlook.com web interface to Spanish and re-created the account from the email app, "Junk" still stayed "Junk", but it's possible that if I had originally created the account as es-ES that it might have already been localized, and it'd be good to know if that's the case or not.  Thanks!

Comment 4

[Beatriz Rodríguez [:brg]]

Not sure if this is what you want to know:
- I change my account language from Spanish to English in the web via PC.
- Re-create the account in Flame(device in Spanish) and the list of folders is the same: in Spanish with Junk.
- Enter in the web account in Flame(Spanish) and the list of folders is in English with Junk.

Ni me if you need any other test please.

Comment 5

[Beatriz Rodríguez [:brg]]

This issue is reported in certification and waived but we need a fix in next releases. I know is very late for 2.1 but we do need it in 2.2 or it will block future certifications.

Comment 6

[Stephany Wilkes]

Triage is blocking based on the fact that this will be a cert blocker.

Comment 7

[GitHub Autolander Bot]

Attachment: [gaia-email-libs-and-more] asutherland:activesync-infer-junk-folder > mozilla-b2g:master

Comment 8

[Andrew Sutherland [:asuth] (he/him)]

Comment on attachment 8567529 [details] [review]
[gaia-email-libs-and-more] asutherland:activesync-infer-junk-folder > mozilla-b2g:master

Test fails without the fix, passes with it.

Note that there are several important caveats, some of which are front-endy, so directing review at :jrburke:

- This only affects newly perceived junk folders.  So practically speaking, this will only fix things for new accounts.  Especially since we're sorta past the v2.2 deadline now, I think this is the most pragmatic risk tradeoff that does not involve blowing away all our current account state.  Also, this way is a lot less work and the argument "existing users clearly aren't bothered by seeing the word junk".

- Although the backend change adds a pretty big list of junk folder names that we'll detect, the front-end will still only localize the names junk/bulk mail/spam because the UI still just uses MailFolder.name and that still just uses MailAPI.l10n_folder_name which has some hard-coded mapping logic.  We have a pending bug on more aggressively just using the type and forgetting this guard.  But this bug is not that bug.  (Although I have no problem if someone who was not me wanted to fix that bug at the same time as this bug.)

Comment 9

[James Burke [:jrburke]]

Comment on attachment 8567529 [details] [review]
[gaia-email-libs-and-more] asutherland:activesync-infer-junk-folder > mozilla-b2g:master

r+ on inspection, the test case passing is proof enough for me, change is straightforward.

Agreed on limiting this to new account creations given branch desires and that this issue is not crying out to be fixed/causing high levels of confusion.

As for the front end localization issue, if the fix is just removing the `&& lowerName === ...` checks in mailapi.js, l10n_folder_name(), then I can work up a pull request for that. If you have a specific bug ID already in mind for it, feel free to mention it, otherwise if you say it is a new one, I will create a ticket for it and do the change.

Comment 10

[Andrew Sutherland [:asuth] (he/him)]

(In reply to James Burke [:jrburke] from comment #9)
> Comment on attachment 8567529 [details] [review]
> As for the front end localization issue, if the fix is just removing the `&&
> lowerName === ...` checks in mailapi.js, l10n_folder_name(), then I can work
> up a pull request for that. If you have a specific bug ID already in mind
> for it, feel free to mention it, otherwise if you say it is a new one, I
> will create a ticket for it and do the change.

So, I was thinking of a combination of bug 905869 (specifically point 2 in bug 905869 comment 0) and bug 855198.  The former is the behavioral change (use the localized strings no matter what the underlying folder name is) in bug 905869, and the implementation change to use data-l10n-id to this end (bug 855198).  We don't have to be so regimented about it, of course.

Comment 11

[Andrew Sutherland [:asuth] (he/him)]

Noting that some of those bugs currently are talking about adding localized strings for "All Mail" and "Starred" or whatever, so we should make sure to spin-off bugs as appropriate or just note close those original bugs, etc. etc. etc.

Comment 12

[James Burke [:jrburke]]

Comment on attachment 8567529 [details] [review]
[gaia-email-libs-and-more] asutherland:activesync-infer-junk-folder > mozilla-b2g:master

In a bizarre turn of events I shall now ask you to review your own pull request! Specifically, I added this commit:

https://github.com/asutherland/gaia-email-libs-and-more/commit/774b83030baa7ae97a499e1f2c2dc4bb74388864

It just uses the folder type to decide if the localized string for those matching types should be used.

Comment 13

[GitHub Autolander Bot]

Attachment: [gaia] jrburke:bug1128883-email-activesync-infer-junk-folder > mozilla-b2g:master

Comment 14

[James Burke [:jrburke]]

Comment on attachment 8568255 [details] [review]
[gaia] jrburke:bug1128883-email-activesync-infer-junk-folder > mozilla-b2g:master

Carrying over the GELAM r+ for the Gaia change as it is a straight copy. Will wait for try tests to pass then will ask for checkin-needed.

Comment 15

[James Burke [:jrburke]]

GELAM change merged in mozilla-b2g/gelam master:
https://github.com/mozilla-b2g/gaia-email-libs-and-more/commit/79683a20e2c4b52606016af924d7cf6b398ff029

from pull request:
https://github.com/mozilla-b2g/gaia-email-libs-and-more/pull/370

(manual merge since pull request in that tree did not have the bug number in the pull request, I should have caught that)

Comment 16

[GitHub Autolander Bot]

Pull request has landed in master: https://github.com/mozilla-b2g/gaia/commit/fea218be41330d86dc77fa99d3abc7c65e91df6f

Comment 17

[James Burke [:jrburke]]

Comment on attachment 8568255 [details] [review]
[gaia] jrburke:bug1128883-email-activesync-infer-junk-folder > mozilla-b2g:master

[Approval Request Comment]

[Bug caused by] (feature/regressing bug #):
Existing weakness with activesync protocol lacking a formal type to indicate the junk folder.

[User impact] if declined:
Junk folder for activesync not properly identified as junk and not properly localized.

[Testing completed]:
The gelam repo that originated this fix contains automated tests proving the functionality.

[Risk to taking this patch] (and alternatives if risky):
Low. 

[String changes made]:
none: existing l10n string for junk that was already passed in previously is used.

Note that this change will only affect newly created accounts. Existing accounts will not be converted. This tradeoff was done to limit the need for database upgrades and scope risk since the change is coming later in the 2.2 cycle.

Comment 18

[Ryan VanderMeulen [:RyanVM]]

v2.2: https://github.com/mozilla-b2g/gaia/commit/8999c3af38d69106880e080b961fb0075857fb6c

Comment 19

[Beatriz Rodríguez [:brg]]

Thanks everyone for the effort to fix it. I have just tested it in Flame 2.2 (Gecko-fe48991.Gaia-ca64f2f) with the device in Spanish, I can see "Basura" instead of Junk folder in gmail&hotmail accounts. With the device in French, I can see "indésirables".