813473 - [Email] Is not possible to set up the email with domain hotmail.es

Status: VERIFIED FIXED

(Firefox OS Graveyard :: Gaia::E-Mail, defect, P1)

Description

[mbarone]

Attachment: error_screenshot

This bug is happening with every build

Pre-Condition: create an account in hotmail.es

STR
1. Open email app
2. Configure a new email account with the domain hotmail.es

ACTUAL RESULT: The app returns an error (An unknown error occurred)
EXPECTED RESULT: The account must be configured correctly

Comment 1

[Jim Porter (:squib)]

How do you create a hotmail.es account? I only get to choose between @hotmail.com and @live.com.

Comment 2

[Jim Porter (:squib)]

mbarone: Can you try manually setting up the account? Just choose ActiveSync and enter "m.hotmail.com" for the server.

Comment 3

[Jim Porter (:squib)]

(In reply to Jim Porter (:squib) from comment #1)
> How do you create a hotmail.es account? I only get to choose between
> @hotmail.com and @live.com.

Nevermind, I figured it out.

So the solution here is to add an autoconfig file for hotmail.es (we need one for live.com too, and one for hotmail.com would improve performance). The downside of this is that I have no idea how many similar domains there are, and there's an upper limit to the number of autoconfig files we can reasonably include.

There also appears to be a version of Google apps for Hotmail, but that uses MX records, and tells you to point to something like 442368421.pamx1.hotmail.com[1], so we should be good there.

Andrew, any opinions on this? I know I'd discussed similar issues before on Github, but I don't recall a conclusion.

As an aside, it is especially unfortunate that even Microsoft can't obey the ActiveSync specs, since Autodiscovery should work, and clients should be able to use it, but we have to work around their deficiencies here.

[1] http://kb.siteground.com/article/How_to_point_my_MX_records_to_Windows_Live.html

Comment 4

[Jim Porter (:squib)]

Here's the old github issue: https://github.com/mozilla-b2g/gaia/issues/2526

Comment 5

[Andrew Sutherland [:asuth] (he/him)]

What is the domain hotmail.es actually uses?  We punted on domain guessing for autoconfig, like trying imap.DOMAIN and mail.DOMAIN... is it just m.DOMAIN for hotmail or something like that?

Comment 6

[Jim Porter (:squib)]

(In reply to Andrew Sutherland (:asuth) from comment #5)
> What is the domain hotmail.es actually uses?  We punted on domain guessing
> for autoconfig, like trying imap.DOMAIN and mail.DOMAIN... is it just
> m.DOMAIN for hotmail or something like that?

It's m.hotmail.com. I just tried m.hostmail.es, and it doesn't work at all. Basically, we need to map all Hotmail domains to use m.hotmail.com, and that seems like a good job for our autoconfig files.

Comment 7

[Jim Porter (:squib)]

Before I write the patch for this, do we have a policy on how we're handling regional configurations like this? We have a few options:

1) Add configs for all the various Hotmails: this would fix the issue for everyone, but I don't want to end up with hundreds of autoconfig files, since that would be hard to maintain.

2) Let network operators add configs for the domains in their regions: this requires additional work for the providers and wouldn't help people on our nightlies, but it would fix the issue for most people.

3) Update the Mozilla ISP DB files to include ActiveSync configuration information: like (1), this would fix the issue for everyone, but would be a little bit slower. It also might cause some incompatibilities with other (badly-written) clients that use the ISP DB.

4) Do nothing: anyone on domains other than @hotmail.com would have to manually configure their accounts, which means hitting "Manually set up account" after the initial failure and typing in "m.hotmail.com".

Comment 8

[Andrew Sutherland [:asuth] (he/him)]

(In reply to Jim Porter (:squib) from comment #7)
> Before I write the patch for this, do we have a policy on how we're handling
> regional configurations like this? We have a few options:

I think the gaia policy is that we punted on regional configuration issues to v2.  However, supporting e-mail domains is not really a regional issue (although the relative importance of different domains does matter).  I think picking what domains to show buttons for or bias towards in autocompletion/select boxes would be regional.

> 1) Add configs for all the various Hotmails: this would fix the issue for
> everyone, but I don't want to end up with hundreds of autoconfig files,
> since that would be hard to maintain.

I think hotmail is likely big enough in all markets that we want to make sure that it works and gets fast-pathed so we look good.  So I think the short answer is yes, add the autoconfig files for hotmail.

I agree that we don't want hundreds of manually maintained files... more below.

 
> 2) Let network operators add configs for the domains in their regions: this
> requires additional work for the providers and wouldn't help people on our
> nightlies, but it would fix the issue for most people.

Do you mean having them change the gaia e-mail app and add autoconfig files, or have ISPs stand up autoconfig.DOMAIN servers?  Either way, this does not sound desirable.


> 3) Update the Mozilla ISP DB files to include ActiveSync configuration
> information: like (1), this would fix the issue for everyone, but would be a
> little bit slower. It also might cause some incompatibilities with other
> (badly-written) clients that use the ISP DB.

Supporting/reusing the ISP database effort seems like the right long-term solution.  As long as we discuss adding ActiveSync entries on tb-planning and we make sure that no supported Thunderbird gets regressed, this seems fine, especially as the main consumer I know of is Evolution, and I believe they just ship their own copy of the database.

As I think I mentioned before, the privacy people would prefer if we just shipped the ISP database with the app.  Because of the many-domains-one-actual-config problem (which I believe the server resolves with symlinks), this would imply some type of compilation phase.  I think a reasonable plan here would be to aggregate entries based on some small prefix of their domain into a single file which could then compress reasonably well.

A simple approach might be to put pre-chewed JSON reps of the autoconfig decision for our app for each domain in the file, preceded by their domain and a character that cannot legally exist in JSON so that we can just use indexOf and avoid instantiating a huge object tree.  We would also support a short-hand that indicates that we should look up the configuration from another domain so we aren't duplicating entries (although we might need to perform a second local XHR request.)  So with a prefix of 2, "hotmail.com" and "hotmail.es" would both end up in "pf-ho.hack", we do an indexof("\nhotmail.es=", find "hotmail.es=hotmail.com", then do indexOf("\nhotmail.com=", find "hotmail.com={JSON}", and go to town.

We would not do that for v1, that's just a longer term strategy.

Comment 9

[Jim Porter (:squib)]

(In reply to Andrew Sutherland (:asuth) from comment #8)
> I think hotmail is likely big enough in all markets that we want to make
> sure that it works and gets fast-pathed so we look good.  So I think the
> short answer is yes, add the autoconfig files for hotmail.

That's reasonable. I'm probably going to hold off on adding hotmail.com for now though, since that's a good practical test of our autodiscovery code.

> Do you mean having them change the gaia e-mail app and add autoconfig files,
> or have ISPs stand up autoconfig.DOMAIN servers?  Either way, this does not
> sound desirable.

I meant the former.

> Supporting/reusing the ISP database effort seems like the right long-term
> solution.  As long as we discuss adding ActiveSync entries on tb-planning
> and we make sure that no supported Thunderbird gets regressed...

I'll post a message to the list today or tomorrow.

> As I think I mentioned before, the privacy people would prefer if we just
> shipped the ISP database with the app.

This would be very unfortunate, since it would make life much harder for anyone running a small server to provide autoconfig information to us.

Comment 10

[Andrew Sutherland [:asuth] (he/him)]

(In reply to Jim Porter (:squib) from comment #9)
> > As I think I mentioned before, the privacy people would prefer if we just
> > shipped the ISP database with the app.
> 
> This would be very unfortunate, since it would make life much harder for
> anyone running a small server to provide autoconfig information to us.

Anyone running a small server should just stand up the autoconfig domain themselves or name their hosts something predictable so that we can guess them when we get around to implementing domain name guessing.

The people using the small server are the ones potentially out-of-luck...

And I think those are probably reasonable trade-offs in the name of avoiding a single point of failure, especially one with privacy and security ramifications.  (This is all assuming we get MX and/or SRV resolution from the platform or do it ourselves.)

Comment 11

[Jim Porter (:squib)]

(In reply to Andrew Sutherland (:asuth) from comment #10)
> (In reply to Jim Porter (:squib) from comment #9)
> > This would be very unfortunate, since it would make life much harder for
> > anyone running a small server to provide autoconfig information to us.
> 
> Anyone running a small server should just stand up the autoconfig domain
> themselves or name their hosts something predictable so that we can guess
> them when we get around to implementing domain name guessing.

Of all the things we transmit during autoconfig, polling the autoconfing.<DOMAIN> URLs are probably the most dangerous privacy-wise, since they're HTTP and send the email address along. Polling the Mozilla ISPDB is pretty safe, since it's HTTPS and we only send the domain.

Comment 12

[Andrew Sutherland [:asuth] (he/him)]

(In reply to Jim Porter (:squib) from comment #11)
> Of all the things we transmit during autoconfig, polling the
> autoconfing.<DOMAIN> URLs are probably the most dangerous privacy-wise,
> since they're HTTP and send the email address along. Polling the Mozilla
> ISPDB is pretty safe, since it's HTTPS and we only send the domain.

So, that's a thorny issue related to the current state of DNS security, etc. and there was a fair amount of discussion about that that is worth revisiting.

But I think the key thing is that Mozilla's ISP database is an attractive single-point-of-failure target.  To be able to MITM all of the users of the e-mail app performing an initial signup when there is no mozilla-run ISP database requires gaining control of all of the networks in use by those users when they are signing up.  To be able to MITM them with a mozilla-run ISP database just requires compromising the ISP database server (and having us get to the point where we ask the ISP database).  Whether or not the user reveals their e-mail address to the ISP database is some irrelevant since the attacker can just return an entry to a server they entirely control.

Comment 13

[Jim Porter (:squib)]

Pull request up: https://github.com/mozilla-b2g/gaia/pull/6671

Comment 14

[Jim Porter (:squib)]

Fixed: https://github.com/mozsquib/gaia/commit/83cfe8ad9f05d869b5794a1835dd1975c5e32836

Comment 15

[Richard Newman [:rnewman]]

Why are we not fixing Bug 14328 and using RFC 6186 for SRV discovery?

Comment 16

[Andrew Sutherland [:asuth] (he/him)]

(In reply to Richard Newman [:rnewman] from comment #15)
> Why are we not fixing Bug 14328 and using RFC 6186 for SRV discovery?

I appreciate the DNS SRV advocacy, but this is a resolved bug about a server that we want to talk to using ActiveSync.  I don't think RFC 6186 covers activesync, and it doesn't immediately appear that hotmail.es even has SRV entries for the POP protocol we don't want to use.

Comment 17

[Jim Porter (:squib)]

(In reply to Andrew Sutherland (:asuth) from comment #16)
> (In reply to Richard Newman [:rnewman] from comment #15)
> > Why are we not fixing Bug 14328 and using RFC 6186 for SRV discovery?
> 
> I appreciate the DNS SRV advocacy, but this is a resolved bug about a server
> that we want to talk to using ActiveSync.  I don't think RFC 6186 covers
> activesync, and it doesn't immediately appear that hotmail.es even has SRV
> entries for the POP protocol we don't want to use.

Autodiscovery *can* use DNS SRV, although I don't think hotmail.es does.

Comment 18

[Nick Davidson]

Attachment: yahoo.es account unable to set automatically.

Unable to verify account set up for hotmail.es email. Please provide test email with hotmail.es domain including username /password for us to test on a device.
We are unable to set up one even we set language on device to spanish only 2 options for domain hotmail.com and live.com.
I setup a test account [e-mail] / pass: QQ111111 - and unable to set it up on a device (see attachment pic)

Comment 19

[Jim Porter (:squib)]

That yahoo.es account works fine for me. It takes about 15 seconds though, since we don't have a fast-path for that domain.

To make a hotmail.es account, click on the "Sign up now" button on the hotmail page, and then change the "EN-US" in the URL to "ES-ES".

Comment 20

[Andrew Sutherland [:asuth] (he/him)]

(In reply to Nick Dee from comment #18)
> Created attachment 697150 [details]
> yahoo.es account unable to set automatically.

For account creation errors, the screenshot of the progress dialog is not useful to us.  The page with the error message is useful, specifically the specific error code in brackets (which does not get translated), like "[no-config-info(mx404)]".  You can just re-type the error; we don't need a screenshot.

Additionally, please provide adb logcat info as described in:
https://wiki.mozilla.org/Gaia/Email/RequiredBugInfo

Comment 21

[Jim Porter (:squib)]

(In reply to Andrew Sutherland (:asuth) from comment #20)
> (In reply to Nick Dee from comment #18)
> > Created attachment 697150 [details]
> > yahoo.es account unable to set automatically.
> 
> For account creation errors, the screenshot of the progress dialog is not
> useful to us.

My guess was that it just took longer than expected, which doesn't surprise me, since we don't have a local autoconfig file for yahoo.es. Thus, we go through a bunch of other, fairly slow steps before we query the Mozilla ISPDB.

Comment 22

[Nick Davidson]

verified - on Unagi build 20130102070202 - I was able to set up [e-mail] / pss. QQ111111 and it works perfect.

however yahoo email [e-mail] / pass: QQ111111 -  unable to set it up on a device it is stack on a message "please wait while your account is set up" for more than 30 min.

just for a difference I put [e-mail] instead of [e-mail] in settings and it worked immediately (even it is ".es" account not ".com") any feedback on this?

Comment 23

[Jim Porter (:squib)]

(In reply to Nick Dee from comment #22)
> just for a difference I put [e-mail] instead of
> [e-mail] in settings and it worked immediately (even it is ".es"
> account not ".com") any feedback on this?

That's to be expected. We fast-path yahoo.com, but yahoo.es has to go through the whole autoconfig process (which involves polling Yahoo's servers for a few kinds of autoconfig file before we fall back to the Mozilla ISPDB). Since this works for me, and only takes about 15 seconds, we'd need a logcat to have any hope of diagnosing this. See comment 20.

Comment 24

[Nick Davidson]

Attachment: Log of trying to set up yahoo.es account on Unagi

Yahoo.es account still unable to set up on a device Repro on 2 devises there is no error message it is just infinite spinner indicator on a screen with message "please wait while your account is set up" (see pic) I attached log to the test case.

should we open separate bug for yahoo.es and close this one since it is for hotmail and hotmail.es account is fixed.???

Comment 25

[Andrew Sutherland [:asuth] (he/him)]

(In reply to Nick Dee from comment #24)
> Yahoo.es account still unable to set up on a device Repro on 2 devises there
> is no error message it is just infinite spinner indicator on a screen with
> message "please wait while your account is set up" (see pic) I attached log
> to the test case.

The log is troubling because it looks like our local file lookup for yahoo.es is the last thing to happen.
 
> should we open separate bug for yahoo.es and close this one since it is for
> hotmail and hotmail.es account is fixed.???

Indeed, please open a separate bug.  It made sense to post on this bug for the hotmail.es issue since you were trying to verify the original fix, but yahoo.es is decidedly a different problem since it's an IMAP connection, and from the log it seems like there might be a gecko platform issue or a systemic problem in our autoconfig mechanism.

Comment 26

[Andrew Sutherland [:asuth] (he/him)]

(In reply to Andrew Sutherland (:asuth) from comment #25)
> The log is troubling because it looks like our local file lookup for
> yahoo.es is the last thing to happen.

I'm able to reproduce this on today's build too.  It seems very specific to the ".es" suffix.  I got the same failure for the made-up "zxdfghpdfg.es" but not the made-up "zxdfghpdfg.js"

Comment 27

[Nick Davidson]

this bug is closed for similar issue with yahoo.es domain emails there is a new bug https://bugzilla.mozilla.org/show_bug.cgi?id=826468