527802 - Autoconfig small domains (even if just 1 user) on Mozilla server and publish in ISP DB

Status: RESOLVED WONTFIX

(Thunderbird :: Account Manager, defect)

Description

[Rob Mueller]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5
Build Identifier: 

The autoconfig in TB3 (https://bugzilla.mozilla.org/show_bug.cgi?id=450710) is a great step forward for usability, but there's a couple of problems for ESPs.

(Taken from email discussion)
---

1. We can't use https://autoconfig.domain.tld/ because there might be lots of potential domain.tld's hosted by a service, so you don't want to have an SSL cert for every one of them

2. We can't use http://autoconfig.domain.tld/ because both DNS and HTTP are insecure. Are you more worried about one than the other? DNS cache poisoning or HTTP man-in-the-middle modification?

3. Everything else involves some domain.tld -> service-url mapping system. We can't do a SRV/TXT DNS lookup in TB, because there's no code to do that (only A record IP lookups), and it's DNS, so it's still insecure. If you use a DB, it's limited to what's in the DB

So what if you still use a service url like https://live.mozillamessaging.com/autoconfig/domain.tld, but rather than getting a file from a fixed database stored at mozilla, it's just a proxy service instead? Basically it does some sort of SRV/TXT DNS lookup, contacts the requested server, fetches the data and returns it to the user?

Yes in theory there's still an insecure DNS lookup occuring at mozilla, but given that basically all email transfers over insecure SMTP with insecure DNS MX record lookups anyway, I'm going assume you're more concerned about this on the client side than on a server (eg. it's easier to poison/man-in-the-middle attack a user machine than a server machine).

This solution removes the need for DNS SRV/TXT lookups in TB, allows all domains to get autoconfig support (they just need the right SRV/TXT record) and in fact, there's absolutely no client changes needed at all, the only change needed is that rather than https://live.mozillamessaging.com/autoconfig/ being a directory of files, it's just a script that does some proxy work.

The most basic solution I can think of would be that for domain.tld, you do a TXT query on mozautoconfig.domain.tld, and then just use the TXT returned value as the service url.

Of course the proxy script could be smart, and it could look at a local DB first, and if not found there, then do the TXT query + proxy work. That way, you get the best of what you have now (eg gmail, yahoo, hotmail, etc work out of the box), and any providers that want to add support can do so straight away as well.

---

David Ascher said:

Sounds good to me.  In theory I think a truly decentralized model would 
be ideal, but as a next-best-thing it feels pretty good. It's also 
backwards compatible, which appeals!

I suggest you file a bug, and make sure to cc 
[e-mail], who'se responsible for 
live.mozillamessaging.com.  We can discuss the details in the bug.

---

Reproducible: Always

Comment 1

[Rob Mueller]

I'm happy to help out with whatever server side work is needed to make this happen. This would be really helpful for ESPs like us (FastMail) that have 10,000's of users using their own domain.

Comment 2

[Ben Bucksch (:BenB)]

Changing summary, because it sounded to me like a dup of bug 526496.

The question is whether we trust DNS, not whether it's client or server. IMHO, if we decide to rely on DNS, we should do that directly on the client.

Comment 3

[Philippe M. Chiasson (:gozer)]

(In reply to comment #2)
> Changing summary, because it sounded to me like a dup of bug 526496.
> 
> The question is whether we trust DNS, not whether it's client or server.

I'd lean towards trusting DNS to the extend that most e-mail infrastructure already implicitly trusts it. (Another way to say is is that it's not a less trustworthy solution) 

> IMHO, if we decide to rely on DNS, we should do that directly on the client.

I agree in general, that the best decentralized workforce for something like that is the collection of Thunderbird clients. However, since this is an important feature, and the clients lack TXT/SRV DNS query suport, we could see the server-side proxy solution as a temporary solution until we fix/implement the missing DNS components.

Comment 4

[Ben Bucksch (:BenB)]

The quick implementation solution is the one with HTTP(s) fetch to the ISP from the client ( 2) above). That's already implemented, we just need to add one function call.

Comment 5

[David Ascher (:davida)]

As I've mentioned in other channels, that change will not make it into 3.0 which is very code frozen, and as soon as we decentralized hosting of these files we make it much harder to change the schema if we need to.

Comment 6

[David Ascher (:davida)]

In addition to doing DNS-based lookups for ESPs, we could also extend this feature to do a Google-apps-for-your-domain specific check, which would give us some millions of domains, and hide the google-specificity in the server-side, which I suspect is a feature (if they change how they do things, we can update w/o having to ship & update millions of client installs).

Comment 7

[Rob Mueller]

A sample implementation is available.

http://robm.fastmail.fm/downloads/MozAutoConfig.pm

You'll need to create a config block in apache like this:

PerlModule MozAutoConfig

<Location /autoconfig/>
  AddDefaultCharset Off
  SetHandler perl-script
  PerlSetVar known_domains_path "/path/to/known/domains"
  PerlSetVar cache_domains_path "/path/to/cache/"
  PerlHandler MozAutoConfig::handler
</Location>

Here's what it does at the moment. Given a domain xyz.com to lookup, it
does these steps:

1. If xyz.com is a "known domain" (based on a stat into a directory of
   files), return the content of that file, finish
2. If xyz.com is a "cached domain" (based on a stat into a direcotry of
   files), return the content of that file, finish
3. Do a DNS lookup, TXT records for mozautoconfig.xyz.com and MX records
   for xyz.com
4. If the TXT record is present, and begins "v=mozautoconfig1 ", look at
   the remainder of the record:
  a) If it's "domain:abc.com", repeat the whole process for abc.com
  b) If it's "url:https?://def.com/something
      i) Replace %%DOMAIN%% in the url with the original domain
         requested
     ii) Fetch the resulting URL. Check the content response for sanity
    iii) Cache the response and return the content, finish
5. If the MX record is present, find the lowest priority MX record. If
   it's from a known list (currently a hardcoded hash in the code),
   find the corresponding domain, and repeat the whole process for
   that domain

So if you want to test it out, I've got it setup running at FastMail
right now. Some examples:

* A "known" domain (I've downloaded the gmail.com XML file locally)

URL: http://mozautoconfig.messagingengine.com/gmail.com

* A domain that has a TXT domain:messagingengine.com, that then has a
  TXT url. e.g.

$ dig +short mozautoconfig.fastmail.fm TXT
"v=mozautoconfig1 domain:messagingengine.com"

URL: http://mozautoconfig.messagingengine.com/fastmail.fm

* A domain that has an MX record in1.smtp.messagingengine.com, which is
  a "known MX" that maps to the domain fastmail.fm, that then repeats
  the above steps. e.g.

$ dig +short sent.com MX
10 in1.smtp.messagingengine.com.
20 in2.smtp.messagingengine.com.

URL: http://mozautoconfig.messagingengine.com/sent.com

Comment 8

[Ben Bucksch (:BenB)]

Compare bug 534722 (config directly from ISP)

Comment 9

[Jesse Thompson]

> In addition to doing DNS-based lookups for ESPs, we could also extend this
> feature to do a Google-apps-for-your-domain specific check, which would give us
> some millions of domains

Don't you know that Google lets end-users of a domain enable Apps on behalf of their domain without authorization?  It would be disastrous if you assume that just because a domain exists Google Apps that Google hosts email for that domain.

Comment 10

[Jesse Thompson]

> 5. If the MX record is present, find the lowest priority MX record. If
>    it's from a known list (currently a hardcoded hash in the code),
>    find the corresponding domain, and repeat the whole process for
>    that domain

Using MX for the "guessing" algorithmic approach to client autoconfiguration might be acceptable.  However, no implementation should assume that MX is in any way authoritative in terms of client configuration.

Comment 11

[Jesse Thompson]

As an email service administrator that hosts over 250 domains, I like your TXT records approach.  It should be easy to publish "v=mozautoconfig1 domain:hostingprovider.com" in all of the domains, and then put up https://mozautoconfig.hostingprovider.com/ on our web server.

I do worry about the DNS poisoning threat.  Your reasoning for trusting DNS more because mozilla is less susceptible to poisoning might hold true.  On the other hand, there is a lot more at stake with this.  If MX records are poisoned on a few MTAs, then a few email messages are stolen.  If these TXT records are poisoned, then account credentials can be stolen, and all hell breaks loose.

Another idea:
Is it possible to also query https://emailaddressdomain/something?
This might be more tolerable to those domains that already have a certificate and don't want to shell out for an additional certificate.
Additionally, how feasible would it be to support a redirect?  i.e. so that
https://emailaddressdomain/something or https://autoconfig.emailaddressdomain
has content that instructs your code to fetch the config from
https://autoconfig.hostingproviderdomain or https://hostingproviderdomain/something
Or something like that.  I'm not completely sure of the feasibility or implications of these ideas.  But it seems like it would be easier for domains/hosting providers to set up, which might help spur adoption.

Comment 12

[Anon]

I'll spare the perhaps not so helpful comments and get to the point - what the hell was mozilla thinking with this?

I understand there are some users who do not know the difference between an SMTP and IMAP server and agree that they shouldn't need to, but...

Despite everything I've read leading up to this not once has anyone considered the privacy implications, yet again.

- who decided that by _default_ this option should be enabled??  

- who are Mozilla to know what providers I use? 


Telling me, or the thousands of other email administrators that they potentially now have to sign up / submit documentation on/about their infrastructures to yet another outside organization is absolutely ridiculous.  I'm further puzzled by this because if indeed you are only targeting the top X providers, the documents for setting up email with those providers are quite easy to find.  

The ends do not justify the means you've taken.

I'm still trying to figure out how this got into production.  The abuse potential you've opened Thunderbird users to is astounding.  It took me all of 2 minutes to hijack this, despite wrapping the callhome in SSL and I'm not in China.  

The  only thing you've done is add more questions about what mozilla application are doing.  Then again, this seems to be common from Mozilla lately.

Comment 13

[Matthias Versen [:Matti]]

>I understand there are some users who do not know the difference between an
>SMTP and IMAP server and agree that they shouldn't need to, but...
right..

>- who decided that by _default_ this option should be enabled??  
this isn't enabled at all, there is not even code for it and that's the reason why this bug is still marked NEW and not "fixed".

>- who are Mozilla to know what providers I use? 
They don't know and they don't have to. It doesn't matter...

>Telling me, or the thousands of other email administrators that they
>potentially now have to sign up / submit documentation on/about their
>infrastructures to yet another outside organization is absolutely ridiculous. 

They (administrators) don't have to submit any information if they don't want it.

>I'm still trying to figure out how this got into production.
This is not in production and there is no code and no patch for this as you can easy see in this bug.

>The abuse potential you've opened Thunderbird users to is astounding.  It took 
>me all of
>2 minutes to hijack this, despite wrapping the callhome in SSL and I'm not in
>China.  

If you can redirect SSL you can redirect the user email traffic. Where is the additional security risk then ?

Comment 14

[Anon]

You may be correct, this was posted to the wrong bug I think.  I am sorry.  
Perhaps it  should have gone under 450710 which did implement the feature via HTTP.  

When I created a new account I noticed this background request:

GET /autoconfig/foo.com HTTP/1.1|Host:live.mozillamessaging.com|User-Agent:Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv%3a1.9.1.9) Gecko/20100317 Thunderbird/3.0.4|Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8|Accept-Language:en-us,en;q=0.5|Accept-Encoding:gzip,deflate|Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.7|Keep-Alive:300|Connection:keep-alive

... and the basis for my frustration above.

The issues mostly stem from this being tried _first_, vs. asking the user.

I would actually prefer this, if at all, be a simple <domainname>/<somefilename>  , or in DNS as an SRV.  There is then no single point of control outside of that domain.

SSL out of the box gives you encryption between two points, but not guaranteed authentication or secrecy end to end.  So it is true that I can redirect email and DNS if I wanted to, but only if they are somewhere on my network.  With what is presented here, I would be able to regardless of where they are.  Configuration vs. MITM.  

Unless this was to happen all the time, in which case there are other issues.

Sorry again.

Comment 15

[Ben Bucksch (:BenB)]

> Perhaps it should have gone under 450710

No. This belongs on the newsgroup, if anywhere. It is bound to be a heated discussion (if taken seriously).

> <domainname>/<somefilename>, or in DNS as an SRV

Well, before you post, do your homework by reading the docs and the previous discussion. There is the wiki page, 3-4 newsgroup threads over time of 2.5 years, and 4 bugs (which you'll all easily find while you read). There were concrete reasons why we did the things like this.

Comment 16

[Ben Bucksch (:BenB)]

We now have better means to fulfill the original requirements.
WONTFIX