Closed Bug 450710 Opened 13 years ago Closed 12 years ago

Mail account auto-configuration via a database of ISP configs fetched via HTTP

Categories

(Thunderbird :: Account Manager, enhancement, P1)

enhancement

Tracking

(Not tracked)

RESOLVED FIXED
Thunderbird 3

People

(Reporter: BenB, Assigned: BenB)

References

(Depends on 1 open bug, )

Details

Attachments

(1 file)

Related bugs (other approaches to do autoconfig): bug 342242, bug 422814.
Priority: -- → P1
Flags: wanted-thunderbird3?
This is the code for
- a representation of account configuration, in a JS object
- reading a config from XML
  <https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat#XML>
- creating an account (IMAP or POP3 and SMTP) in the backend
  (hey, that actually works, surprinsingly!)
- Test code which creates an account ben-gmail.com, hooked up in the File menu.

bienvenu, can you please review?
David Ascher, if you want, you can take a look, too.

I am now ready to integrate with you guys.
Next steps for me would be:
- Hook up with UI, to:
  - Enter values
  - Fetch config from network
- Wrap bienvenu's guess code in a function returning an AccountConfig
Attachment #334235 - Flags: review?
Attachment #334235 - Flags: review? → review?(bienvenu)
I wounder if we could read some values from XML to set them in configuration editor accordingly. I remember this is was part of idea.
I don't know what you mean. But directly implements the first steps of <https://wiki.mozilla.org/Thunderbird:Autoconfiguration>.
There are lot more settings I would like to control from this file, like delete model, junk setting, etc.
https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat - page says its in TODO currently (All settings and enum values)
Can we take this elsewhere? This has nothing to do with the code.
Please email me about it which exact settings you need, in which situation/role and why.
FYI, the settings are intentionally limited, as I don't want to give ISPs too much freedom, leave a lot to us (defaults) and the user (preferences).
Surprisingly, this even works when there are no accounts yet and it's creating the first account.
For the record, Nikolay Shopik's concern was about Thunderbird enterprise deployment and preconfiguration (in his case just 50 users). I considered that briefly, but haven't thought about that in much detail. It's clear that I don't want to give ISPs the level of freedom of configuration that some enterprise deployments may ask for (admins in a company have more rights to pick options for users than ISPs have for their customers). Main question is how to differentiate. Probably by config deployment method - allow the config file to be on disk, and then give it more freedom, but restrict it more when fetched from the web/DB. I consider that something to think about and work on after the ISP stuff is done, as that has much higher priority for me. (I do care about enterprises, though.)
There are Mission Control Desktop
http://developer.mozilla.org/en/docs/MCD%2C_Mission_Control_Desktop_AKA_AutoConfig
which is slightly unarranged, but probably do job
Patch 1 (minor stuff fixed) checked into
https://hg.mozilla.org/users/dascher_mozilla.com/autoconfig/
per permission from David A. on IRC. (Also updated the branch to comm-central trunk.)
I don't like the idea that Mozilla will need to maintain the mail configurations server, and I think it will take too much manpower for us. 

Instead, I've described my idea in bug 411898, and it is basically to use the DNS SRV records to give the mail providers the ability to control the configurations from their own domain, and giving other mail software vendors the ability to use the same interface.  
Tomer, please read carefully https://wiki.mozilla.org/Thunderbird:Autoconfiguration which describe not only this idea.
...Alternatively, just try to contact https://autoconfig.emailaddressdomain/mail/mozilla.xml?emailaddress=emailaddress and see whether that host/URL exists.
The nice thing about autoconfiguration is that multiple approaches can be tried.  I think we can get quite far with port probing. Even further with a few hosted config files for large ISPs.  And even further with distributed approaches.  We can have as many approaches as people are interested in coding, with some obvious maintenance challenges with some approaches if they scale.  There are technical issues w/ the DNS based approach that make it hard to try now, as dmose pointed out in bug 342242 comment #15.

related: bug 367499
Those interested in this bug might like to know about our effort for collecting ISP data:
http://spreadsheets.google.com/ccc?key=p49SW32nNYX0otkRc3UZUJA&hl=en_GB

Gerv
Gerv -- yeah, thanks we know. It'd be good to make sure that what we're collecting in the spreadsheet maps well to the AccountConfig stuff we need.

When it's further along, we can add the ability for the dialog, upon failure to look up the file, but successful configuration (either through probing or manual configuration), to submit that data (w/ no personal info) back to us for review.

We also need to come up with a way of reviewing these configuration files, to ensure they're accurate (and secure).
An approach that allowed for detection via either using the NS or MX records for a domain would be pretty handy for anyone who hosts their email with their web host (including many small businesses).  This would also facilitate catching Google Apps users.

Remember, mail provider may not necessarily be running the DNS (Google is a major example, so is bigfish.com, etc.).

I think a list of 20 or so would go a long way to tackle many small business email configurations.
Robert look for sepparate bug 342242
Comment on attachment 334235 [details] [diff] [review]
Logic code for account creation

clearing request - I suspect this has changed somewhat; we should review this stuff once it's basically all working together, I think.
Attachment #334235 - Flags: review?(bienvenu)
Duplicate of this bug: 447943
Flags: wanted-thunderbird3? → wanted-thunderbird3+
Depends on: 407190
This auto configuration feature is brilliant and badly needed.  We run a number of 100+ user windows terminal servers and setting up Email clients for users is a repetitive, brainless task that just takes too long.  This proposed automation would go a long way to resolving the issue. 

Is this feature available and testable in the current vsn of Shredder  (3.0b2pre) ? Im using the nightly builds and can test it in a Win2k3r2 x64 environment.
Blocks: autoconfig
Any news here?
When we can see this great feature working?

And... Someone know what ISP have https://autoconfig.emailaddressdomain/mail/mozilla.xml?emailaddress=emailaddress??

Regards,
Renato
I understand that this proposal assumes that users will always have their email addresses under the provider's domain(s), like in typical freemail providers (which are a minority compared to non-free and corporate). Hence, it does not cover the case where a single provider gives email accounts under arbitrary domains (for example, domains that have been previously purchased by the user). Actually, this is the usual case for non-free email providers.

Moreover, the statement "I don't want to give ISPs too much freedom" is intriguing: for example, do you think that letting the ISPs to set the names of the "Junk" and "Sent" folders (like other settings required to interoperate with webmail and other clients) is unacceptable?
> the case where a single provider gives email accounts under arbitrary domains

There are tricks to make this work, e.g. to check the domain of the MX listed for the email domain. In the cases you mentioned, example.com would probably have an MX entry of mail-inbound.provider.com or similar, so we could check https://autoconfig.provider.com/.... I haven't implemented yet yet, though.

> that letting the ISPs to set the names of the "Junk" and "Sent" folders
> ... webmail

This has been suggested in bug 475139. (I myself haven't really made up my opinion yet.)
Renato, this feature is checked in as part of bug 422814, but hasn't been enabled in nightly builds yet.
> example.com would probably have an MX entry of mail-inbound.provider.com

oh, nevermind. We had that proposal in the discussion on the newsgroups, but there were security concerns due to DNS being insecure. OTOH, we now show the configuration to the user, so the threat is lessened. I'll consider the domain hosting case when we reconsider the DNS approach, thanks for bringing it to attention again.

I think this feature, as is, can already help the huge amount of users with @bigisp.com and @bigwebmailer.com email addresses, though.
(In reply to comment #28)
> oh, nevermind. We had that proposal in the discussion on the newsgroups, but
> there were security concerns due to DNS being insecure. OTOH, we now show the
> configuration to the user, so the threat is lessened. I'll consider the domain
> hosting case when we reconsider the DNS approach, thanks for bringing it to
> attention again.
Problem not just because DNS insecure but because Gecko doesn't have code for requesting SRV records.
(In reply to comment #29)
> Problem not just because DNS insecure but because Gecko doesn't have code for
> requesting SRV records.

This is right, but I wonder why 99% of the discussion is around deployment techniques like DNSSEC or web services, instead of focusing on the core functionality.

A simple import/export functionality like the one I proposed in bug https://bugzilla.mozilla.org/show_bug.cgi?id=389275 would solve the issue for most purposes (included the simple case of some user willing to disseminate TB configurations for his/her non-expert friends), and deployment/extensions could be discussed later.
I implemented this as part of bug 422814.

FIXED.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Blocks: 527802
I am a bit confused over what is actually implemented regarding fetching data from the ISP database that is created.

The algorithm I see do look up the data given the domain name in the email address.

I would like the algorithm to be like this:

1. Use the domain name in the email address, and try to fetch data from the ISP database (or from the ISP or whatever is now implemented)

2. If there is no data found in step 1, repeat step 1 with the domain name that is the target  of the MX of the domain in the email address.

Is that what you implemented Ben? It seems a bit unclear to me...
Patrik,
what's implemented is documented on
<https://wiki.mozilla.org/Thunderbird:Autoconfiguration>.
1. Config files on harddisk, in <installdir>/isp/*.xml.
3. Try to find the config at the Mozilla server
4. Guess config

We don't use MX, because that's DNS and that's insecure, although I changed my opinion on that based on several facts and might implement something like that (and other DNS-based stuff) in the future.
You need to log in before you can comment on or make changes to this bug.