221030 - Unable to set up IMAP over SSL using the New Account Wizard

Status: RESOLVED FIXED

(Thunderbird :: Account Manager, defect, P1)

Description

[mkhusid]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20030916
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20030916

The New Account Wizard does not allow to set up IMAP-SSL server.

This problem may seem as something cosmetic for a single user application (just
go to Tools/Account Settings, prefs.js and fix it), but it is nearly a
showstopper for a multi-user deployment. I am working on a site with up to
several hundred users now. The site OS are Redhat Linux and Solaris 9.

Reproducible: Always

Steps to Reproduce:
1. Run a new installation of Thunderbird. A new account wizard (inherited from
mailnews) pops up.
2. Set up incoming server type as IMAP and incoming server as your-server.ip
3. Complete the wizard.


Actual Results:  
Thunderbird assumes that a server is non-SSL and defaults to port 143.
A few unacceptable things happen:
1. A password is prompted. My presumption if user enters password, it will be
sent unencrypted via the network. BAD IDEA.
2. The server will not validate the logon (the server is supposed to tell client
that no connections are accepted on port 143).
3. A user gets an error message that the password is wrong. An error message
itself is quite misleading.

Expected Results:  
Allow to set up SSL type of the incoming server.

Comment 1

[mkhusid]

Forgot to mention. I posted on forums.mozillazine.org but got no reply. I can
volunteer to implement a fix myself if someone can guide me in the right dimension.

Comment 2

[Phil Karn]

I agree that this is a very serious problem for many naive users. At least some
very widely used IMAP servers disallow unencrypted connections; I think this is
actually required by the IMAP spec. So the new account wizard should
*definitely* ask the user if the connection is to be encrypted. Better yet, it
should try it both ways and figure it out on its own.

Also, some imap servers (like uw-imapd) require the client to present a
directory name where the IMAP folders are kept, and will not work without it.
This is common enough that the new account wizard should prompt for this setting
as well.

Comment 3

[lloyd]

Attachment: AccountWizard.dtd

Comment 4

[lloyd]

this has been an annoyance for me too for some time.  i have some code that
might be a fix.  i'm new to xul and javascript, though - some review would be
nice.  seems to be working for me.

there's a modified locale dtd - i only did en-US.

i haven't contributed before, so i don't know the mechanics.  i attached some diffs:

/mozilla/mailnews/base/prefs/resources/locale/en-US/AccountWizard.dtd
/mailnews/base/prefs/resources/content/AccountWizard.js
/mailnews/base/prefs/resources/content/AccountWizard.xul
/mailnews/base/prefs/resources/content/aw-server.js
/mailnews/base/prefs/resources/content/aw-done.js

Comment 5

[lloyd]

Attachment: AccountWizard.js

Comment 6

[lloyd]

Attachment: AccountWizard.xul

Comment 7

[lloyd]

Attachment: aw-done.js

Comment 8

[lloyd]

Attachment: aw-server.js

Comment 9

[Fabian Rodriguez:MagicFab]

Can you add an attachment of a screenshot ?

Comment 10

[lloyd]

Attachment: screenshot - account wizard - server

Comment 11

[lloyd]

Attachment: screenshot - acct wiz - "done" page

Comment 12

[lloyd]

i should mention that i made the code changes to:  Mozilla 1.8a3, build 2004072106

so, this isn't against the thunderbird code.  my assumption was that this is
something we'd want in moz and thunderbird, and thunderbird branches off of
mozilla, correct?

Comment 13

[Stephen Donner [:stephend] Not actively reading bugmail]

lloyd, you have to ask for reviews:

http://www.mozilla.org/hacking/code-review-faq.html

Comment 14

[David :Bienvenu]

Lloyd, can we get one complete -u diff for all your changes? I think this
enhancement is something we want...

Comment 15

[lloyd]

sorry, i'm not even sure if i have the code anymore.  it's been about 18 months.
 i'm not merging this into my own build, i just offered the code in hopes that
it would find itself into the regular build (or at least inspire someone else to
improve/rewrite it), since it's an issue we ran into repeatedly with our users.

hopefully the code that's there is useful, though i suspect someone who's had
more than about an hour of familiarity with the codebase (which is about what i
had) should be able to polish it off pretty quickly.

IAC, i'll search my drive at work in the next couple days.

Comment 16

[Magnus Melin [:mkmelin]]

Attachment: mockup of serverinfo page: pop vs imap, pop too long

I have been looking into this. I guess we should allow TLS and secure
authentication also. Problem is, for POP the server info page becomes too long.


So, something would have to move
- move smtp server setting to page of its own?
- global inbox settings?

What do you think?

Comment 17

[Magnus Melin [:mkmelin]]

Attachment: proposed fix

Went ahead and created a patch. This patch
- adds the security options for pop/imap
- also adds a checkbox for secure news 
- moves the global inbox setting to the finish page

This needs both r and sr right?

Comment 18

[Magnus Melin [:mkmelin]]

Attachment: Screenshots for the patch in attachment 186130

Comment 19

[David :Bienvenu]

Comment on attachment 186130 [details] [diff] [review]
proposed fix

switching r/sr - no sense in my looking at this if Scott doesn't want it :-)

Comment 20

[Alexander Sack]

David, maybe you can push mscott to at least take a look at this and drop a short statement why this is not feasible/considered?

Comment 21

[David :Bienvenu]

For the next release, we should consider this.  Ideally, we'd do an auto-detection of whether SSL or TLS works, and auto-configure.

Comment 22

[Alexander Sack]

David, thanks. you refer with "next release" to thunderbird 2 or next 1.8.0 branch release?

Comment 23

[David :Bienvenu]

neither - the next (probably major) release after 2.0

Comment 24

[rsx11m]

(In reply to comment #21)
> For the next release, we should consider this.  Ideally, we'd do an
> auto-detection of whether SSL or TLS works, and auto-configure.

I found this filed as bug 394487 along with a suggestion for a possible scan order. Judging from the increasing number of "cannot connect" posts at MZ forums, and the fact that more and more providers are requiring encryption these days, having more options in the initial account setup is crucial. First-time users may be frustrated twice by first not finding any option in the account wizard to specify the port and encryption method obtained from their providers, then again when the first connection fails. Taking the guesswork out by a scanning mechanism would certainly be the best solution, I agree.

A few more thoughts:
 - Include SMTP in this bug, as providers are more likely to require encryption also for outgoing e-mails these days.
 - Offer port options rather than just encryption, e.g., TLS may be on either port 25 or 587; or, add the port to the attachment 186131 [details] layout.

Possible port options to select in the account wizard:
 - POP3:  110 (implies try TLS), 995 (use SSL)
 - IMAP:  143 (implies try TLS), 993 (use SSL)
 - SMTP:   25 (implies try TLS), 465 (use SSL), 587 (try TLS)

Note that none of the options has "no encryption" by default, assuming the "try TLS" option can be used if TLS is not supported by the server. 

Comment 25

[Marcia Knous [:marcia]]

Nominating for Tbird 3.

Comment 26

[kami petersen]

as google is rolling out free imap over ssl to access gmail accounts this suddenly becomes rathter more important... as this convoluted setup instruction reflects:

http://mail.google.com/support/bin/answer.py?answer=77662

Comment 27

[rsx11m]

Supporting comment #25. However, ideally a solution for the next major release should extend the options for all three protocols (IMAP, POP, SMTP) in the Account Wizard.

Updating the special setup dialogs for Gmail's new IMAP access is handled in bug 400931 as an enhancement request. Given that many - if not most - providers require special port and encryption settings for their services, the issue goes well beyond Gmail. It seems more desirable though to have a solid generic setup interface than trying to provide separate ISP-specific "easy setup" dialogs.

Comment 28

[Magnus Melin [:mkmelin]]

Comment on attachment 186130 [details] [diff] [review]
proposed fix

Obsoleting, badly bitrotted.

Comment 31

[Boying Lu]

Attachment: patch

Comment 32

[Boying Lu]

Attachment: SmtpServerSetting.js

Put this file under mailnews/base/prefs/resources/content

Comment 33

[Boying Lu]

Attachment: SmtpServerSettings.xul

Put this file under mailnews/base/prefs/resources/content

Comment 34

[Boying Lu]

Attachment: screenshot

Comment 35

[Boying Lu]

Attachment: screenshot

Comment 36

[Magnus Melin [:mkmelin]]

Boying: to include new files in the patch you can use the "cvsdo add" command and generate the patch with something like cvs diff -upN9. (cvsdo is part of cvsutils)

Also, "Advance" should probably be "Advanced..." 

Comment 37

[Boying Lu]

Attachment: new patch

1. add include new files
2. change "Advance" to "Advanced"
3. change the access key from "a" to "A"
4. clean up codes in the previous patch

Comment 38

[Boying Lu]

Attachment: new screenshot

Comment 39

[[e-mail]]

Comment on attachment 314535 [details] [diff] [review]
new patch

I've discovered that there's a plan for this; I'll post the URL in a sec. It's based on your patch for bug 426497 but to reduce clutter the global inbox checkbox is moved to the last page and menulists are used instead of radiogroups.

Comment 40

[[e-mail]]

Bug 326076 is another place that switched from a radio group to a menulist.

Comment 41

[Magnus Melin [:mkmelin]]

SSL is getting very common, really think we need to sort this out for tb3.

Comment 42

[Bryan Clark (DevTools PM) [:clarkbw]]

There are some designs in progres sketched out at:
http://wiki.mozilla.org/MailNews:Account_Wizard
and specifically for email:
http://wiki.mozilla.org/MailNews:Account_Wizard:Email

Comment 43

[rsx11m]

> (comment #42) http://wiki.mozilla.org/MailNews:Account_Wizard:Email#Page_3

I like that the menu lists use less space and would be extendable for further encryption protocols (if any new ones come up). The authentication on Page 4 is good too and separates connection and authentication better than having both on the same page. However, I still think that the *port* numbers should be offered somewhere, which initially can be set to the respective defaults as done now. Keep in mind that the port may not be uniquely derivable from the encryption protocol used, especially for SMTP (e.g., port 25 vs. 587).

Comment 44

[rsx11m]

This is Page 3 extended by port fields next to the server name, corresponding to the current Server Settings tab. While comment #34 introduced a separate page with "Advanced" button, this may be a little more "cluttering" but avoids a separate page for the port:

   /Choose:/
   [*] POP [ ] IMAP

   /explanatory text/

   Incoming Server: [ mail.doe.mail     ] Port: [110]
   [ ] Use Secure Connection [  TLS, if available \/]

   ----

   /explanatory text/

   Outgoing Server: [ smtp.doe.mail     ] Port: [ 25]
   [ ] Use Secure Connection [  TLS, if available \/]

   [[ < Back ]] [[ Next > ]] [[ Cancel ]]

Comment 45

[Bryan Clark (DevTools PM) [:clarkbw]]

That looks good to me.  I don't think it adds too much clutter considering it's necessity.

Can you update the wiki page to reflect this change?

Comment 46

[rsx11m]

> (comment #45) Can you update the wiki page to reflect this change?

Done. I've also updated Page 6 (summary page) to reflect the additional options, please modify any of those as you see fit.

Comment 47

[Ronald Killmer]

These proposed revisions will enhance User configuration for the U.S. Dept. of Defense which uses SSL for IMAP and SMTP connections. I suspect that the other governmental Depts are similar.

Comment 48

[Bryan Clark (DevTools PM) [:clarkbw]]

- some updates, already put into bug 326076 -

The Account Wizard wiki has been updated to reflect some discussion coming out
of the wiki talk page.  This includes a better menu list item.

Also bug 422814 has some work on auto-probe for connection type and port config

Comment 49

[Matthew Andrews]

I'm not certain of the privacy implications, but perhaps following Outlook 2007's lead and guess the proper ports/security settings, working from more secure to less secure?

Comment 50

[David :Bienvenu]

moving to b2 - this is completely dependent on the new account config stuff landing, which may or may not happen for b1, but it's not useful to track this separately.

Comment 51

[David :Bienvenu]

still trying to get this for b2, but not blocking b2

Comment 52

[David :Bienvenu]

the new mail acount setup wizerd in bug 422814 fixes this.