Closed Bug 286581 Opened 19 years ago Closed 14 years ago

IMAP AUTHENTICATE EXTERNAL with TLS and given client certificate

Categories

(MailNews Core :: Networking: IMAP, enhancement)

enhancement
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
Thunderbird 3.3a2

People

(Reporter: goetz, Assigned: Bienvenu)

References

(Blocks 2 open bugs, )

Details

(Whiteboard: [GS][gssolved])

Attachments

(1 file, 3 obsolete files)

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.2) Gecko/20040906
Build Identifier: Thunderbird 1.0

After the successfull TLS handshake with a given client certificate
you may use the authentication method EXTERNAL and use the credentials
in the client certificate to authenticate the client.

This would allow to log in without giving a pass phrase.



Reproducible: Always

Steps to Reproduce:
1. Setup an IMAP server to use TLS,  configure it to support client certificates
2. load a client cert appropirate for the IMAP server into Thunderbird
3. connect to the IMAP server
4. A Box for the pass phrase of the security will pop up
5. A client certificate is selected automatically and send to the server
6. even if the server signals it supports "AUTH=EXTERNAL"
   the authentication CRAM-MD5 is choosen
   (and a pass phrase for the server is asked)


Actual Results:  
I was asked for the pass phrase for the server.
(CRAM-MD5 as authentication was choosen)


Expected Results:  
choose the authentication method EXTERNAL to let the server
fetch the authentication from the client certificate...


(sanitized) server log file extract of thunderbird access:
(server is cyrus imapd)

[...]

Mar 17 17:29:48 hal cyrus/1993[31083]: accepted connection
Mar 17 17:30:00 hal cyrus/1993[31083]: Doing a peer verify
Mar 17 17:30:00 hal cyrus/1993[31083]: Doing a peer verify
Mar 17 17:30:00 hal cyrus/1993[31083]: mystore: starting txn 2147483650
Mar 17 17:30:00 hal cyrus/1993[31083]: mystore: committing txn 2147483650
Mar 17 17:30:00 hal cyrus/1993[31083]: received client certificate
Mar 17 17:30:00 hal cyrus/1993[31083]: subject=[...]/CN=goetz
Mar 17 17:30:00 hal cyrus/1993[31083]: starttls: TLSv1 with cipher AES256-SHA
(256/256 bits new) authenticated as goetz
Mar 17 17:30:06 hal cyrus/1993[31083]: login: hal.shomitefo.de [192.168.201.5]
goetz CRAM-MD5+TLS User logged in

(sanitized) server log file extract with imtest
(imtest -t goetz.pem -m EXTERNAL -u goetz -a goetz hal.shomitefo.de)

[...]
Mar 17 17:50:23 hal cyrus/1143[31199]: accepted connection
Mar 17 17:50:26 hal cyrus/1143[31199]: Doing a peer verify
Mar 17 17:50:26 hal cyrus/1143[31199]: Doing a peer verify
Mar 17 17:50:26 hal cyrus/1143[31199]: mystore: starting txn 2147483661
Mar 17 17:50:26 hal cyrus/1143[31199]: mystore: committing txn 2147483661
Mar 17 17:50:26 hal cyrus/1143[31199]: received client certificate
Mar 17 17:50:26 hal cyrus/1143[31199]: subject=[...]/CN=goetz
Mar 17 17:50:26 hal cyrus/1143[31199]: starttls: TLSv1 with cipher AES256-SHA
(256/256 bits new) authenticated as goetz
Mar 17 17:50:26 hal cyrus/1143[31199]: login: hal.shomitefo.de [192.168.201.5]
goetz EXTERNAL+TLS User logged in 

(no ask for IMAP pass phrase)
see also Bug 290787 and Bug 28200
These bugs require some bigger changes.

Adding Auth EXTERNAL requires only few changes:
Pseudo code, I didn't look into the Thunderbird sources:

After started SSL secured session (port 993 or after STARTTLS):

if (capabilities_string_contains("AUTH=EXTERNAL"))
{
   loginNameB64 = do_base64(get_login_name());
   sendLine("AUTHENTICATE EXTERNAL %s",loginNameB64);
   if(serverAnswerIsOK())
      return AUTHENTICATED;
}
... continue with normal authentication functions
This is an automated message, with ID "auto-resolve01".

This bug has had no comments for a long time. Statistically, we have found that
bug reports that have not been confirmed by a second user after three months are
highly unlikely to be the source of a fix to the code.

While your input is very important to us, our resources are limited and so we
are asking for your help in focussing our efforts. If you can still reproduce
this problem in the latest version of the product (see below for how to obtain a
copy) or, for feature requests, if it's not present in the latest version and
you still believe we should implement it, please visit the URL of this bug
(given at the top of this mail) and add a comment to that effect, giving more
reproduction information if you have it.

If it is not a problem any longer, you need take no action. If this bug is not
changed in any way in the next two weeks, it will be automatically resolved.
Thank you for your help in this matter.

The latest beta releases can be obtained from:
Firefox:     http://www.mozilla.org/projects/firefox/
Thunderbird: http://www.mozilla.org/products/thunderbird/releases/1.5beta1.html
Seamonkey:   http://www.mozilla.org/projects/seamonkey/
Still open in 1.5b1

I really would like to authenticate to the IMAP server just using client
certificates.

This would allow to remove the user pass phrase on the server
OS: Linux → All
Version: unspecified → Trunk
still not working? see Bug 322145 and Bug 327001 
Since I upgraded my mail server to Debian Etch he has a problem
and AUTH=EXTERNAL isn't working anymore... :-(
I haven't tested TB 2.0.0.0.
But my old TB (1.5.0.10) wasn't able to do AUTH EXTERNAL with the old server...
I work on Sun Java System Messaging Server, and have had reports from customers
who are unhappy that Thunderbird always prompts for the password when using
STARTTLS with client certifictes, even when SASL EXTERNAL is advertised by
the server.  I intend to refer them to this bug.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Assignee: dveditz → bienvenu
Component: Security → Networking: IMAP
Product: Thunderbird → Core
QA Contact: thunderbird → networking.imap
Hardware: PC → All
Product: Core → MailNews Core
GoeTz have you been able to test with Tb 2.0 ? is it working better now ?
Unfortunately the missing STARTTLS support somehow dropped from the shelf, because there always were other more pressing things to do.

I've checked my servers log and it still shows auth method CRAM-MD5+TLS

But that could be a problem in my servers configuration...
Bug 515371 doesn't make much sense without this being implemented first.
Blocks: 515371
(In reply to comment #10)
> Bug 515371 doesn't make much sense without this being implemented first.

No, GSSAPI is also a password-less login, and we support GSSAPI.
Authentication is not IMAP deepened so I suggest you to change component, usually this cover POP3/IMAP/SMTP but not LDAP.
Assignee: bienvenu → nobody
each protocol needs to have the authentication support added separately, so an other alternative is to file separate bugs on the other protocols.
It's not really clear for me here does AUTHETICATION=EXTERNAL is actually implement? And this bug describing problems ONLY when TLS connection enabled for AUTHETICATION=EXTERNAL?
Attached patch wip (obsolete) — Splinter Review
if you turn on use secure auth for the imap server, and run with this patch, TB will do an authenticate external against the server.
Assignee: nobody → bienvenu
Thanks David, but I assume you probably wanna land this after Bug 525238 is landed for UI consistency?
Nikolay, yes, I'm aware of bug 525238 and I don't intend to bitrot that patch. The hard part will be to get this patch working. It will be easy to port it to the post bug 525238 world.

Which reminds me, I guess we should only try this mechanism if the user has enabled SSL or TLS (though auth=external is purposely vague about what the external mechanism is).
Maybe we could put additional option like bug 525238 patch doing right now for other types auth, specially for AUTH EXTERNAL. And when this option selected we could force SSL/TLS or warn user.
this builds against the trunk, though I haven't run it.
Attachment #432600 - Attachment is obsolete: true
Attached patch patch that builds against 3.1rc1 (obsolete) — Splinter Review
forgot to mention that my patch (which is mostly David's with a few more changes) is somewhat neutered by bug 568929 It is still usable, and I have been using it against my test server for about a week now.
Comment on attachment 448079 [details] [diff] [review]
patch that builds against 3.1rc1

getting onto my review queue - one change is that we'll need the suite string added
Attachment #448079 - Flags: review?(bienvenu)
Comment on attachment 445163 [details] [diff] [review]
patch that builds against the trunk (untested)

this looks good to me.
Attachment #445163 - Flags: review+
Comment on attachment 448079 [details] [diff] [review]
patch that builds against 3.1rc1

clearing 3.1 version review request - we need to land this on the trunk first and get it a bit of baking.
Attachment #448079 - Flags: review?(bienvenu)
Comment on attachment 445163 [details] [diff] [review]
patch that builds against the trunk (untested)

oops, we want the 3.1 patch, but modified so that it's a hg qimportable...I'll attach in a sec.
Attachment #445163 - Flags: review+
bienvenu, we were talking about how a more specific AUTH type would do good: AUTH SSL-CLIENT-CERT, AUTH IPADDR, AUTH WHAT-OTHER-THING-DID-WE-HAVE-THERE .
I'd much rather not have "EXTERNAL" establish as standard, because I consider it to be too unspecific (with assumptions, like SSL client cert selection dialog and similar). Can we wait with supporting this "standard" until we have this sorted out?
This adds an other auth mechanism, AUTH EXTERNAL, which in the real world is a TLS cert, at least for IMAP, though technically it could be an IP address or any other external auth mechanism. So I think External would be more generally accurate, but TLS Certificate would be more specific and meaningful.
Attachment #445163 - Attachment is obsolete: true
Attachment #448079 - Attachment is obsolete: true
Attachment #480669 - Flags: ui-review?(clarkbw)
Ah, and as if proving my comment 27, you just added a good example why it matters :)
> authExternal=TLS Certificate
(in UI)
(In reply to comment #27)
> bienvenu, we were talking about how a more specific AUTH type would do good:
> AUTH SSL-CLIENT-CERT, AUTH IPADDR, AUTH WHAT-OTHER-THING-DID-WE-HAVE-THERE .
> I'd much rather not have "EXTERNAL" establish as standard, because I consider
> it to be too unspecific (with assumptions, like SSL client cert selection
> dialog and similar). Can we wait with supporting this "standard" until we have
> this sorted out?

EXTERNAL is a standard that exists in the real world and even if we get a more specific RFC proposal adopted by server vendors, it could take years before it sees the light of day. In the meantime, I'd like to support what's out there...

I agree that the UI choice shows how vague EXTERNAL is, but I think the situation is a bit different for IMAP than SMTP, in that EXTERNAL for IMAP most likely means a TLS Cert. I kinda tend to prefer accuracy (External) but I'd like Bryan's opinion.
> EXTERNAL is a standard that exists in the real world

Is it? I had the impression that the support is scarce, on both server and client side. I.e. there are a few implementations, but not many, and therefore hardly deployed and used. I just don't want to help it spread, because I think it's badly specced, that's all.

You may want to support it in autoconfig XML, too. I just added "TLS-client-cert" as auth method to the definition <https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat#XML>.

Also, given that it's so rarely used, I think we should hide the UI option, unless it's used. Those sites that use it need infrastructure anyway and they can put up an autoconfig server.
Whiteboard: [GS] http://gsfn.us/t/t4f9
> I kinda tend to prefer accuracy (External)

I think in the UI we should speak plainly, saying what it really is in practice, i.e. "TLS client certificate" (note the "client" vs. patch).
(In reply to comment #31)
> > EXTERNAL is a standard that exists in the real world
> 
> Is it? I had the impression that the support is scarce, on both server and
> client side. I.e. there are a few implementations, but not many, and therefore
> hardly deployed and used. I just don't want to help it spread, because I think
> it's badly specced, that's all.

I think the implementations aren't scarce; it's more the deployments. I.e., most server admins don't turn it on.
 
> 
> You may want to support it in autoconfig XML, too. I just added
> "TLS-client-cert" as auth method to the definition
> <https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat#XML>.

Yes, thanks, I was going to ask you about that.

> 
> Also, given that it's so rarely used, I think we should hide the UI option,
> unless it's used. Those sites that use it need infrastructure anyway and they
> can put up an autoconfig server.

I'm not sure we can hide the UI option now that we've gone down the road of letting the user pick the auth mechanism, because we need to show that some auth mechanism has been chosen. Which is a bit of a slippery slope...
Yeah, that's why I said "hide the UI option, unless it's used"... I already do the same for some other auth methods, it's a one-liner to make this "hide unless used", see the prefs UI code.
(In reply to comment #31)
> Also, given that it's so rarely used, I think we should hide the UI option,
> unless it's used. Those sites that use it need infrastructure anyway and they
> can put up an autoconfig server.

The flip side to that would be; it is already hard enough to deploy, please don't make it harder.
SASL EXTERNAL has been IETF standards track since 1997 (RFC 2222).  For IMAP/POP with STARTTLS, RFC 2595 requires that the server remain in unauthenticated state after STARTTLS.  So "EXTERNAL" is the one and only IETF standards track way to authenticate using a client certificate with IMAP/POP.

IMAPS/POPS were never standardized and when it comes to client certificates so both variants of server implementation (enters authenticated state or not after client cert provided) are valid, and there's no way to tell the difference between the two with POPS.  That means there is only one interoperable way to do POP+TLS+client-certificates: the way described in RFC 2595.  For IMAPS, a client can at least key off IMAP's "PREAUTH" greeting so it can interoperate with both valid protocol variants (client should use AUTHENTICATE EXTERNAL if the banner isn't PREAUTH).

If the server allows the client to use "EXTERNAL", then the client isn't sending a password, so it's a more secure option for the client than any mechanisms vulnerable to eavesdropping, dictionary or brute force (LOGIN, PLAIN, CRAM-MD5, DIGEST-MD5, etc).  It should thus be used by default, at least when the client sends a client certificate via TLS first.

Turning on client certificates in a server is a pain: the admin has to install a trusted CA for signing client certificates, and specify the algorithm to map from the certificate subject to a useful user identity in the server's back-end identity system (in our case, mapping from the client certificate subject to an LDAP search or DN is necessary to find the user's metadata in LDAP on the server side).  So please have EXTERNAL work by default as standardized in 1997 at least for TLS client certificates.

While I've heard of proposals to standardize more specific variants of the SASL "EXTERNAL" mechanism, none are standardized yet to my knowledge so none are implemented yet.  I don't object to these proposals as the one for TLS client certs will simply be a synonym for "EXTERNAL" and I'll ignore the others in our server implementation, but I view them as more about architectural purity than utility.  Why wait when we have something that works now?
> While I've heard of proposals to standardize more specific variants of the
> SASL "EXTERNAL" mechanism

Do you have references to that? Because we were talking about doing exactly that ourselves.

> I'll ignore the others [more specific variants] in our server implementation

This is why I don't want us to support EXTERNAL.

> I view them as more about architectural purity than utility

The problems for the client are concrete, as you can see from our UI wording and client cert selection dialog that we need.
bienvenu: As for "working by default" that Chris Newman said, you may want to add it to guessConfig.js, too (in the same swoop when you add it to autoconfig XML).
The internet draft proposing variants of the EXTERNAL mechanism is here:

http://tools.ietf.org/html/draft-josefsson-sasl-external-channel

I don't see how this provides any significant benefit to the client as the client has to present the client certificate dialog and use it for authentication before the server can usefully advertise the mechanism.  IMHO, any server that asks for a client certificate from clients during SSL/TLS negotiation and won't accept valid client certificate authentication is misconfigured.  Servers should never ask for client certificates if they won't work (our server won't ask unless both the name mapping is configured and a trusted CA cert is installed).

Thunderbird will work much better with deployed standards-compliant IMAP/POP servers with client certificates if it implements the 1997 EXTERNAL standard rather than waiting for some future un-deployed variant that provides no additional client utility beyond a clearer name.  The benefit the new EXTERNAL-* family provides is to allow the client to specify which credentials to use in the event there are multiple valid credentials (e.g. IPsec credentials or VPN credentials or something like that in addition to TLS).  Multi-protocol identify federation is just not mature enough for that to be a problem yet.

The product I work on, presently called Oracle Communications Messaging Exchange Server (previously iPlanet and SunONE Messaging Server), has advertised AUTH=EXTERNAL when a usable client certificate is provided during STARTTLS for many years.  If EXTERNAL-TLS is approved by the IESG for standards track publication, I will recommend changing the product to add support for that mechanism in addition to EXTERNAL but only for future releases.  Thunderbird's support or lack of support for EXTERNAL will have no impact on that decision.
Bryan, what we need your input on is the UI for this authentication choice.

Should we use TLS Certificate as the menu item, or External?

and

Should we only show it as a choice if that's the current choice, i.e., setup by autoconfig, or using the config editor? Or should we allow the user to pick the choice?
> If EXTERNAL-TLS is approved by the IESG for standards track publication
> I will recommend changing the product to add support for that mechanism

Thanks.
Hopefully earlier, too, because to be a standard, there need to be 2 working implementations.

> I don't see how this provides any significant benefit to the client

Chris, the problem is the ambiguity. EXTERNAL could be anything of: TLS client cert, IP address, SMTP-after-POP3, any other method. The whole idea of EXTERNAL is that it's defined in an ambiguous way, that it doesn't define anything actually, and that's a fundamental problem, because it removes it from our control (which means we can't know why it fails etc.) and thus also from the end user's control.

> any server that asks for a client certificate from clients during SSL/TLS
> negotiation

Note that TLS client cert can theoretically happen later as well, via TLS renegotiation.
Does the spec nail down that the user must *already* be authenicated at the point when the server sends this EXTERNAL capability? It sounds like it, but it could also be so interpreted that the server could do a TLS renego to get a client cert *after* the client chose AUTH EXTERNAL.
This is very important to have nailed down.

BTW: Did anybody test this patch what happens when we open a secondary IMAP connection to the same server+account (we open up to 5 in parallel, by default, IIRC)? Does Thunderbird give the client cert selection dialog every time or does it remember the selection?
I have not gotten around to testing David's latest patch (I'm hoping to before the week is out... but the pile on my desk is getting deeper), but with the patch that I attached 2010-05-28 I am only prompted to select a cert once.
David: I recommend "TLS certificate" or "Client Certificate" as that's what's used to actually authenticate.

Ben: To become an IETF proposed standard, an internet draft has to have IETF rough consensus and be reviewed as technically complete and competent by the IESG -- there's no implementation requirement although implementation experience can be helpful.  Prior to proposed standard status (IESG approval) an internet draft is subject to change and shipping products is a great way to create interop problems or experimental variants that confuse the market.  An IETF draft standard, a higher tier of standard, has the requirement for two independent interoperable implementations.  The typical sequence is "prototype -> proposed standard -> implement -> interop -> draft standard".

The ambiguity is a matter of technical purity at the moment. There is no server implementation I'm aware of that requests a client cert during STARTTLS, then subsequently advertises EXTERNAL with any meaning other than "use the client cert".  Further, RFC 2595 (standardized in 1999) makes the expected IMAP/POP client behavior for TLS client certificate authentication clear.

As for TLS renegotiation, any sane mail server and client implementation should now have that (mis)feature of TLS shut off by default to improve security.  So that's a non-issue.
(In reply to comment #43)
> David: I recommend "TLS certificate" or "Client Certificate" as that's what's
> used to actually authenticate.

If you want to be pedantic (which I have been accused of in the past :-) something like SASL-External-TLS Client Certificate would accurately describe the the authentication method. Pick your audience, then pick your description; the rest of us will adjust.

> Ben: [...]
> 
> The ambiguity is a matter of technical purity at the moment. There is no server
> implementation I'm aware of that requests a client cert during STARTTLS, then
> subsequently advertises EXTERNAL with any meaning other than "use the client
> cert".  Further, RFC 2595 (standardized in 1999) makes the expected IMAP/POP
> client behavior for TLS client certificate authentication clear.

Speaking as someone who neither implements (i.e. writes/supports the code for) the server side nor the client side, but must make the two talk to each other; it is my opinion that both the client and the server should implement any/all standards _in entirety_ and let the site administrators chose what best fits their requirements. If that was done now, the best case scenario would be that Auth EXTERNAL with TLS client certificates would work "out of the box", the worst case would be that the client presents a valid certificate and the server responds with authentication failed because the IP address was incorrect and the client doesn't know why... but honestly that isn't any worse than the situation we are in now.
> The ambiguity is a matter of technical purity at the moment. There is no server

Chris, is there any server which uses EXTERNAL for anything else than TLS client certs (in any situation)?

Is there any server which advertizes EXTERNAL without having gotten a TLS client cert yet and does TLS renego to get a client cert *after* the client issued EXTERNAL?
Sorry, you answered the last question as "I think no". Note that renego is not entirely out, as RFC 5746 gives a secure way to do renego.
Furthermore, RFC 2595 answers my second question entirely: "The SASL [SASL] EXTERNAL mechanism MAY be used to authenticate once TLS client credentials are successfully exchanged". So, indeed the user must *already* be authenticated via TLS client certs when the server sends EXTERNAL as option.
So, I think the only question for clarkbw is whether TLS Certificate should show up as a choosable auth method (unless BenB removes his objection to having that show up - I would maintain that NTLM is more obscure than TLS Certificate, so the bar is low)
bienvenu, I'm not really strongly "objecting".
I would like the text to be changed to "TLS Client Certificate", though (TLS always has server certs, but this is all about the *client* having one, too).
(In reply to comment #49)
> bienvenu, I'm not really strongly "objecting".
> I would like the text to be changed to "TLS Client Certificate", though (TLS
> always has server certs, but this is all about the *client* having one, too).

The client doesn't send server certs to the server :-) But I'll definitely defer to Bryan.
I can confirm that Trunk + attachment 480669 [details] [diff] [review] works with Cyrus imapd 2.3.16

imap: starttls: TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits new) authenticated as Stacy Millions
imap: login: Bonsai.Millions.Ca [192.168.64.4] stacy EXTERNAL+TLS User logged in
Comment on attachment 480669 [details] [diff] [review]
proposed fix against trunk

The text labels look ok.  I'm a bit concerned about adding another vague auth mechanism, Ben and I worked a lot on the current list to keep it sane.  I'll plus the text but keep watching the comments for any further suggestions.
Attachment #480669 - Flags: ui-review?(clarkbw) → ui-review+
I'm going to land Stacy's patch in the next day or two, unless someone has a strong objection, so we can get some 3.3a1 testing with this.
(In reply to comment #53)
> I'm going to land Stacy's patch in the next day or two, unless someone has a
> strong objection, so we can get some 3.3a1 testing with this.

No objection but this missed 3.3a1, could you land it now so we have it in the next beta/alpha?
fix checked in, thx, Stacy.
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
bug 613962 filed for adding autoconfig support for this option.
Blocks: 613962
posted follow up comment in http://gsfn.us/t/t4f9
Whiteboard: [GS] http://gsfn.us/t/t4f9 → [GS][gssolved]
Target Milestone: --- → Thunderbird 3.3a2
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: