511921 - Document workarounds for SSL Wildcards no longer matching multiple levels of domain names (Domain Name mismatch error) after Thunderbird 2.0.0.23 upgrade

Status: RESOLVED WONTFIX

(Thunderbird :: Security, defect)

Description

[Dave Miller [:justdave]]

Thunderbird prior to 2.0.0.23 still contained the bug that allowed * in an SSL cert to match more than one atom of a hostname (which actually violates the spec).

Thunderbird 2.0.0.23 changed the behavior so that a domain name with more than one atom in the spot where the * is in the cert name properly rejects the cert as an invalid hostname.  Dreamhost has mailservers named with a pattern like: a1.postal.mail.dreamhost.com.  Their cert says *.mail.dreamhost.com.

The problem is that TB 2.x has no UI for overriding a certificate mismatch.  If the cert doesn't match, you're screwed.

Comment 1

[Samuel Sidler (old account; do not CC)]

Bugs like this are *exactly* why we wanted more testing and more outreach to the Thunderbird community during this release cycle. Unfortunately, we got either.

I'm not sure there's an easy way to fix this on the branch.

Comment 2

[Reed Loden [:reed]]

Sounds like dreamhost should fix their infra, honestly.

Comment 3

[Daniel Veditz [:dveditz]]

What do outlook express dreamhost customers do? MS doesn't support the old wildcards, do they have a cert override mechanism?

Is there an addon for this?

Comment 4

[David :Bienvenu]

(In reply to comment #3)
> What do outlook express dreamhost customers do? 

They may not use SSL.

Comment 5

[Dave Miller [:justdave]]

(In reply to comment #1)
> Bugs like this are *exactly* why we wanted more testing and more outreach to
> the Thunderbird community during this release cycle. Unfortunately, we got
> [n]either.

I think the problem is that TB 2.0 sucked so bad in general that most of the testing community is on the vastly superior 3.0 betas already (I know I am, personally - both of my parents got nailed by this though).

Comment 6

[Dave Miller [:justdave]]

An extension that added cert override UI to TB 2 would be awesome I think, and avoid having to do something drastic to the app itself.

Comment 7

[Dave Miller [:justdave]]

Hmm, looks like there is one, sort of.

https://addons.mozilla.org/en-US/thunderbird/addon/1964

With that installed, I can override the cert and connect and retrieve mail.  Unfortunately, it puts back the old-style dialog where it only overrides a hostname mismatch for that session.  You have to OK it again every time you restart Thunderbird.  But that's better than having it not work at all.

Comment 8

[David :Bienvenu]

would running tb 3 b3, adding the cert override, and going back to tb 2 work? (Other than marking all your imap folders for offline use, of course).

I don't think an extension could hook into failures to connect to the imap server from our existing network connections with 2.0, but an extension could perhaps look through your accounts, check if any of them used ssl, and try to connect, and put up the cert exception dialog on error, and allow the user to do the cert override. The new autoconfig dialog does something like this. I don't know if it uses any capabilities that aren't in 2.0 to do this, but there's definitely a chance that it does.

Comment 9

[Dave Miller [:justdave]]

doesn't going backwards on a version screw up your profile?

On the other hand, if I bother to install 3.0b3 for them, I'll just leave it.  They'll be better off with that in the long run anyway. ;)  It's just the matter of coping with changes of behavior and so forth.

Comment 10

[Phil Ringnalda (:philor)]

(In reply to comment #4)
> (In reply to comment #3)
> > What do outlook express dreamhost customers do? 
> 
> They may not use SSL.

If I set up Outlook Express to connect to mail.philringnalda.com with SSL, I get a once-per-session dialog saying that "A certificate chain processed, but terminated in a root certificate that is not trusted by the trust provider." and if I click Yes to continue, it merrily uses the *.mail.dreamhost.com cert for mail.philringnalda.com without ever mentioning the domain mismatch to me. There are old instructions on the web for getting rid of that by looking up the actual server IP and putting that in your HOSTS file, but those might predate the a1.balanced.*.dreamhost.com days, and the instructions for finding your a1.etc URL seem to be outdated, so I'm not sure what OE does with that currently, though I presume just the same "untrusted" dialog.

Comment 11

[Nelson Bolyard (seldom reads bugmail)]

setenv NSS_USE_SHEXP_IN_CERT_NAME 1
will restore the cert name matching old behavior,
and the vulnerability that comes with it.
This is a lot less risky for email than for web browsing, I think.

Comment 12

[Dave Miller [:justdave]]

Is that environment variable checked at runtime every time that test is done?  i.e. if an extension is made that just sets that environment variable at application startup, would that be sufficient?  (On some OSes, notably Mac OS X, it's quite difficult to set environment variables before launching an app - not all that hard actually, but the people likely to be able to handle doing it are probably the types that would have upgraded to the betas already :)

Comment 13

[Nelson Bolyard (seldom reads bugmail)]

It is called once, the first time that any cert's host name string is compared
to the URI's host name during an SSL handshake.
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/security/nss/lib/certdb/certdb.c&rev=1.101&mark=1458#1446

Comment 14

[Dan Mosedale (:dmosedale, :dmose)]

Sounds like the trick for an extension would be to cause the variable to get set sufficiently early in the startup process.

Comment 15

[Roland Tanglao needinfo please :rolandtanglao, :mohnkuchen, :adobo, :sinigang, :roland]

filed the bug with DreamHost on their contact form, their forum and Get Satisfaction at:
 http://getsatisfaction.com/dreamhost/topics/bug_in_dreamhostss_ssl_certificates_for_email_exposed_in_thunderbird_2_0_0_23 and 
http://discussion.dreamhost.com/showflat.pl?Cat=&Board=forum_troubleshooting&Number=121635&page=0&view=collapsed&sb=5&o=14&part=

Comment 16

[Grey Hodge (jX)]

This occurred when DH changed the way we're to check out email accounts at mail.yourdomain.com with the full email address. Not just this version.

An easier workaround is this:

1) Go to http://www.kloth.net/services/nslookup.php
2) Enter mail.yourdomain.com, type ANY, and the nameserver should be ns1.dreamhost.com
3) copy one of the hostnames after the 0 in the results into TB as your mail server.

Comment 17

[Phil Ringnalda (:philor)]

(In reply to comment #0)
> Dreamhost has mailservers named with a pattern like:
> a1.postal.mail.dreamhost.com.  Their cert says *.mail.dreamhost.com.

Part of the problem here is that they trumpeted that a1.whatever "solution" and then never really said much of anything about having deprecated it: according to http://wiki.dreamhost.com/Certificate_Domain_Mismatch_Error you should actually be using (assuming postal is actually the server in the Account Status dropdown) postal.mail.dreamhost.com, and once I figured out that when the dropdown said my email server was homiemail-master, what they actually meant was homie.mail.dreamhost.com, a fresh profile on 2.0.0.23 connected fine with just a Website Certified by an Unknown Authority dialog that offered to remember permanently.

Comment 18

[Roland Tanglao needinfo please :rolandtanglao, :mohnkuchen, :adobo, :sinigang, :roland]

I summarized the workarounds in a pro-active Get Satisfaction topic here:
http://getsatisfaction.com/mozilla_messaging/topics/thunderbird_2_0_0_23_breaks_dreamhost_email_over_ssl

Comment 19

[Dave Miller [:justdave]]

(In reply to comment #17)
> (In reply to comment #0)
> > Dreamhost has mailservers named with a pattern like:
> > a1.postal.mail.dreamhost.com.  Their cert says *.mail.dreamhost.com.
> 
> Part of the problem here is that they trumpeted that a1.whatever "solution"
> and then never really said much of anything about having deprecated it:
> according to http://wiki.dreamhost.com/Certificate_Domain_Mismatch_Error you
> should actually be using (assuming postal is actually the server in the
> Account Status dropdown) postal.mail.dreamhost.com

Confirmed on my dad's laptop.  This indeed gets rid of the dialog on every startup. :)

Comment 20

[Nelson Bolyard (seldom reads bugmail)]

IMO, that's really the right solution.  
I'd go so far as to say this bug is invalid.
The solution is to use the correct host name.

Comment 21

[Dave Miller [:justdave]]

Dreamhost isn't the only ISP that's done this though.  But yes, that's definitely the right solution for Dreamhost.

Comment 22

[Dan Mosedale (:dmosedale, :dmose)]

Sounds like we should add this info to the release notes.

Comment 25

[Mark Banner (:standard8)]

I've re-summarised the bug to help users find it when searching bugzilla and it is clear it applies to more than just dreamhost customers.

Comment 26

[Michael Krieger]

It's not only Dreamhost at all.  No surprise that SSL has one certificate per IP address.  We have a cluster of mail servers that accept mail for about 5000 domains, each in the format of mail.yourdomain.com and smtpauth.yourdomain.com.

Until such time that SSL is negotiated after a Host: header, wildcards are a practical item.  This seems to work in Outlook Express and Outlook (wildcards) by  customer's reports.

-M

Comment 27

[Reed Loden [:reed]]

(In reply to comment #26)
> Until such time that SSL is negotiated after a Host: header, wildcards are a
> practical item.  This seems to work in Outlook Express and Outlook (wildcards)
> by  customer's reports.

We've supported SNI for years... As long as the server is doing it correctly, the client (Thunderbird) should handle SNI just fine, afaik.

Comment 28

[Michael Krieger]

Curiously, mail.*.com and mail.*.* also doesn't work, in the same way that just '*' doesn't work.

Comment 29

[Nelson Bolyard (seldom reads bugmail)]

1) This is not a bug.  Thunderbird's use of wildcards in certificates is 
now compliant with the relevant RFCs for use of certificates with IMAP, POP 
and SMTP over TLS, e.g. RFC 2595 and RFC 4954, and NNTP over TLS, RFC 4642.  
It formerly was too lax.  Now it is standards compliant.  The current behavior is actually a bug FIX.  See bug 159483.

2) It's not a regression.  It can't be a regression if it's not a bug.

3) The constraints on wildcards now imposed by Mozilla products are no more
limiting than those imposed by Microsoft.  The same standards-compliant 
wildcards that work in Microsoft email clients also work in Thunderbird.

4) All the email protocols (IMAP, POP3, SMTP) permit TLS handshakes to 
begin well after other aspects of the email protocol have been performed,
using extensions to each of those protocols, known generically as "StartTLS".
Thunderbird implements the StartTLS extensions for each of those protocols.
This should obviate all the issues that SNI fixes for https.  In other words,
the server should have all the info it needs to present the certificate with
the desired host name, even without implementing SNI.  With SNI, the servers 
have no excuses left for not being able to present a certificate with the 
desired host name.

5) In every case, each SMTP/IMAP/POP server has (at least) one host name
that can be correctly configured into the server's certificate, and the 
user's thunderbird account can be configured to connect to that server by
that host name, eliminating all host name mismatches completely.  

Some users don't like the fact that they must configure Thunderbird to use  
the provider's domain name for the server, and complain loudly that they must use the mail service provider's domain name, instead of their own personal domain name. My wife has a name for such a complaint: a Waaaambulance.

Comment 30

[Daniel Veditz [:dveditz]]

(In reply to comment #28)
> Curiously, mail.*.com and mail.*.* also doesn't work, in the same way that
> just '*' doesn't work.

According to the spec the wildcard has to be in the leftmost domain label, and it can't be a global wildcard.

(In reply to comment #29)
> 2) It's not a regression.  It can't be a regression if it's not a bug.

From a user's POV--and the Mozilla project tries to take a user-centric view--"it worked yesterday; today I can't get mail" is the very definition of a regression. Maybe we WONTFIX it if we think it's the right change and that we can address the user problem with better education, but for the user it's still a problem "caused" by upgrading Thunderbird. This is especially bad if the "regression" leads them to downgrade to the insecure previous version and turn off future updates.

I don't think the wildcard issue is the real problem though. Servers shouldn't be using non-standard wildcards that don't work in other clients. The real issue is that users cannot now override the cert failures. This should be affecting other mail servers that use incorrect certs other than wildcard certs (such as self-signed certs).

We now need to backport bug 429843 and probably bug 480026 I think

> 
> 3) The constraints on wildcards now imposed by Mozilla products are no more
> limiting than those imposed by Microsoft.  The same standards-compliant 
> wildcards that work in Microsoft email clients also work in Thunderbird.
> 
> 4) All the email protocols (IMAP, POP3, SMTP) permit TLS handshakes to 
> begin well after other aspects of the email protocol have been performed,
> using extensions to each of those protocols, known generically as "StartTLS".
> Thunderbird implements the StartTLS extensions for each of those protocols.
> This should obviate all the issues that SNI fixes for https.  In other words,
> the server should have all the info it needs to present the certificate with
> the desired host name, even without implementing SNI.  With SNI, the servers 
> have no excuses left for not being able to present a certificate with the 
> desired host name.
> 
> 5) In every case, each SMTP/IMAP/POP server has (at least) one host name
> that can be correctly configured into the server's certificate, and the 
> user's thunderbird account can be configured to connect to that server by
> that host name, eliminating all host name mismatches completely.  
> 
> Some users don't like the fact that they must configure Thunderbird to use  
> the provider's domain name for the server, and complain loudly that they must
> use the mail service provider's domain name, instead of their own personal
> domain name. My wife has a name for such a complaint: a Waaaambulance.

Comment 31

[Nelson Bolyard (seldom reads bugmail)]

Dan, 
> users cannot now override the cert failures.
If that is so, it is due to a PSM change, not to an NSS change.
NSS's present abilities to override cert errors is no more and no less 
than it was 5 years ago.  PSM's is quite different.

Comment 32

[Kai Engert [:KaiE:]]

Dan, Nelson, I'm confused.

Dan, did you say 
  > users cannot now override the cert failures
with Thunderbird 2.0.0.23 ?

Nelson, if this is about Thunderbird 2, I think there have been no changes to PSM either. TB 2 did not receive the new certificate exceptions code, but in my understanding TB 2 still uses those old click-through-dialogs.

If you're saying that the introduction of a new NSS changed TB in a way so that the click-through-dialogs no longer work, I don't see how this failure can be caused by PSM changes. Could it be that NSS now gives a different error code, an error code that is not mapped to one of the old click-through dialogs?

Maybe I just don't see the full picture, clarifications welcome.

Comment 33

[Kai Engert [:KaiE:]]

(In reply to comment #8)
> would running tb 3 b3, adding the cert override, and going back to tb 2 work?

The cert override that was added in FF 3 / TB 3 requires new code at the application level.

Cert overrides (exceptions) added using TB/FF 3's new user interface will not have an effect when returning to FF/TB 2.

Comment 34

[Dave Miller [:justdave]]

(In reply to comment #32)
> Nelson, if this is about Thunderbird 2, I think there have been no changes to
> PSM either. TB 2 did not receive the new certificate exceptions code, but in
> my understanding TB 2 still uses those old click-through-dialogs.

As far as I can tell, Thunderbird 2 does not *have* a certificate override dialog at all, unless you install the extension mentioned in Comment 7.  Thunderbird 1.5 had one, and it disappeared in 2.0.  I remember setting my parents up on Dreamhost in TB 2 when DH still had a self-signed cert, and had to manually import the certificate into the certificate manager because TB wouldn't prompt me to override it, it just wouldn't connect.

What changed in 2.0.0.23 is that the old bug that allowed a * to match multiple levels of a domain name went away, which broke Thunderbird's ability to connect to a server with a cert that was previously accepted without an override.

TB 3 puts the cert override dialog back and improves on it a LOT (the bugs Dan references above that he's suggesting get backported).

Comment 35

[Nelson Bolyard (seldom reads bugmail)]

Thanks for that great explanation, Dave.  Makes good sense. 
So, it's not that a change made TB2 users lose the ability to override
errors.  They never had that.  It just caused some users to experience 
an error they had not experienced before, and so had previously not 
desired to override.

Comment 36

[Kaspar]

(In reply to comment #7)
> Hmm, looks like there is one, sort of.
> 
> https://addons.mozilla.org/en-US/thunderbird/addon/1964
> 
> With that installed, I can override the cert and connect and retrieve mail.

Did you mean

  https://addons.mozilla.org/en-US/thunderbird/addon/2131
  ("Remember Mismatched Domains")

instead? Cert Viewer Plus (the one you mention in comment 7) doesn't have any code dealing with cert exception handling, actually... (I'm pretty sure of that, believe me :-)

Comment 37

[Charles]

Bingo... the 'Remember Mismatched Domains' extension fixed it for us (50+ workstations)... but this was a major pain.

Bug fixes that CREATE ADDITIONAL PROBLEMS should also provide a workaround - the code from the extension allowing a permanent override should have been integrated as well.

Comment 38

[Dan Mosedale (:dmosedale, :dmose)]

Charles, yeah the consequences of this patch were unfortunate.  Sadly, there was a zero-day exploit that made all users vulnerable to a code execution exploit with, so we had no choice in the matter.

At this point, I think the best we're going to be able to do for the 2.x series is to figure out how to document this appropriately.  This probably involves updating the release notes; we might also want to mention something on the start page.  Assign to rebron for further consideration...

Comment 39

[Charles]

I understand the reason, what I don't understand is why code wasn't also included that did the same thing as the 'Remember Mismatched Domains' extension - or heck, just get the extension authors permission to include it in the core TBird code.

Anyway, not complaining really... just venting a little, as, of course, the first person to get bit by this in my office was the boss/owner of the company, so I had to scramble to figure out what happened and find a fix. Thankfully it only took me a few minutes to find the extension, but a bit longer to get the rest of the PCs fixed up...

I'm really looking forward to TBird 3... but I hope I'm able to customize the main UI to my satisfaction like I have 2...

Comment 41

[Daniel Veditz [:dveditz]]

This is not going to "block" us from releasing future Thunderbird 2 security updates, but it sure would be nice to document.

Comment 42

[Wayne Mery (:wsmwk)]

Roland, Jen, can this be closed?

this may have been blocker at some point, but surely it isn't now.