add security exception loop: more than one invalid SSL certicifcate per host:port

RESOLVED INVALID

Status

()

Core
Security: PSM
RESOLVED INVALID
2 years ago
2 years ago

People

(Reporter: Dmitry Karasik, Unassigned)

Tracking

47 Branch
All
Unspecified
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Reporter)

Description

2 years ago
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36

Steps to reproduce:

Hello,

For the last week I tried to figure out how to fix the annoying never-ending cycle of showing "Add security exception" dialog. The problem manifests itself when I use a corporate account which uses a self-signed certificate and there's no way to get them to use a correct one. It seems that others hit the same problem, but it has never been fully addressed [1].

What I found is somewhat bizarre. The Outlook IMAP server I use returns not one invalid certificate, but TWO different ones [2] (that's the video only, but recompiling with debug prints confirms two different certificates). Seemingly, at random. Possibly that is because two machines answering same IP address, but I could never find out.

What's relevant here is the way thunderbird (and mozilla, too) store these certificates in cert_override.txt - it is one host:port pair per certificate. What happens when thunderbird opens several SSL requests and tries to validate each certificate, the first certificate is getting flagged as a "security exception", is saved, and then comes another certificate, and overwrites the first one, and the loop continues. It seems that my thunderbird opens more than one SSL connection (one per IMAP folder perhaps?), and this results in the sequence of dialogs.

I actually started to create a patch that would allow to store more than one entry per same host:port, but my C++ is quite rusty, and I've never hacked on mozilla code before. Therefore I'd like to ask for advice before going on - do you think this problem can be solved this way? I can try to hack the patch myself, but I'd rather be sure that there's a consensus about it.

Thank you,
Dmitry

[1] http://www.google.com/search?q=thunderbird+add+security+exception+loop
[2] https://vimeo.com/161968734
(Reporter)

Updated

2 years ago
Hardware: Unspecified → All
(Reporter)

Comment 1

2 years ago
upd: I'm playing with v38 thunderbird because I couldn't find a newer version that builds on my box. However, looking at the trunk:mozilla/security/manager/ssl/nsCertOverrideService.cpp there's nothing that indicates to that the problem was solved
Version: unspecified → 38 Branch

Comment 2

2 years ago
(In reply to Dmitry Karasik from comment #1)
> upd: I'm playing with v38 thunderbird because I couldn't find a newer
> version that builds on my box. However, looking at the
> trunk:mozilla/security/manager/ssl/nsCertOverrideService.cpp there's nothing
> that indicates to that the problem was solved

Good research.

FWIW, next beta and alpha (earlybird) are found at http://www.mozilla.org/en-US/thunderbird/channel/
(Reporter)

Comment 3

2 years ago
Thank you. Just tried Earlybird 47.0a2, same problem.
Version: 38 Branch → 47 Branch
(Reporter)

Comment 4

2 years ago
Created attachment 8739680 [details] [diff] [review]
patch for 45.0b4

Comment 5

2 years ago
Your patch is in the security part of the core Mozilla code, which affects both Firefox and Thunderbird.

So in answering "Therefore I'd like to ask for advice before going on - do you think this problem can be solved this way? I can try to hack the patch myself, but I'd rather be sure that there's a consensus about it." you'll need to get comments from Firefox people. Usually it is best if you can demonstrate that the problem you are trying to fix occurs in Firefox as well as Thunderbird, as that is more likely to get their attention. You'll need to change the Product to Core and the Component to Security:PSM
(Reporter)

Updated

2 years ago
Component: Security → Security: PSM
Product: Thunderbird → Core
(Reporter)

Comment 6

2 years ago
Thank you!

Indeed, the problem occurs in Firefox as well, and here's how it can be demonstrated. I'm adding a primitive https server below that serves two invalid certificates in turns. As expected, the first time Firefox tries to connect, it warns about the invalid certificate, and we're allowed to add it as an exception. The next time it tries to connect, the program serves another bad certificate, and we're also allowed to add it. However on the third connect, when the program server the first certificate, one would have expected that Firefox has already the first bad cert in cert_override.txt, but it has been overwritten already by the second one.

I don't think though that I myself would qualify this as a bug in Firefox, because if a site owner does this to its SSL keys then there are bigger problems with this site anyway :) But for Thunderbird and Outlook IMAP server it is a real problem.

I'm attaching the tar file as full example, and dunplicating the code here:

use strict;
use warnings;
use IO::Socket::SSL;

my $sslid = 1;
sub ssl_server 
{ 
        $sslid = ($sslid == 1 ) ? 2 : 1;
        my $sock = IO::Socket::SSL->new(
                ReuseAddr     => 1,
                Listen        => 1,
                LocalPort     => 443,
                Proto         => 'tcp',
                SSL_cert_file => "$sslid.crt",
                SSL_key_file  => "$sslid.key",
        ) or die "failed to listen: $!";
        return $sock;
}

$SIG{PIPE} = 'IGNORE';
while ( 1 ) {
        my $sock = ssl_server->accept;
        while (<$sock>) {
                s/[\n\r]+//;
                last unless length;
        }
        print $sock "Serving certificate #$sslid\n";
        $sock->close;
}
(Reporter)

Comment 7

2 years ago
Created attachment 8739860 [details]
ssl2.tar

primitive https server with 2 invalid cerificates
(Reporter)

Comment 8

2 years ago
And, for what it worth, here's the version for IMAP, again, doing nothing but emulating two invalid certificates:

use strict;
use warnings;
use IO::Socket::SSL;

my $sslid = 1;
sub ssl_server 
{ 
	$sslid = ($sslid == 1 ) ? 2 : 1;
	my $sock = IO::Socket::SSL->new(
		ReuseAddr     => 1,
		Listen        => 1,
		LocalAddr     => "127.0.0.1",
		LocalPort     => 9931,
		Proto         => 'tcp',
		SSL_cert_file => "$sslid.crt",
		SSL_key_file  => "$sslid.key",
	) or die "failed to listen: $!";
	return $sock;
}

$SIG{PIPE} = 'IGNORE';
while ( 1 ) {
	my $sock = ssl_server->accept;
	$_ = <$sock>;
	m/^(\w+)\s/;
	print $sock "$1 BYE: Serving certificate #$sslid\n";
	$sock->close;
}
I wonder if you could write a custom add-on to do this for you, perhaps using parts of an existing add-on as a model: https://addons.mozilla.org/en-US/firefox/addon/skip-cert-error/ (source code at https://github.com/foudfou/skipCertError )

Something like that would certainly be faster than trying to get a code change to "ride the trains", if the PSM module owners would even accept this change.
What clients connect to this server without errors? Are they really self-signed (issuer == subject) or are they issued from some corporate root cert you could install in Thunderbird?
(Reporter)

Comment 11

2 years ago
I'm answering several questions at once here:

1) I've probably misused term self-signed - what I meant by that is that the subject and the issuer are same organization, - but not same entity. These are certificates issued by a root cert; I've imported both the root cert and the ones transported over SSL, but for some reason Thiderbird ignores these (i.e. allows to be imported, but still show popups asking to add exception to the very same certs). So I don't even know if that a SSL or Thunderbird issue.

2) The clients that connect without errors are native Windows outlook and gnus; that's what my colleagues are using seemingly without this problem.

3) I agree that custom add-on would be a less invasive solution. Theoretically, catching the exception dialog and pushing the necessary buttons sounds feasible. I only tried to hack js to disable the exception dialog altogether just to see how it goes, but it didn't go well - thunderbird started eating 100% cpu, so I didn't dig further.

I'm okay if this change doesn't resonate well with devs. I agree that there are more questions left unanswered.

Comment 12

2 years ago
> These are certificates issued by a root cert; I've imported both the root cert and the ones transported over
> SSL, but for some reason Thiderbird ignores these (i.e. allows to be imported, but still show popups asking to 
> add exception to the very same certs)

You need to also trust the root cert for SSL.

edit->preferences->advanced->certificates->View Certificates->Authorities.

Select the imported root cert.
Click Edit Trust...
check "This certificate can identify websites."
Click OK.

> 2) The clients that connect without errors are native Windows outlook and gnus; that's what my colleagues are 
> using seemingly without this problem.

Most likely the root cert has been imported into the Windows trust store, which outlook and the IE will see.

bob

Comment 13

2 years ago
I think this is bug 528922. Yes, I believe it's basically due to server clustering. Outlook doesn't have the problem since it doesn't normally even use POP/IMAP when communicating with exchange.
(Reporter)

Comment 14

2 years ago
Thank you everyone

I've tried the suggestion by Robert, and yes, it works. But here's the catch: I've imported not the certificates I could view and export from View Certificates->Servers page (i.e those two intermittent ones), but the one I've recovered manually by connecting to the Outlook by openssl (the root one, see below). I currently see no way to recover the root certificate using TB alone.

As an extra detail, here's an output from openssl that might help here:

$ openssl s_client -connect localhost:9931 -showcerts < /dev/null > output
Loading 'screen' into random state - done
depth=1 /DC=net/DC=my.company.name etc stuff
verify error:num=20:unable to get local issuer certificate
verify return:0

$ cat output
Certificate chain
 0 s:/CN=my.company.cn
   i:/DC=net/DC=.../CN=Issuing CA Multipurpose
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
 1 s:/DC=net/.../CN=Issuing CA Multipurpose
   i:/DC=net/.../CN=Root Ca
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=hostname
issuer=/DC=net/.../CN=Issuing CA Multipurpose
---
No client certificate CA names sent
---
SSL handshake has read 3497 bytes and written 431 bytes
---
New, TLSv1/SSLv3, Cipher is RC4-MD5
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : RC4-MD5
    Session-ID: ...
    Session-ID-ctx: 
    Master-Key: ...
    Key-Arg   : None
    Start Time: 1460562653
    Timeout   : 300 (sec)
    Verify return code: 20 (unable to get local issuer certificate)
---
* OK Microsoft Exchange Server 2007 IMAP4 service ready
Unless I'm misunderstanding, Firefox is working as intended here - it's the server and/or the client that's misconfigured.
Status: UNCONFIRMED → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.