623785 - Can't connect to sites which require client certificate

Status: VERIFIED DUPLICATE of bug 624075

(Core :: Security: PSM, defect)

Description

[Jürgen Kreileder]

User-Agent:       Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b9pre) Gecko/20110106 Firefox/4.0b9pre
Build Identifier: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b9pre) Gecko/20110106 Firefox/4.0b9pre

With 20110106 I can't connect to sites which require the client to identify with a certificate.  It worked fine in yesterday's build, so maybe it was broken by http://hg.mozilla.org/mozilla-central/rev/257af9cad364 (bug #613977).

The problem shows up both with security.default_personal_cert set to "Select Automatically" and "Ask Every Time".


Reproducible: Always

Steps to Reproduce:
Connect to HTTPS site which requires the client to identify by certificate
Actual Results:  
The connection setup fails after validating the server's cert:
SSL peer was unable to negotiate an acceptable set of security parameters.
(Error code: ssl_error_handshake_failure_alert)

Expected Results:  
1. Automatic selection of client certificate or prompt to select one manually
2. Setup of SSL connection

Both, the server and client certs are signed by a private CA which has been imported to Firefox.

Comment 1

[Matthias Versen [:Matti]]

enough information including regression range -=> marking new
asking for blocking because this is a very recent regression

Comment 2

[Honza Bambas (:mayhemer)]

Just a note: we DON'T have ANY tests for client certificate authentication even we have an infrastructure for such tests.  Big mistake.

Comment 3

[Patrick McManus [:mcmanus]]

All of syn retry was backed out just a minute or two ago as
 http://hg.mozilla.org/mozilla-central/rev/c1b1fde638f9 .. so that should resolev the immediate issue of the description accurately identified the cset in question.

I will make testing this a condition of relanding post ff 4.0 (bug 623948) and dup this to that bug.

Comment 4

[Honza Bambas (:mayhemer)]

On my local IIS setup requiring a client cert bound to a local user the scenario works even with the Patrick's patch.  Probably server specific.

Jurgen, I would be interested in more details as what is the server (apache/iis/what ever) you are using and how is it exactly configured.  We might want to have a regression test for this.  

Thanks.

Comment 5

[Jürgen Kreileder]

It's a pretty standard Apache 2.2 configuration on Ubuntu Maverick (apache 2.2.16-1ubuntu3.1).

If I remember correctly the only two modified options I have are:
SSLMutex default
and
SSLCipherSuite TLSv1:SSLv3:!SSLv2:!aNULL:!eNULL:!NULL:!EXP:!DES:!MEDIUM:!LOW:@STRENGTH

The configuration in the vhost is:

SSLEngine On
SSLVerifyClient require
SSLCACertificateFile /etc/ssl/certs/blackdown.pem
SSLCADNRequestFile /etc/ssl/certs/blackdown.pem
SSLCertificateFile /etc/apache2/ssl/blog.cert
SSLCertificateKeyFile /etc/apache2/ssl/blog.key

Server and clients use 4096-bit RSA keys.

Comment 6

[Jürgen Kreileder]

My first guess was wrong, the problem is actually caused by http://hg.mozilla.org/mozilla-central/rev/d2856d5970b6 (bug 580790 and bug 619487).

Making nsPrefService::CheckAndLogBackgroundThreadUse() return true unconditionally works around the problem for me.

Here are some stack traces I hit when accessing a site which requires client certs.  Apparently there's an SSL background thread which does access prefs in this case (nsPSMBackgroundThread -> nsSSLThread -> ... -> CheckAndLogBackgroundThreadUse)

#0  nsPrefService::CheckAndLogBackgroundThreadUse () at /Users/jk/devel/mozilla/modules/libpref/src/nsPrefService.cpp:983
#1  0x0000000100221f5b in nsPrefBranch::GetIntPref (this=0x105b321b0, aPrefName=0x101ec5e07 "security.OCSP.enabled", _retval=0x12872245c) at /Users/jk/devel/mozilla/modules/libpref/src/nsPrefBranch.cpp:235
#2  0x000000010022a530 in nsPrefService::GetIntPref (this=0x105b268a0, aPrefName=0x101ec5e07 "security.OCSP.enabled", _retval=0x12872245c) at nsPrefService.h:62
#3  0x00000001011a6298 in nsNSSCertificateDB::GetIsOcspOn (this=0x127eb3990, aOcspOn=0x128722784) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsNSSCertificateDB.cpp:1368
#4  0x00000001011b6652 in nsNSSCertificate::hasValidEVOidTag (this=0x11dc51ac0, resultOidTag=@0x128722878, validEV=@0x128722934) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsIdentityChecking.cpp:1018
#5  0x00000001011b6a47 in nsNSSCertificate::getValidEVOidTag (this=0x11dc51ac0, resultOidTag=@0x128722878, validEV=@0x128722934) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsIdentityChecking.cpp:1125
#6  0x00000001011b6c41 in nsNSSCertificate::GetIsExtendedValidation (this=0x11dc51ac0, aIsEV=0x128722934) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsIdentityChecking.cpp:1151
#7  0x00000001011528e9 in AuthCertificateCallback (client_data=0x0, fd=0x107a0fad0, checksig=1, isServer=0) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsNSSCallbacks.cpp:1014
#8  0x00000001057332b4 in ssl3_HandleCertificate (ss=0x107121a00, b=0x107101e99 "", length=0) at ssl3con.c:7904
#9  0x0000000105734f82 in ssl3_HandleHandshakeMessage (ss=0x107121a00, b=0x107101004 "", length=3733) at ssl3con.c:8603
#10 0x00000001057353cb in ssl3_HandleHandshake (ss=0x107121a00, origBuf=0x107121d68) at ssl3con.c:8727
#11 0x000000010573609e in ssl3_HandleRecord (ss=0x107121a00, cText=0x128722ca0, databuf=0x107121d68) at ssl3con.c:9066
#12 0x000000010573728b in ssl3_GatherCompleteHandshake (ss=0x107121a00, flags=0) at ssl3gthr.c:209
#13 0x000000010573a142 in ssl_GatherRecord1stHandshake (ss=0x107121a00) at sslcon.c:1258
#14 0x00000001057462d7 in ssl_Do1stHandshake (ss=0x107121a00) at sslsecur.c:151
#15 0x00000001057487cc in ssl_SecureSend (ss=0x107121a00, buf=0x127a753f0 "GET /munin/blackdown.de/sphere.blackdown.de/index.html HTTP/1.1\r\nHost: blog.blackdown.de\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b9pre) Gecko/20110108 Firefox/4.0b9pre\r\nAccept:"..., len=608, flags=0) at sslsecur.c:1213
#16 0x0000000105748969 in ssl_SecureWrite (ss=0x107121a00, buf=0x127a753f0 "GET /munin/blackdown.de/sphere.blackdown.de/index.html HTTP/1.1\r\nHost: blog.blackdown.de\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b9pre) Gecko/20110108 Firefox/4.0b9pre\r\nAccept:"..., len=608) at sslsecur.c:1258
#17 0x0000000105751162 in ssl_Write (fd=0x107a0fad0, buf=0x127a753f0, len=608) at sslsock.c:1652
#18 0x000000010114da95 in nsSSLThread::Run (this=0x12858a580) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsSSLThread.cpp:1045
#19 0x000000010114ccca in nsPSMBackgroundThread::nsThreadRunner (arg=0x12858a580) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsPSMBackgroundThread.cpp:44
#20 0x00000001055532a7 in _pt_root (arg=0x12858a6c0) at /Users/jk/devel/mozilla/nsprpub/pr/src/pthreads/ptthread.c:187
#21 0x00007fff81200536 in _pthread_start ()
#22 0x00007fff812003e9 in thread_start ()

#0  nsPrefService::CheckAndLogBackgroundThreadUse () at /Users/jk/devel/mozilla/modules/libpref/src/nsPrefService.cpp:983
#1  0x0000000100221a69 in nsPrefBranch::GetCharPref (this=0x105b321b0, aPrefName=0x101ec9748 "security.default_personal_cert", _retval=0x128722330) at /Users/jk/devel/mozilla/modules/libpref/src/nsPrefBranch.cpp:205
#2  0x000000010022a5b8 in nsPrefService::GetCharPref (this=0x105b268a0, aPrefName=0x101ec9748 "security.default_personal_cert", _retval=0x128722330) at nsPrefService.h:62
#3  0x0000000101168200 in nsGetUserCertChoice (certChoice=0x128722990) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsNSSIOLayer.cpp:2833
#4  0x000000010116b7e5 in nsNSS_SSLGetClientAuthData (arg=0x107a30d10, socket=0x107a0fad0, caNames=0x1287229f0, pRetCert=0x107121f68, pRetKey=0x107121f70) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsNSSIOLayer.cpp:2948
#5  0x000000010572e22b in ssl3_HandleCertificateRequest (ss=0x107121a00, b=0x1071010b2 "\016", length=0) at ssl3con.c:5540
#6  0x000000010573501a in ssl3_HandleHandshakeMessage (ss=0x107121a00, b=0x107101004 "\005\003\004\001\002@", length=174) at ssl3con.c:8619
#7  0x00000001057353cb in ssl3_HandleHandshake (ss=0x107121a00, origBuf=0x107121d68) at ssl3con.c:8727
#8  0x000000010573609e in ssl3_HandleRecord (ss=0x107121a00, cText=0x128722ca0, databuf=0x107121d68) at ssl3con.c:9066
#9  0x000000010573728b in ssl3_GatherCompleteHandshake (ss=0x107121a00, flags=0) at ssl3gthr.c:209
#10 0x000000010573a142 in ssl_GatherRecord1stHandshake (ss=0x107121a00) at sslcon.c:1258
#11 0x00000001057462d7 in ssl_Do1stHandshake (ss=0x107121a00) at sslsecur.c:151
#12 0x00000001057487cc in ssl_SecureSend (ss=0x107121a00, buf=0x127a753f0 "GET /munin/blackdown.de/sphere.blackdown.de/index.html HTTP/1.1\r\nHost: blog.blackdown.de\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b9pre) Gecko/20110108 Firefox/4.0b9pre\r\nAccept:"..., len=608, flags=0) at sslsecur.c:1213
#13 0x0000000105748969 in ssl_SecureWrite (ss=0x107121a00, buf=0x127a753f0 "GET /munin/blackdown.de/sphere.blackdown.de/index.html HTTP/1.1\r\nHost: blog.blackdown.de\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b9pre) Gecko/20110108 Firefox/4.0b9pre\r\nAccept:"..., len=608) at sslsecur.c:1258
#14 0x0000000105751162 in ssl_Write (fd=0x107a0fad0, buf=0x127a753f0, len=608) at sslsock.c:1652
#15 0x000000010114da95 in nsSSLThread::Run (this=0x12858a580) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsSSLThread.cpp:1045
#16 0x000000010114ccca in nsPSMBackgroundThread::nsThreadRunner (arg=0x12858a580) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsPSMBackgroundThread.cpp:44
#17 0x00000001055532a7 in _pt_root (arg=0x12858a6c0) at /Users/jk/devel/mozilla/nsprpub/pr/src/pthreads/ptthread.c:187
#18 0x00007fff81200536 in _pthread_start ()
#19 0x00007fff812003e9 in thread_start ()

#0  nsPrefService::CheckAndLogBackgroundThreadUse () at /Users/jk/devel/mozilla/modules/libpref/src/nsPrefService.cpp:983
#1  0x0000000100221f5b in nsPrefBranch::GetIntPref (this=0x105b321b0, aPrefName=0x101ec5e07 "security.OCSP.enabled", _retval=0x12872245c) at /Users/jk/devel/mozilla/modules/libpref/src/nsPrefBranch.cpp:235
#2  0x000000010022a530 in nsPrefService::GetIntPref (this=0x105b268a0, aPrefName=0x101ec5e07 "security.OCSP.enabled", _retval=0x12872245c) at nsPrefService.h:62
#3  0x00000001011a6298 in nsNSSCertificateDB::GetIsOcspOn (this=0x127eb3990, aOcspOn=0x128722784) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsNSSCertificateDB.cpp:1368
#4  0x00000001011b6652 in nsNSSCertificate::hasValidEVOidTag (this=0x12227c620, resultOidTag=@0x128722878, validEV=@0x128722934) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsIdentityChecking.cpp:1018
#5  0x00000001011b6a47 in nsNSSCertificate::getValidEVOidTag (this=0x12227c620, resultOidTag=@0x128722878, validEV=@0x128722934) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsIdentityChecking.cpp:1125
#6  0x00000001011b6c41 in nsNSSCertificate::GetIsExtendedValidation (this=0x12227c620, aIsEV=0x128722934) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsIdentityChecking.cpp:1151
#7  0x00000001011528e9 in AuthCertificateCallback (client_data=0x0, fd=0x107a71160, checksig=1, isServer=0) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsNSSCallbacks.cpp:1014
#8  0x00000001057332b4 in ssl3_HandleCertificate (ss=0x107121a00, b=0x107101e99 "", length=0) at ssl3con.c:7904
#9  0x0000000105734f82 in ssl3_HandleHandshakeMessage (ss=0x107121a00, b=0x107101004 "", length=3733) at ssl3con.c:8603
#10 0x00000001057353cb in ssl3_HandleHandshake (ss=0x107121a00, origBuf=0x107121d68) at ssl3con.c:8727
#11 0x000000010573609e in ssl3_HandleRecord (ss=0x107121a00, cText=0x128722ca0, databuf=0x107121d68) at ssl3con.c:9066
#12 0x000000010573728b in ssl3_GatherCompleteHandshake (ss=0x107121a00, flags=0) at ssl3gthr.c:209
#13 0x000000010573a142 in ssl_GatherRecord1stHandshake (ss=0x107121a00) at sslcon.c:1258
#14 0x00000001057462d7 in ssl_Do1stHandshake (ss=0x107121a00) at sslsecur.c:151
#15 0x00000001057487cc in ssl_SecureSend (ss=0x107121a00, buf=0x127a753f0 "GET /munin/blackdown.de/sphere.blackdown.de/index.html HTTP/1.1\r\nHost: blog.blackdown.de\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b9pre) Gecko/20110108 Firefox/4.0b9pre\r\nAccept:"..., len=608, flags=0) at sslsecur.c:1213
#16 0x0000000105748969 in ssl_SecureWrite (ss=0x107121a00, buf=0x127a753f0 "GET /munin/blackdown.de/sphere.blackdown.de/index.html HTTP/1.1\r\nHost: blog.blackdown.de\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b9pre) Gecko/20110108 Firefox/4.0b9pre\r\nAccept:"..., len=608) at sslsecur.c:1258
#17 0x0000000105751162 in ssl_Write (fd=0x107a71160, buf=0x127a753f0, len=608) at sslsock.c:1652
#18 0x000000010114da95 in nsSSLThread::Run (this=0x12858a580) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsSSLThread.cpp:1045
#19 0x000000010114ccca in nsPSMBackgroundThread::nsThreadRunner (arg=0x12858a580) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsPSMBackgroundThread.cpp:44
#20 0x00000001055532a7 in _pt_root (arg=0x12858a6c0) at /Users/jk/devel/mozilla/nsprpub/pr/src/pthreads/ptthread.c:187
#21 0x00007fff81200536 in _pthread_start ()
#22 0x00007fff812003e9 in thread_start ()

#0  nsPrefService::CheckAndLogBackgroundThreadUse () at /Users/jk/devel/mozilla/modules/libpref/src/nsPrefService.cpp:983
#1  0x0000000100221a69 in nsPrefBranch::GetCharPref (this=0x105b321b0, aPrefName=0x101ec9748 "security.default_personal_cert", _retval=0x128722330) at /Users/jk/devel/mozilla/modules/libpref/src/nsPrefBranch.cpp:205
#2  0x000000010022a5b8 in nsPrefService::GetCharPref (this=0x105b268a0, aPrefName=0x101ec9748 "security.default_personal_cert", _retval=0x128722330) at nsPrefService.h:62
#3  0x0000000101168200 in nsGetUserCertChoice (certChoice=0x128722990) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsNSSIOLayer.cpp:2833
#4  0x000000010116b7e5 in nsNSS_SSLGetClientAuthData (arg=0x107a0e840, socket=0x107a71160, caNames=0x1287229f0, pRetCert=0x107121f68, pRetKey=0x107121f70) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsNSSIOLayer.cpp:2948
#5  0x000000010572e22b in ssl3_HandleCertificateRequest (ss=0x107121a00, b=0x1071010b3 "\016", length=0) at ssl3con.c:5540
#6  0x000000010573501a in ssl3_HandleHandshakeMessage (ss=0x107121a00, b=0x107101004 "\006\003\004\005\006\001\002", length=175) at ssl3con.c:8619
#7  0x00000001057353cb in ssl3_HandleHandshake (ss=0x107121a00, origBuf=0x107121d68) at ssl3con.c:8727
#8  0x000000010573609e in ssl3_HandleRecord (ss=0x107121a00, cText=0x128722ca0, databuf=0x107121d68) at ssl3con.c:9066
#9  0x000000010573728b in ssl3_GatherCompleteHandshake (ss=0x107121a00, flags=0) at ssl3gthr.c:209
#10 0x000000010573a142 in ssl_GatherRecord1stHandshake (ss=0x107121a00) at sslcon.c:1258
#11 0x00000001057462d7 in ssl_Do1stHandshake (ss=0x107121a00) at sslsecur.c:151
#12 0x00000001057487cc in ssl_SecureSend (ss=0x107121a00, buf=0x127a753f0 "GET /munin/blackdown.de/sphere.blackdown.de/index.html HTTP/1.1\r\nHost: blog.blackdown.de\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b9pre) Gecko/20110108 Firefox/4.0b9pre\r\nAccept:"..., len=608, flags=0) at sslsecur.c:1213
#13 0x0000000105748969 in ssl_SecureWrite (ss=0x107121a00, buf=0x127a753f0 "GET /munin/blackdown.de/sphere.blackdown.de/index.html HTTP/1.1\r\nHost: blog.blackdown.de\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b9pre) Gecko/20110108 Firefox/4.0b9pre\r\nAccept:"..., len=608) at sslsecur.c:1258
#14 0x0000000105751162 in ssl_Write (fd=0x107a71160, buf=0x127a753f0, len=608) at sslsock.c:1652
#15 0x000000010114da95 in nsSSLThread::Run (this=0x12858a580) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsSSLThread.cpp:1045
#16 0x000000010114ccca in nsPSMBackgroundThread::nsThreadRunner (arg=0x12858a580) at /Users/jk/devel/mozilla/security/manager/ssl/src/nsPSMBackgroundThread.cpp:44
#17 0x00000001055532a7 in _pt_root (arg=0x12858a6c0) at /Users/jk/devel/mozilla/nsprpub/pr/src/pthreads/ptthread.c:187
#18 0x00007fff81200536 in _pthread_start ()
#19 0x00007fff812003e9 in thread_start ()

Comment 7

[Kaspar]

(In reply to comment #6)
> Here are some stack traces I hit when accessing a site which requires client
> certs.  Apparently there's an SSL background thread which does access prefs in
> this case (nsPSMBackgroundThread -> nsSSLThread -> ... ->
> CheckAndLogBackgroundThreadUse)

Yes. Your second and fourth stack are illustrating the issue I'm describing in bug 624075 comment 5. Either this bug should be duped to 624075, or vice versa (but the Component should definitely be adapted, "Networking: HTTP" isn't quite right).