1267643 - Firefox fails with a handshake error when trying to connect to a site requesting a client certificate signed by a different CA than itself

Status: RESOLVED FIXED

(Core :: Security: PSM, defect, P3)

Description

[Alain Knaff]

We have an SSL server running lighttpd whose server certificate is signed by the etat.lu CA.
It accepts client certificates signed by a different CA (AEV).
Client certificates are mandatory (ssl.verifyclient.enforce = "enable")

Chrome and curl can connect all right in such a setup.

However, Firefox, throws an SSL_ERROR_HANDSHAKE_FAILURE_ALERT (despite having a matching client certificate installed).

If I temporarily exchange the *server* certificate, and install one signed by AEV, than even Firefox works.

Comment 1

[:Cykesiopka]

Thanks for filing the bug.

Currently, this behaviour is expected:
> https://hg.mozilla.org/mozilla-central/annotate/6fc34759465e/security/manager/ssl/nsNSSIOLayer.cpp#l2152
> https://hg.mozilla.org/mozilla-central/annotate/6fc34759465e/security/manager/ssl/nsNSSIOLayer.cpp#l2266

It looks like this behaviour was already present when client auth functionality was added back in 2001:
> https://hg.mozilla.org/experimental/gecko-dev/rev/edb2945ef1c7#l1.569
if ^ doesn't work:
> https://github.com/mozilla/gecko-dev/commit/3a1a52d207f6935f2d15719f6f5a58daefe57330#diff-1f8efd3d29ccd85c52e1dce6a10f889fR1237

I don't know how valuable this filtering is, and if it's useful to most users of client auth.

Unfortunately (and annoyingly), there is no bug number mentioned in the commit message, so I can't find out the original justification for this.

I was also unable to find the appropriate corresponding bug (assuming it was even filed).

Comment 2

[:Cykesiopka]

keeler: Any opinion on whether we should keep or get rid of the filtering?

Comment 3

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Looking at the TLS rfcs, this feature is a SHOULD, but we implemented it incorrectly in the first place. Apparently "If the certificate_authorities list in the certificate request message was non-empty, one of the certificates in the [client] certificate chain SHOULD be issued by one of the listed CAs." (rfc 5246 section 7.4.6). So we shouldn't be filtering by the direct issuer of the client certificate but rather by certificates where there exists a chain to a certificate with one of the specified DNs. This is complicated by the fact that NSS doesn't appear to allow PSM to set the chain directly. Instead, it takes the certificate and key and calls CERT_CertChainFromCert, which appears to build a chain by matching issuer to subject until it finds a self-issued certificate (I don't think it even verifies signatures, although it does look like it checks for basic constraints).

As I maintain that it's not the browser's job to verify the client certificate (and hence we shouldn't even be building these chains), I'm tempted to say we should just skip this filtering altogether. I guess it comes down to what's better for the user - should we remove this (mostly broken) feature at the cost of showing them potentially unrelated certificates? I guess that experience will be as if the server always sends an empty certificate_authorities. Unfortunately, if the user has opted for the browser automatically selecting a client certificate (why is that even possible? It's terrible for privacy), we're more likely to choose the wrong one if the server does send a non-empty certificate_authorities.

Here's maybe what we should do:

* Implement a more persistent "remember I chose this client certificate for this site" feature (i.e. make it work across restarts)
* Remove the "automatically choose a certificate" option
* Remove the filtering based on certificate_authorities

Alain - as a temporary workaround, make sure your server is sending a certificate_authorities list that contains the issuer DN of the client certificates you're expecting (AEV, looks like).

Comment 4

[Kershaw Chang [:kershaw]]

Hi Dana,

I have some questions below. Could you take a look and let me know what to do?

Here's maybe what we should do:

• Implement a more persistent "remember I chose this client certificate for
  this site" feature (i.e. make it work across restarts)

Is this mandatory for bug 1512478?
If yes, I am not sure what is the best way to write/read data to disk. Maybe use DataStorage?

• Remove the "automatically choose a certificate" option

I can't see this checkbox on the client certificate dialog. Maybe this is already removed?

• Remove the filtering based on certificate_authorities

It seems that we haven't had a conclusion yet. Could you confirm that we really want to do this?

Comment 5

[Dana Keeler (she/her) (use needinfo) [:keeler]]

(In reply to Kershaw Chang [:kershaw] from comment #4)

• Implement a more persistent "remember I chose this client certificate for
  this site" feature (i.e. make it work across restarts)

Is this mandatory for bug 1512478?

It would be a great help if you could do at least some of this work. I'm available to review code and if you have any questions.
(Incidentally, this reminded me we also need a way to say "stop using this client certificate" - maybe we could extend the "Clear Cookies and Site Data..." functionality in the site identity drop-down menu?)

If yes, I am not sure what is the best way to write/read data to disk. Maybe use DataStorage?

DataStorage does seem like the right kind of thing to use here.

• Remove the "automatically choose a certificate" option

I can't see this checkbox on the client certificate dialog. Maybe this is already removed?

It's in the "Certificates" section of about:preferences (see also the preference "security.default_personal_cert").

• Remove the filtering based on certificate_authorities

It seems that we haven't had a conclusion yet. Could you confirm that we really want to do this?

Yes, let's do this.

Comment 6

[Kershaw Chang [:kershaw]]

Hi Dana,

Since the changes you proposed need some UI work, which could make this bug more complicated, and I don't have a lot of time working on socket process project, do you think I can just do the "Remove the filtering based on certificate_authorities" part in this bug?
I can file follow-up bugs for other things and work on those bugs after bug 1512478.

What do you think?

Comment 7

[Dana Keeler (she/her) (use needinfo) [:keeler]]

The more I think about it, the more I'm coming to the conclusion that the CA filtering can't possibly work and isn't something we want to do, so yes, just go ahead and remove it and we can address the other issues later.

Comment 8

[Kershaw Chang [:kershaw]]

Attachment: Bug 1267643 - Remove client certificate filtering based on CA names

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9bed62de3d16
Remove client certificate filtering based on CA names r=keeler

Comment 10

[Bogdan Tara[:bogdan_tara | bogdant]]

Backed out changeset 9bed62de3d16 (Bug 1267643) for browser_urlbar_speculative_connect_not_with_client_cert.js failures

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=linux%2Cx64%2Casan%2Cmochitests%2Ctest-linux64-asan%2Fopt-mochitest-browser-chrome-e10s-4%2Cm%28bc4%29&fromchange=1b48b5718737480251c30c2d44752f61057eabe1&tochange=9c2e6c4a65ec4f458e71cfe24b85b82b433654a8&selectedJob=258750644

Backout link: https://hg.mozilla.org/integration/autoland/rev/9c2e6c4a65ec4f458e71cfe24b85b82b433654a8

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=258750644&repo=autoland&lineNumber=4042

[task 2019-07-29T09:38:11.896Z] 09:38:11 INFO - TEST-START | browser/components/urlbar/tests/browser/browser_urlbar_speculative_connect_not_with_client_cert.js
[task 2019-07-29T09:38:12.986Z] 09:38:12 INFO - TEST-INFO | started process screentopng
[task 2019-07-29T09:38:13.572Z] 09:38:13 INFO - TEST-INFO | screentopng: exit 0
[task 2019-07-29T09:38:13.572Z] 09:38:13 INFO - Buffered messages logged at 09:38:11
[task 2019-07-29T09:38:13.573Z] 09:38:13 INFO - Entering test bound setup
[task 2019-07-29T09:38:13.574Z] 09:38:13 INFO - Console message: [JavaScript Error: "uncaught exception: Object" {file: "resource://testing-common/PromiseTestUtils.jsm" line: 112}]
[task 2019-07-29T09:38:13.576Z] 09:38:13 INFO - Buffered messages logged at 09:38:12
[task 2019-07-29T09:38:13.578Z] 09:38:13 INFO - running tls server at https://localhost:40450/
[task 2019-07-29T09:38:13.578Z] 09:38:13 INFO - Leaving test bound setup
[task 2019-07-29T09:38:13.580Z] 09:38:13 INFO - Entering test bound popup_mousedown_no_client_cert_dialog_until_navigate_test
[task 2019-07-29T09:38:13.580Z] 09:38:13 INFO - Searching for 'ocal'
[task 2019-07-29T09:38:13.582Z] 09:38:13 INFO - The url of the second item is https://localhost:40450/
[task 2019-07-29T09:38:13.583Z] 09:38:13 INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_urlbar_speculative_connect_not_with_client_cert.js | The second item has the url we visited. -
[task 2019-07-29T09:38:13.584Z] 09:38:13 INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_urlbar_speculative_connect_not_with_client_cert.js | The second item is selected -
[task 2019-07-29T09:38:13.585Z] 09:38:13 INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_urlbar_speculative_connect_not_with_client_cert.js | A valid string reason is expected -
[task 2019-07-29T09:38:13.588Z] 09:38:13 INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_urlbar_speculative_connect_not_with_client_cert.js | Reason cannot be empty -
[task 2019-07-29T09:38:13.589Z] 09:38:13 INFO - Accepted TLS client connection
[task 2019-07-29T09:38:13.589Z] 09:38:13 INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_urlbar_speculative_connect_not_with_client_cert.js | expecting chooseCertificate to be called -
[task 2019-07-29T09:38:13.590Z] 09:38:13 INFO - Buffered messages finished
[task 2019-07-29T09:38:13.590Z] 09:38:13 INFO - TEST-UNEXPECTED-FAIL | browser/components/urlbar/tests/browser/browser_urlbar_speculative_connect_not_with_client_cert.js | should have only one client certificate available - Got 2, expected 1
[task 2019-07-29T09:38:13.593Z] 09:38:13 INFO - Stack trace:
[task 2019-07-29T09:38:13.593Z] 09:38:13 INFO - chrome://mochikit/content/browser-test.js:test_is:1591
[task 2019-07-29T09:38:13.594Z] 09:38:13 INFO - chrome://mochitests/content/browser/browser/components/urlbar/tests/browser/browser_urlbar_speculative_connect_not_with_client_cert.js:chooseCertificate:43
[task 2019-07-29T09:38:13.594Z] 09:38:13 INFO - TLS handshake done
[task 2019-07-29T09:38:13.594Z] 09:38:13 INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_urlbar_speculative_connect_not_with_client_cert.js | expecting an HTTP/1.1 GET request -
[task 2019-07-29T09:38:13.595Z] 09:38:13 INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_urlbar_speculative_connect_not_with_client_cert.js | chooseCertificate must have been called -
[task 2019-07-29T09:38:13.595Z] 09:38:13 INFO - Leaving test bound popup_mousedown_no_client_cert_dialog_until_navigate_test
[task 2019-07-29T09:38:13.599Z] 09:38:13 INFO - Console message: [JavaScript Error: "The character encoding of the plain text document was not declared. The document will render with garbled text in some browser configurations if the document contains characters from outside the US-ASCII range. The character encoding of the file needs to be declared in the transfer protocol or file needs to use a byte order mark as an encoding signature." {file: "https://localhost:40450/" line: 0}]
[task 2019-07-29T09:38:13.599Z] 09:38:13 INFO - onStopListening
[task 2019-07-29T09:38:13.600Z] 09:38:13 INFO - GECKO(2939) | MEMORY STAT | vsize 20975726MB | residentFast 1689MB
[task 2019-07-29T09:38:13.600Z] 09:38:13 INFO - TEST-OK | browser/components/urlbar/tests/browser/browser_urlbar_speculative_connect_not_with_client_cert.js | took 1640ms

Comment 11

[Kershaw Chang [:kershaw]]

Sorry about this.
I've updated the patch for fixing this.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3eb2d7a85e7b
Remove client certificate filtering based on CA names r=keeler

Comment 13

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/3eb2d7a85e7b