634697 - Remember user's client certificate selection across sessions

Status: RESOLVED FIXED

(Core :: Security: PSM, enhancement, P1)

Description

[Tomáš]

User-Agent:       Mozilla/5.0 (Windows NT 6.1; rv:2.0b11) Gecko/20100101 Firefox/4.0b11
Build Identifier: Mozilla/5.0 (Windows NT 6.1; rv:2.0b11) Gecko/20100101 Firefox/4.0b11

When page requires user certificate for https connection, the dialog for selecting right certificate comes every time after browser restart, although, option at the bottom of dialog to remember my choice was previously selected. (certificate is already successfully imported in browser)

Reproducible: Always

Steps to Reproduce:
0.Goto site configured for https connection requiring user certificate, select right certificate and mark checkbox to remember this choice in the dialog, continue.

1.Restart Firefox
2.Goto site configured for https connection requiring user certificate
Actual Results:  
The dialog for selecting certificate shows up again.


Expected Results:  
Firefox should connect to the site without any inquiries about selecting certificate.

Comment 1

[Tomáš]

Attachment: Screenshot of dialog (czech language)

Comment 2

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

Adg asked to have this bug assigned to them.

Comment 3

[Aman Deep Gautam]

I was not able to reproduce the bug on Firefox/4.0b11 as told. If you check the button corresponding to the "permanently store this exception" firefox successfully remembers->options->advanced->encryption->view certificates->servers.

Comment 4

[u279076]

Tomas, can you try to reproduce this with the latest nightly Firefox 4.0b12pre?  Thanks.

Comment 5

[Tomáš]

I can confirm, that FF 4.0b12 (not nightly) is still behaving as described before.

Also when the dialog for choosing the certificate is canceled it won't come back after attempt for another connection to the server.

----
Firefox 4.0b12 
Mozilla/5.0 (Windows NT 6.1; rv:2.0b12) Gecko/20100101 Firefox/4.0b12

Comment 6

[Tomáš]

(In reply to comment #3)
> I was not able to reproduce the bug on Firefox/4.0b11 as told. If you check the
> button corresponding to the "permanently store this exception" firefox
> successfully remembers->options->advanced->encryption->view
> certificates->servers.

The certificate is client-side so the exception dialog in: options->advanced->encryption->view certificates->servers
is not useful cause there is no server URL to remember the exception for.

The dialog I described is IMHO shown at very rare circumstances as for professional use mostly.. But still used.. 

It's just a glitch with lower importance..

Comment 7

[u279076]

Adding Paul to the cc list of this bug. Paul, is there some interaction with Session Store that could be causing this bug?

Tomas, would it be possible for you to either link or attach a test case we can use to try to reproduce this bug?

Comment 8

[Paul O'Shannessy [:zpao] (not bugmail, email directly)]

(In reply to comment #7)
> Adding Paul to the cc list of this bug. Paul, is there some interaction with
> Session Store that could be causing this bug?

Session Restore doesn't do anything with certs, so highly unlikely.

Comment 9

[mm.bugzilla]

I also have this problem with an Aladdin E-Token.

Comment 10

[Gero]

I had the same 'issue' as described above, but I recognized the flag "Options->Advanced->Encryption->Certificates - When a site asks for a certificate:" (or similar, using German version). I switched from "Ask everytime" to "Select one automatically".
However, I'm not sure what this means exactlly does - in my opinion this misses a third option like "Ask only if new" or something. Maybe "Ask everytime" just ignores already stored site-cert associations?

---
Firefox 7.0.1 (German)
Win 7 x64

Comment 11

[Tim Berners-Lee]

I have been annoyed by this problem for years, mentioned in in talks, discussed it with Anne van K in the TAG, and general taken it as indicating a lack of interest at Mozilla in client-side certs.   (Chrome does not have this problem, and so is easier to use if you use client certs a lot.)  At least from the discussion here it seems to be recognized as  bug -- but has a status of "UNCONFIRMED" surprises me.  So maybe it difficult to reproduce.  If it is supposed to work, where is the site->cert mapping stored?

Comment 12

[Dana Keeler (she/her) (use needinfo) [:keeler]]

For what it's worth, I can reproduce the bug. You're right that client-side certificate-related features aren't a high priority right now, since the majority of our users don't use them. It's unfortunate, but limited engineering resources mean we can't address everything we might want to.

Comment 13

[Ikonta]

Sufficient support of client certificates is one of the reasons, I use FireFox.

(In reply to Tim Berners-Lee from comment #11)
> So maybe it difficult to reproduce.

To my experience reproduction of this issue requires not very typical configuration: it need user, having several the same time valid client certificates in a certification chain. That, as I know, is not typical.
In the rest cases auto-fit client certificate works rather well.

(In reply to David Keeler [:keeler] (use needinfo?) from comment #12)
> For what it's worth, I can reproduce the bug. You're right that client-side
> certificate-related features aren't a high priority right now, since the
> majority of our users don't use them. It's unfortunate, but limited
> engineering resources mean we can't address everything we might want to.

Orienting on popularity NOW you can miss future.
Support of client certificates is not widely spread (yet?), but very _important_ (!) feature.

This feature (client certificates) is lessly used, than it should, because of insufficient support in clients.
And is lessly supported because of rare use.

P.S. Not long ago I've generate a certificate set for localhost to reproduce this issue.
If Mozilla developers need, I can attach these certs to this bug and, if necessary, consult in configuration of Apache web server (although it seems to be trivial).

Comment 15

[Dana Keeler (she/her) (use needinfo) [:keeler]]

This bug could have a more straightforward summary.

Comment 17

[Greg Colyer]

Ref. my reply to related Thunderbird bug 803975 comment #6

In the present context, does the behaviour happen even if the preference "Ask me every time" is not selected?

In any case there is still Tim Berners-Lee's question in comment #11:
> If it is supposed to work, where is the site->cert mapping stored?

Comment 21

[David Balažic]

Starting a few FF versions ago, now (v57.0.4) the "Remember this decision" doesn't work even in the same session (no restarts in between). After a while FF will ask the user to select a certificate for a site for which the user already selected it.

A related question is: what is the difference between enabling or disabling "Remember this decision" ? In a quick test it seems there is no difference. In both cases the choice is remembered (for a while).

(In reply to Tim Berners-Lee from comment #11)
> I have been annoyed by this problem for years
or for decades

> taken it as indicating a lack of interest at Mozilla in client-side certs.
Apparently. The subpar (to put it mildly) support in browsers drives both users and developers to abandon using certificates for client authentication and use other less secure alternatives.
Which the makes browser developers say: But look, nobody uses this stuff anyway...

> So maybe it difficult to reproduce.
Nope, it happens every time, reliably.

Comment 22

[DvD]

I can also confirm that this still doesn't work. Working on a site with a single page application now forces me to select the certificate on each server interaction, even though I selected "Remember this decision".

Comment 23

[Rik]

The problem is that in recent versions the certificate exception get deleted along with other data when "Site preferences" is checked in "Clear history".
That's quite bad.
Even going in privacy & security --> view certificates --> servers --> add exception
the other servers already present there stay, the custom ones get deleted.

In Firefox Developer Edition, having this limitation is very bad because I use to delete everything quite often to make tests, and my LAN certificate has to be confirmed as a new exception day after day! :-( :-( :-( :-(

It was better as it was before.
I haven't found any workaround yet.

Comment 24

[Moritz Birghan [:mbirghan]]

Attachment: Bug 634697 - Remember user's client certificate selection across sessions

Comment 25

[David Balažic]

(In reply to Tomáš from comment #6)

(In reply to comment #3)
The dialog I described is IMHO shown at very rare circumstances as for
professional use mostly.. But still used..

No, it happens to everyone who uses certificates* to do their taxes online, access their bank account etc.

• even with a single installed certificate, the dialog is shown each time

Comment 26

[Moritz Birghan [:mbirghan]]

Attachment: Bug 634697 - Remember user's client certificate selection across sessions

Comment 27

[Matthias Radde]

Found this bugzilla-entry by googleing for a solution for my problem - I have two client-certificates - one already expired and one is valid.
By default the already expired certificate is selected (due to the lower serial-number?) which is annoying because no error-message is displayed if I try to connect to the server with this expired client-cert - just an empty page. Checked with the webdeveloper-tools - there is a message that is at least a hint for the error that just occured ("SSL_ERROR_EXPIRED_CERT_ALERT").
And yes - I checked the "remeber this decision" for the valid client-cert with the higher serial-number. But after a browser-restart the expired certificate is the default-selection again.
Yes - the workaround would be to remove the expired client-cert from Firefox' certificate-store. But a workaround does not eliminate a bug ;-)

If I can help with some testing from user's side (as I am "only" a webdeveloper and webserver-admin, not a Firefox-programmer) I will be happy to do so.

Comment 28

[Moritz Birghan [:mbirghan]]

Attachment: Bug 634697 - Add permanent storage to user's client certificate selection

Depends on D54336

Comment 29

[Moritz Birghan [:mbirghan]]

Attachment: Bug 634697 - Refactoring nsClientAuthRememberService to work as a service

Depends on D54336

Comment 30

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a538b0497336
Refactoring nsClientAuthRememberService to work as a service r=keeler

Comment 31

[Stefan Hindli [:stefan_hindli]]

Backed out changeset a538b0497336 (Bug 634697) for causing android build bustages in /builds/worker/workspace/build/src/security/manager/ssl/nsNSSComponent.cpp CLOSED TREE

https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=c572e0307a58272cc5f04320cd1d9a75a5018704&searchStr=android%2C5.0%2Cx86-64%2Casan%2Cbuild-android-x86_64-asan-fuzzing%2Fopt%2C%28bof%29&selectedJob=290853789

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=290853789&repo=autoland&lineNumber=33084

Comment 32

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4143bf08dfe6
Backed out changeset a538b0497336 for causing android build bustages in /builds/worker/workspace/build/src/security/manager/ssl/nsNSSComponent.cpp CLOSED TREE

Comment 33

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Moritz - when you fix the build issues, let's land this in a new bug so that the original purpose of this bug doesn't get lost.

Comment 34

[Phabricator Automation]

Comment on attachment 9126100 [details]
Bug 634697 - Refactoring nsClientAuthRememberService to work as a service

Revision D62585 was moved to bug 1618710. Setting attachment 9126100 [details] to obsolete.

Comment 35

[Phabricator Automation]

Comment on attachment 9110916 [details]
Bug 634697 - Remember user's client certificate selection across sessions

Revision D54336 was moved to bug 1620976. Setting attachment 9110916 [details] to obsolete.

Comment 36

[:Gijs (out for now; he/him)]

Dana, it looks like this new data storage could effectively be used as a source for (abridged) browser history and should come under our regular data clearing mechanisms, so that "clear recent history...", "forget about this site", the forget button and other mechanisms we have clear the data when the user chooses to do that. Can that happen here, or if not, would you mind filing a follow-up bug to hook this all into nsIClearDataService, if that's not already part of the patches here? That way we don't end up creating a privacy liability when fixing this bug. Thank you!

Comment 37

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Yes, that can happen here. Thanks for the reminder!

Comment 40

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/928233ea49ee
Add permanent storage to user's client certificate selection r=keeler,baku,fluent-reviewers,Gijs

Comment 41

[Dorel Luca [:dluca]]

Backed out changeset 928233ea49ee (bug 634697) for causing leaks. CLOSED TREE

Logs:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=309847107&repo=autoland&lineNumber=1730

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=309847733&repo=autoland&lineNumber=1119

Push with failures:
https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&revision=928233ea49eeca3dddc25d4cdc59995b17f78092

Backout:
https://hg.mozilla.org/integration/autoland/rev/840f993400695f8c67d44b35f5f7f4b2c4bcf5fd

Edit:
Also afected a few other tests, including Mochitest and XPCshell:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=309847069&repo=autoland&lineNumber=1738
https://treeherder.mozilla.org/logviewer.html#?job_id=309847739&repo=autoland

Comment 42

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/441baa36ba93
Add permanent storage to user's client certificate selection r=keeler,baku,fluent-reviewers,Gijs

Comment 43

[Atila Butkovits]

Backed out for failure at test_sss_readstate.js.

Backout link: https://hg.mozilla.org/integration/autoland/rev/f6e962ff9eb45359b95ec15f6471436fe74fef47

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedTaskRun=TpIkOjjxRzqYfYePeJLq5Q.0&resultStatus=testfailed%2Cbusted%2Cexception&revision=441baa36ba93128836dc5564ce8d218bcac51422&searchStr=x2

Failure log for xpcshell: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=311406771&repo=autoland&lineNumber=2903

Failure log for mochitest: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=311406741&repo=autoland&lineNumber=8155

Comment 44

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7142dd253e6d
Add permanent storage to user's client certificate selection r=keeler,baku,fluent-reviewers,Gijs

Comment 45

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/7142dd253e6d

Comment 46

[David Balažic]

I tested the 81 nightly.

Observations:

• ("Remember this decision" is selected) clicking Cancel does not really Cancel, but remembers "user wants to use the web site without a client certificate"
  On next use, FF will not ask for certificate choice, but proceed with no certificate. This is IMO wrong. Cancel should mean "don't do anything".

• ("Remember this decision" is NOT selected) clicking Cancel , just brings up the same dialog again. And again. I guess it is displayed for every HTTP request (for every embedded image, JS, CSS and so on ). This is also bad.

Also, in the second case, after I clicked Cancel 5 times, I selected OK. The page was then displayed, but with many elements missing (CSS, images). After I refreshed it, the main HTML did not see the selected certificate (in the first try it did). This is against my expectations. I would expect the first load (when I selected Cancel) to NOT see my certificate, and the second refresh to see it (because I selected one and had the "remember" option selected too).

Comment 47

[Moritz Birghan [:mbirghan]]

These observations should in my opinion be handled in a different bug. They seem to show issues with the general client cert dialog.

This patch is focused on how remembering decisions are handled and persistently stored after the user interaction with the client cert dialog.

Comment 48

[:Gijs (out for now; he/him)]

Indeed - this patch has landed, and we try and stick to 1 issue per bug, or bugs devolve into never-ending sagas where it's unclear which things got fixed for which releases. Please file a bug for the cancel vs. remember behaviour, and a separate one for the refresh behaviour (with more detailed steps and ideally with a testcase, as the behaviour of "the page" may well depend on JS (that was potentially cached from the "cancel" request before?) and to debug that we'd need to see an actual broken case).

Comment 50

[David Balažic]

cancel vs remember is already submitted: bug 537103 , but marked as a duplicate of another bug that is marked INCOMPLETE.
Should I try to revive it or just file a new one?

Comment 51

[:Gijs (out for now; he/him)]

(In reply to David Balažic from comment #50)

cancel vs remember is already submitted: bug 537103 , but marked as a duplicate of another bug that is marked INCOMPLETE.
Should I try to revive it or just file a new one?

Just filing a new one is probably more straightforward. Thank you!

Comment 52

[David Balažic]

OK, opened bug 1657588

Comment 53

[David Balažic]

and bug 1657591 for the "dialog appears several times"

Comment 57

[WGH]

Is it really fixed? I recently updated to 91 ESR (Linux), I still have the popup appearing as it did before. I.e. I check "remember choice", hit OK, and some time after (when connection is reestablished, apparently), the dialog appears again. The same with browser restarts.

Comment 58

[WGH]

And somehow hitting Esc (which is easy to do accidentally as this freaking dialog appears randomly) when this dialog appears somehow is remembered very well, as I'm not prompted again (and certificate is not sent) until browser is restarted.

Comment 59

[Ikonta]

(In reply to WGH from comment #57)

Is it really fixed? I recently updated to 91 ESR (Linux), I still have the popup appearing as it did before. I.e. I check "remember choice", hit OK, and some time after (when connection is reestablished, apparently), the dialog appears again. The same with browser restarts.

Please note, that pop-up confirmation of authentication with selected certificate differs from remembering selected certificate in list of available.

Comment 60

[WGH]

If that's true, then the fix is almost useless. It does not address the main frustration: the dialog appearing over and over again.

Comment 61

[David Balažic]

I use Firefox v94 and it works fine: I select the certificate for a website and it is remembered "forever". It never asks again (for that website).

The list of remembered certificates choices can be seen in Settings / Certificates / View Certificates , in the tab titled "Authentication Decisions".

Comment 62

[WGH]

Okay, thanks for the input! I have such tab, but its contents are empty. Gotta debug it, and depending on the result, will either post a short comment here, or create a new bug.

Comment 63

[WGH]

I tried to solve this on my own, but I couldn't. I created a new bug 1746532

Comment 64

[WGH]

In the end, this was caused by serial numbers longer than 20 bytes. After I reissued certificate with shorter serial, choice is properly remembered. I also had to manually clean up ClientAuthRememberList.txt to make "Authentication Decisions" tab show anything (having even one long serial breaks it competely, apparently).