1492498 - Make certificate exceptions on the new cert error pages permanent by default

Status: RESOLVED FIXED

(Firefox :: Security, enhancement, P1)

Description

[Christian Holler (:decoder)]

(Filing this bug as a follow-up to a Slack discussion.)


Previously, when we hit a self-signed certificate, we allowed the user to add a permanent exception by clicking through several dialogs. We recently changed that to be a temporary two-click exception which cannot be easily made permanent (other than by adding it manually in preferences.

Getting the behavior right here is hard because there are many different scenarios in which we could encounter this, some being bad/discouraged (e.g. internet hosts, MitM attacker on public wireless) and others being ok (e.g. embedded hosts on the local home network).

We cannot easily distinguish between these cases (not on a network level, and even if we could on a network level, it might still be a public LAN with an attacker). 

What we could do however is to check, if we visited this site/host before. If this our "first contact" with the host, it might be worth presenting the user with an easy way to add a permanent exception (and a permanent exception here would be more secure than a temporary exception). If however we have visited this host before and we successfully connected without an exception (regular cert) and we now see a self-signed certificate, that seems like an entirely different situation that makes a MitM very likely and we could make the override harder.

We would still have to figure out technical details on how to confirm that we successfully visited the site before and that it didn't come with a self-signed cert previously.

Comment 1

[Johann Hofmann [:johannh]]

Sorry for the delay on this.

I don't think we can distinctly identify sites that were previously visited with a broken cert. I'm not sure the use case here warrants adding a field for that in places or so. Still, regular visited-ness may suffice.

I'm mainly unsure about two questions:

- How are we communicating to the user that in this situation we actually add a permanent exception for the site, vs. a temporary exception in most other cases? I kind of dislike the idea of doing this implicitly.

- What is the best way to inform the user that they are about to enter (or have entered) a site that could be totally legitimate but requires their absolute trust and attention.

Bram and Meridel, do you have any ideas? We can also chat about it if you want to know more...

Comment 2

[Meridel [:meridel]]

I set time with Bram to discuss UX approach. Will report back.

Comment 3

[Bram Pitoyo [:bram]]

Meridel and I have met and discussed approaches to potentially solve this problem.

She will set up some time to meet with Johann to discuss these approaches. After a reasonable solution has been reached, we’ll work on the design and content together.

Comment 4

[Meridel [:meridel]]

Johann, please see our notes here and comment on the ideas and final proposed approach. https://docs.google.com/document/d/1bRgQ6posV3srQDWTHKr9f_SsVlNQJ9K3tXaaQu6Rhr0/edit

Happy to meet and discuss, as well.

Comment 5

[Daniel Veditz [:dveditz]]

(In reply to Christian Holler (:decoder) from comment #0)

Previously, when we hit a self-signed certificate,

There are several bypassable error conditions; I assume the same UX applies to all of them? The attack scenarios may not.

Closely related to "self-signed" is "unknown issuer". For legit cases "self-signed" is likely to be some appliance/IoT device or (less likely these days) an enthusiast's self-run site. A legit "unknown issuer" is likely to be from a enterprise, anti-virus, or government root that didn't get installed correctly. A malicious MITM attempt could give either error.

"Expired" certs are another overridable error. Those are nearly always legit because we check that the cert is legitimate in all other ways before showing that one. It's potentially a MITM attack if someone was able to steal a server's private key in the past, but the MITM would be limited to an attack on that one server. Even if the private key was known to have been stolen revocation likely doesn't help because CA's only support CRL/OCSP while the certificate is valid (unless the incident was high profile enough that browser vendors include a one-off "known bad" entry for it).

Comment 6

[Johann Hofmann [:johannh]]

I'll try to summarize all the discussions we had on this topic and draw up a plan to move forward in 66:

• We believe that temporary exceptions offer no significant advantage to user safety over permanent exceptions.

  • In a legitimate attack case, damage is most likely already done during first usage, and the user is likely to repeat their decision on the next session anyway.
  • It is possible for the user to revoke the permanent exception in the identity popup.

• There are a number of use cases which are made significantly easier by having permanent exceptions by default.

• The "first contact" idea is interesting, but it complicates things significantly both for implementation and UX, by creating a completely different user experience for self-signed certificates and having a relatively intransparent and complex mechanism by which certs are made permanent or temporary, when viewed from the outside without knowledge of our code.

Hence we want to make all certificate exceptions permanent by default, with a pref to control this behavior.

We still believe, however, that there is value in visibly reminding the user that the website they are using is not secured by traditional means. What Chrome is doing is adding a strong red tint to their identity indicators, along with the "Not Secure" label. We might do something similar, and would like to experiment with showing a notification box as shown here:

https://mozilla.invisionapp.com/share/57NZ8GWJCEZ#/screens/337738678

Which notifies the user and gives them a direct choice (a second chance, so to say) to leave the website again. What I like about this is that it takes some pressure away from the initial override decision by turning the page view into a sort of "trial mode", where the user can see the allegedly insecure page and then make their decision.

You can find a diagram of the user flow here: https://mozilla.invisionapp.com/share/57NZ8GWJCEZ#/screens/337738632

However, there have been concerns that this may be too annoying on first use and that it might be more useful to show on second use. I feel like this is something that Nightly is a good experimenting ground for. We could also consider adding an option to never show this kind of warning again, similar to the popup blocker.

I will file new bugs for the user experience changes and use this bug to add a pref for the permanent exceptions.

Thank you for the patience, please let me know what you think!

Comment 7

[Johann Hofmann [:johannh]]

I also should have mentioned that permanent exceptions actually offer a slight security benefit over temporary exceptions by locking the user into a single certificate override (for only one certificate) instead of potentially accepting a different certificate (which could be from a MitM) on every use.

Comment 8

[Johann Hofmann [:johannh]]

Attachment: Bug 1492498 - Make certificate exceptions on the new cert error pages permanent by default. r=nhnt11,keeler

This includes a new test for the feature and a bit of test cleanup to factor
out all exception related tests into their own test file.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6aefbed9ce43
Make certificate exceptions on the new cert error pages permanent by default. r=nhnt11,keeler

Comment 10

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/6aefbed9ce43

Comment 11

[Johann Hofmann [:johannh]]

Not shipping in 65 :)

Comment 12

[Meridel [:meridel]]

Johann: Can you tell me the current status of this? Are we running a shield study now?

I am considering doing a usability test of the new design but need to understand timing...and if this study would be useful to us.

Comment 13

[Johann Hofmann [:johannh]]

(In reply to Meridel from comment #12)

Johann: Can you tell me the current status of this? Are we running a shield study now?

I am considering doing a usability test of the new design but need to understand timing...and if this study would be useful to us.

I think the plan is to add the warnings mentioned in comment 6 in Nightly and get feedback there. If you have thoughts on a study or requirements in terms of when this needs to be implemented I'm happy to chat about it. :)

Comment 14

[Johann Hofmann [:johannh]]

To be clear, we are shipping exceptions permanent by default either way, this is just about potentially adding additional warnings.

Comment 15

[Meridel [:meridel]]

Thanks, Johann. What's the criteria for deciding whether or not to move forward with the new design/additional warnings? That it does not impact usage and retention?

Comment 16

[Johann Hofmann [:johannh]]

(In reply to Meridel from comment #15)

Thanks, Johann. What's the criteria for deciding whether or not to move forward with the new design/additional warnings? That it does not impact usage and retention?

I remember we said that heavy user interaction (not just people ignoring it) might be a success indicator, in addition to the above mentioned general release criteria, plus overall Nightly population feedback. But we'll refine this.

We should probably discuss this in a separate bug that I'll file right now...

Comment 17

[Johann Hofmann [:johannh]]

Filed bug 1531042.

Comment 18

[Gerhard]

Attachment: proposal for fast selection of temporary or permanent exception

I understand, that you don't want to click that much, but the default option should never be permanent.

My proposal would be to add a third button with the suffixes temporary and permanent, as seen in the attachment.

Comment 19

[Johann Hofmann [:johannh]]

(In reply to Gerhard from comment #18)

Created attachment 9053892 [details]
proposal for fast selection of temporary or permanent exception

I understand, that you don't want to click that much, but the default option should never be permanent.

What makes you say that it should not?

Comment 20

[Gerhard]

You mean why it should not be permanent?

Because the permanent security exception is always the worst solution.

Examples:

1. Public website has a certificate error because the certificate expired.
   Definitively no reason to give a permanent exception, because on the next day it could already work again.

2. A web server in your environment has a self-signed certificate.
   Here you can choose to select permanent.

3. Any unknown site you want to visit but get an certificate warning.
   Seriously only have the options to not visit or give a permanent exception??!!

Why does it hurt to add a third button for temporary. This was the default option in the old design anyway.

You wanted to improve the interface and removing a more secure option which could be implemented easily?

The benefit of a button instead of a checkbox below is, that the user has to actively decide what to do.
So the solution with the third button would be even eliminate the old design flaw of not seeing the checkbox for permanent.

Last point, do you think it is better to go to settings and remove the exception because I wanted it temporary instead of permanent. Compare this effort to the initial effort of this request.

No one, will go to the settings after visiting the site to remove the permanent exception, because he wanted it to be temporary.

Comment 21

[Christian Holler (:decoder)]

(In reply to Gerhard from comment #20)

Because the permanent security exception is always the worst solution.

This is certainly not true. With a permanent exception, you get trust on first use, which can be attacked on the first visit but not at subsequent ones. A temporary exception is vulnerable to MitM on every use, which is why I would even argue that temporary security exceptions are the worst, if you are going to accept it anyway at least once.

Examples:

1. Public website has a certificate error because the certificate expired.
   Definitively no reason to give a permanent exception, because on the next day it could already work again.

I think this bug is not about expired certificates, this is specifically about the "new cert" warning, which is issued for self-signed / unknown certs from what I know.

Edit: Fwiw, not saying that there shouldn't be a checkbox to make it temporary or any kind of UI to do that, but it should not be the default.

Comment 22

[Johann Hofmann [:johannh]]

An important thing to note here (which may not be obvious to you) is that we pin exceptions to the exact certificate and the error type.

This is important in the context of what :decoder is saying, when you accept the certificate permanently there is no risk of being presented a forged certificate on the next try.

Another positive side effect of saving permanent exceptions by default is that users will be exposed to less warnings and as such will be more inclined to read and consider the warning instead of clicking through it.

So in case of example #1 that actually sounds like what we have now is most user friendly. Have an exception as long as the site is broken and after that no exception anymore.

Finally, you can also revoke permanent certificates in the identity panel (the little (i) icon on the left hand side of the URL bar)

Comment 23

[Gerhard]

I see, so we have now different topics here:

1. Usability
   1.1 Number of clicks on the old approach
   I agree, that the old approach needed more clicks than the new approach, that's why I like the new approach. But the new approach has less options.

1.2 How temporary/permanent was handled
As explained in the security topic 2 beneath, security is not increased this way. This way you do not even need temporary anymore, because it is almost impossible to use.

1.3 Loss of functionality
Now I have to click several times if I want the exception NOT to be permanent.
I have to actively think about it at the end of my visit to click the lock symbol, then the arrow and then remove exception.
Alternatively I have to visit the options, ..., actively remove the checkbox for store permanently and then confirm the temporary exception. By far worse then the old approach.
So do I need to open a ticket that I want the same like this ticket only for temporary?

2. Security
   2.1 Argumentation about security
   Most of the users don't click the 'view certificate' button and even less people know what information in the certificate is relevant.
   If you would really be interested in increasing the security, I would recommend to add additional information in addition to the generic error message (e.g. SSL_ERROR_xxx) like this:
   A small table like comparison which resides in the windows, which opens if you click 'Advanced...' on the error page.

Current domain name versus cert domain name
Cert trusted root authority versus rating if secure/trusted or unknown
expiration date versus current date

So that on the first look you can see e.g.:
Domain |visited site: chat.domain.com |certificate: chat.domain.com (e.g. in green)
Root authority | Let's encrypt | trusted by cert store or enterprise root (e.g. in green)
Expiration date | 26.03.2019 | today 27.03.2019 (e.g. in red)

The domain name is the same as I visited, the root Authority of this Cert is trusted and the Date is outdated by 1 day.
Then you know directly that this site should be safe.

Other case:
The domain name does not match, date is ok and the root authority is not trusted, this is suspicious.

Or:
Domain name match, date is ok, but root authority is unknown/not trusted. This would be an example for a self-signed certificate.

With this overview you can make a decision more easily. You could even give a recommendation depending on those fields.

2.2 expected security change
You said only allowing it one time as permanent would decrease the chance for 'man in the middle'.
I totally disagree, because as explained earlier, most of the people don't know how to verify certificates.
So those persons give permanent access to the 'man in the middle' attack.
Several clicking for temporary access would make the thing either more suspicious or they would get annoyed and accept it permanent. Which was your favorite option.

The fact that Firefox not only saves the certificate, but also the generic error to it, won't help in this case.
Because either you do not have man in the middle and a valid certificate ( no warning) or you have man in the middle and you accept it permanent ( future visits show no warning).
And again a small yellow sign is ignored by most of the people and of course don't bother if the click the accept and continue ( with not information about permanent)...

2.3 Have an exception as long as the site is broken and after that no exception anymore.
I do not understand this sentence. How is the exception being removed? It will stay in the certificate manger forever (= meaning of permanent)?
For the expired certificate example. When the certificate is renewed, then there is no warning yes, but the exception stays, it won't be removed.

2.4 less warning = more secure?
This argument cuts both ways. If the warning is more useful and has options to select you could achieve both.
With the comparison and display of the certificate information ( directly if you click advanced and the options for temporary and permanent, I would say you get both. I more secure approach to handle those exceptions and less warnings. With an even easier to use interface ( less clicks). This would be a win-win situation.

2.5 when you accept the certificate permanently there is no risk of being presented a forged certificate on the next try.
I said add an option for temporary, you still can click permanent. As we were able to do so in the old approach ( but there it needed 1 more click)

Comment 24

[Zitronella]

I agree with Gerhad https://bugzilla.mozilla.org/show_bug.cgi?id=1492498#c23 in all respects. Especially point 2.3
Two examples as experience:
1.
Just yesterday I surfed a page
h**ps://136.243.67.167
and made an exception, because I just wanted to look at the page. Behind it was just a "website not found"
At first I did not know how to undo this exception.
2.
Another website had not renewed their certificate for short time. I just wanted to look at this site, so I clicked on the button "Accept Risk and Continue". After 5min I surfed this page again and the Lets Encrypt certificate was renewed. Unfortunately, this page is in the Settings -> Certificates now permanently in it, although it is no longer necessary.
I can delete the exception there cumbersome. But maybe after a long time I will not remember what exceptions were important and which not.
I would like to have the choice whether I trust a website temporarily or completely. This is not possible anymore.

I know from some people (before Firefox 66) that they would prefer that the earlier point of "Save this exception permanently" should not even be activated by default..

Comment 25

[Johann Hofmann [:johannh]]

I understand your concerns, but, as I unsuccessfully tried to convey in comment 22, most of them are based on the false assumption that you are adding an exception for the site, where in reality you are primarily adding an exception for the certificate, which is a big difference in practice. This means that when you encounter an expired certificate and add a permanent exception for it, that exception will not apply to e.g. a self-signed certificate put in place by an attacker a few months later on the same site.

The fact that this is so hard to understand is another reason why I don't believe in giving regular users a choice in this matter.

As an advanced user you can just flip the security.certerrors.permanentOverride pref to false, though I'd like to note that I see no benefit in doing this for either of you.

I also don't think there's any benefit in further discussing the matter in this bug.

Comment 26

[Laurențiu Nicola]

:johannh say I'm on a public network that does HTTPS MITM (they do exist). I need to look something up on Google, the error comes up, and I add an exception. Next time I'm in the same place, I won't know that my communication is being intercepted.

Why are you so against giving users the choice? Having them permanent by default would have reduced the number of prompts in a similar way.

And yes, there is the preference, but if I toggle it I get the prompt every time I access badly managed corporate site X.