800882 - Error message for cert errors for sites with HSTS is unclear as to why we're not allowing an override

Status: RESOLVED FIXED

(Core :: Security: PSM, defect)

Description

[neil]

User Agent: Mozilla/5.0 (Windows NT 6.1; rv:17.0) Gecko/17.0 Firefox/17.0
Build ID: 20121010150351

Steps to reproduce:

I am on the beta download option and on restarting Firefox I was upgraded from 16.0 to 17.0 (security patch).

I now do not have an option to add exceptions when visiting websites with invalid https certificates.




Actual results:

At work I go through a web scanner software/proxy and when visiting websites the security certificate becomes invalid 'this connection is untrusted', specifically gmail and other email clients.

Normally I get an option to 'get me out of here' and to 'add exception'. In this version 17.0 this option is now removed and I do not have the ability to visit the site (you cannot add name to exception list as it changes every visit). 

All I get now is this:
Technical Details
        
        www.gmail.com uses an invalid security certificate.

The certificate is not trusted because the issuer certificate is not trusted.
The certificate is only valid for mail.google.com

(Error code: sec_error_untrusted_issuer)


Expected results:

It should provide me the means to by bypass the error as I DO KNOW what I'm doing.

As it stands it means I cannot access any hpps website with an invalid certificate and have to resort to using Internet Explorer.

Clearly whatever you did to quickly patch the 16.0 error allowing people to see history does not work.

Comment 1

[Matthias Versen [:Matti]]

The upgrade from Firefox16 to Firefox17 is not due to the security fix.
You are on the beta channel and from Firefox16beta you will be upgraded to 17beta

Please try it again in the firefox safemode: help/restart with addons disabled
Does it work there ?

Comment 2

[neil]

Yes, I have tried that, and cleaning everything. It is simply on your update from 16beta to 17beta the button on the 'this connection is untrusted' page under the 'technical detail' has removed the button to add the site to the exemption list.

"Technical Details
        
        www.gmail.com uses an invalid security certificate.

The certificate is not trusted because the issuer certificate is not trusted.
The certificate is only valid for mail.google.com

(Error code: sec_error_untrusted_issuer)"

As above there is no button to add this exception. Going through the options menu is both time-consuming (relatively given every other browser - including current non-version 17 Firefox allows you to bypass the certificate error) and in my case does not work because my proxy must be mangling the certificate each time.

Comment 3

[Matthias Versen [:Matti]]

Attachment: Screenshot from Firefox17beta/windows7

That is what I see on my system with Firefox17beta

Comment 4

[neil]

Probably because you are looking at a different error. My error is sec_error_untrusted_issuer. I have attached a screenshot, the 'I understand the risk' disappeared for me on the update.

Comment 5

[neil]

Attachment: The error producing no exception button

As detailed in my last comment, this is the error I get showing no option to bypass the error in the 17.0 release.

Comment 6

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

This problem is caused by the fact that gmail.com is considered HSTS by our preload list. One consequence of being HSTS is that certificate error exceptions cannot be created or used.

keeler just landed a patch on -beta that (temporarily) removes most Google properties, and many other sites, from our preloaded HSTS list. So, you will be able to add certificate exceptions again in the next beta update.

The long-term solution here is to have your system administrator (IT people) install the CA certificate used by your web scanner software/proxy into Firefox and mark it as being trusted for SSL. Then you will not have to add certificate exceptions for every site.

Comment 7

[Adam Langley]

Chrome also enforces this and our policy is that MITM certificates must be installed. There is no bypass for HSTS certificate errors.

Comment 8

[neil]

Thanks. Are you saying in future stable releases I won't be able to add exceptions to items on this list? It means anyone behind a proxy (we use some kind of  'webscanner') won't be able to use Firefox. Also, it seems a bit random your HSTS list as hotmail allows me to override but gmail does not.

In my case I'm provided with a 'permanent' ssl certificate but I've no idea how to install it as it's just the byte codes, which I guess is a different matter...

Comment 9

[Matthias Versen [:Matti]]

There should be a hint in the error page why it's not possible to override the error.

Comment 10

[Devdatta Akhawe [:devd]]

Agree with Matthias: Chrome currently adds a note 
"You cannot proceed because the website operator has requested heightened security for this domain." Should we add a further note: "Please contact your network/IT administrator if you see this error too often" or something like that?

This is a trivial fix: should we just patch it as part of this bug?

Comment 11

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

lco, please see comment 9 and 10. I agree that we should add a note saying that the website asked us to prohibit the use of the certificate exception buttons.

(In reply to neil from comment #8)
> Thanks. Are you saying in future stable releases I won't be able to add
> exceptions to items on this list?

Yes.

> It means anyone behind a proxy (we use
> some kind of  'webscanner') won't be able to use Firefox. Also, it seems a
> bit random your HSTS list as hotmail allows me to override but gmail does
> not.

It has to do with whether the website has requested the extra protection or not. Hotmail hasn't requested the extra protection.

> In my case I'm provided with a 'permanent' ssl certificate but I've no idea
> how to install it as it's just the byte codes, which I guess is a different
> matter...

Hopefully the IT people who set up the proxy can help you with this. Firefox actually makes it relatively easy to do if your IT people have set up a (hopefully HTTPS) website with a link to the certificate to download. But, otherwise it is pretty tedious.

Comment 12

[Larissa Co [:lco]]

Just wanted to let you guys know that I'm not disregarding this request. I started working on the text, and I realized that it required more than just simply appending a new message to what we currently have because the main points of this error are  different from the standard untrusted cert error. I'd like to redo the entire message, which takes more time. Otherwise, we risk confusing uninformed users with mixed messages. Sorry!

Comment 13

[Larissa Co [:lco]]

Here's a proposal for the text change: http://cl.ly/0k070y292S1j

Can you let me know if it clearly and accurately explains the issue?

Also, the "Technical Info" section text is a little sloppy because I don't know exactly what technical info to include and how to phrase it. I'd appreciate some help with that.

Comment 14

[Devdatta Akhawe [:devd]]

This looks good. Minor issue with the "What should I do?" section. I suggest removing the "Check you've entered the correct site name" etc. If this error pops up, it is highly probable that there is something wrong with the network or something wrong with the HSTS settings. How about

"Someone on the network is trying to monitor your traffic. If this is on a trusted network, contact the network administrator."

Comment 15

[Matej Novak [:matej]]

(In reply to Devdatta Akhawe [:devd] from comment #14)
> This looks good. Minor issue with the "What should I do?" section. I suggest
> removing the "Check you've entered the correct site name" etc. If this error
> pops up, it is highly probable that there is something wrong with the
> network or something wrong with the HSTS settings. How about
> 
> "Someone on the network is trying to monitor your traffic. If this is on a
> trusted network, contact the network administrator."

I stumble over the first few words in the second sentence. Here are a couple of edit options:

"Someone on the network is trying to monitor your traffic. If this is a trusted network, contact the network administrator."

"Someone on the network is trying to monitor your traffic. If you're on a trusted network, contact the network administrator."

Comment 16

[Dana Keeler (she/her) (use needinfo) [:keeler]]

(In reply to Devdatta Akhawe [:devd] from comment #14)
> "Someone on the network is trying to monitor your traffic.

That's not a claim we can make.

My $.02 for the "What Should I Do?" section would be to add something like "Try to visit another website over https. If you are unable to do so, there may be something wrong with your network."
Unfortunately, there's not much anyone can do other than maybe trying to connect from a different network or contacting the website administrator.

Another issue about HSTS (and this should be in a different bug) is that when the https version of a site is down but the http version is up, users will just see that firefox can't connect while some other browsers can - maybe we should figure out how to detect this and throw up a similar informational page?

Comment 17

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

(In reply to David Keeler from comment #16)
> (In reply to Devdatta Akhawe [:devd] from comment #14)
> > "Someone on the network is trying to monitor your traffic.
> 
> That's not a claim we can make.

I agree. It's both too scary (probably, it is a captive portal that is waiting for you to click "I agree" on its ToS or pay, or anti-virus that is trying to help you) and too not-scary (somebody might be doing something much worse than monitoring your traffic). Also, not sure that users understand "traffic," which seems like slang.

> My $.02 for the "What Should I Do?" section would be to add something like
> "Try to visit another website over https. If you are unable to do so, there
> may be something wrong with your network."
> Unfortunately, there's not much anyone can do other than maybe trying to
> connect from a different network or contacting the website administrator.

That would basically be asking the user to do captive portal detection. Firefox could do that automatically itself, and show different messages depending on the result.

> Another issue about HSTS (and this should be in a different bug) is that
> when the https version of a site is down but the http version is up, users
> will just see that firefox can't connect while some other browsers can -
> maybe we should figure out how to detect this and throw up a similar
> informational page?

First, HSTS sites should be redirecting their HTTP traffic to the HTTPS site, so I doubt that this would happen often. But, also, we can gather telemetry on this, if we really want to know (Got SSL error? Open a channel with an "ignore HSTS" flag and LOAD_ANONYMOUS to the http://hostname, and see if that returns anything other than a 30x with Location: https://hostname and the HSTS header set. Record this boolean flag.)

Comment 18

[Larissa Co [:lco]]

Ok, so are there any practical things that we're willing to endorse that the average user can do if they encounter this error? I want to include text like this so that the user doesn't feel like they have no control over their experience.

Comment 19

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

(In reply to Larissa Co from comment #18)
> Ok, so are there any practical things that we're willing to endorse that the
> average user can do if they encounter this error? I want to include text
> like this so that the user doesn't feel like they have no control over their
> experience.

If the problem is "hostname doesn't match the hostname in the cert" then the user probably typed it wrong in the address bar or an attack. The website has previously promised (through HSTS) that they will not use a certificate with the wrong hostname, so we can exclude that possibility.

If the problem is that the cert is expired, then the problem is probably the computer's time is set wrong, or there's a particularly-bad captive portal, or there's an attack. Again, the website promised through HSTS that it would never use an expired certificate, so we should exclude that possibility.

If the problem is that the cert has an untrusted issuer, then it is probably because of a captive portal or because of some cert-related configuration issue on the computer or because of an attack. The website promised through HSTS that it would never use a certificate from an untrusted issuer, so we can exclude that possibility.

Now, it is possible that the site might make a promise and then break it, or we might change some of the assumptions the site made the promise under. But, generally, we have to assume that the likelihood of this happening is so low, and the benefits of HSTS as a security mechanism are so high, that we do not have to consider these possibilities in the UI.

Comment 21

[Lucas Adamski [:ladamski]]

This seems like a misconfigured local device.  If you are using a proxy with its own certificate, you need to import it into your cert store and indicate it as trusted for website authentication.

Comment 26

[abcdzywx]

Okay, the bug I filed yesterday, bug 1049187, was marked a duplicate of this ticket.  Apparently the example that I found where adding an exception for a Fiddler self-signed certificate works with google.com yet not for support.mozilla.com, is because support.mozilla.com uses HSTS.

Here is what I don't understand.  Isn't HSTS just supposed to be a protocol to inform the browser to use HTTPS and not HTTP connections for that site?  Why does this mean that Firefox should prevent a user from adding a certificate exception for those sites?  Don't the new versions of Firefox already provide enough protection with the "browser.xul.error_pages.expert_bad_cert" setting that has to be manually enabled merely to add a certificate exception in the first place?  Why the need for extra restriction?

If "browser.xul.error_pages.expert_bad_cert" is not considered to be the desired browser property for overriding this additional restriction for HSTS sites, then can we please have an additional Firefox property to allow doing this?  The aforementioned property was made user-configurable because Mozilla recognized that "While benefiting the novice-to-intermediate user in terms of security, it can be a hindrance to advanced users..."  I think the same logic applies here and users should have maximum control to decide to override the restriction.

Comment 27

[Dana Keeler (she/her) (use needinfo) [:keeler]]

The HSTS specification is actually very clear on this point:

http://tools.ietf.org/html/rfc6797#section-8.4
8.4. Errors in Secure Transport Establishment

   When connecting to a Known HSTS Host, the UA MUST terminate the
   connection (see also Section 12 ("User Agent Implementation Advice"))
   if there are any errors, whether "warning" or "fatal" or any other
   error level, with the underlying secure transport.  For example, this
   includes any errors found in certificate validity checking that UAs
   employ, such as via Certificate Revocation Lists (CRLs) [RFC5280], or
   via the Online Certificate Status Protocol (OCSP) [RFC2560], as well
   as via TLS server identity checking [RFC6125].

and:

http://tools.ietf.org/html/rfc6797#section-12.1
12.1. No User Recourse

   Failing secure connection establishment on any warnings or errors
   (per Section 8.4 ("Errors in Secure Transport Establishment")) should
   be done with "no user recourse".  This means that the user should not
   be presented with a dialog giving her the option to proceed.  Rather,
   it should be treated similarly to a server error where there is
   nothing further the user can do with respect to interacting with the
   target web application, other than wait and retry.

   Essentially, "any warnings or errors" means anything that would cause
   the UA implementation to announce to the user that something is not
   entirely correct with the connection establishment.

   Not doing this, i.e., allowing user recourse such as "clicking
   through warning/error dialogs", is a recipe for a man-in-the-middle
   attack.  If a web application issues an HSTS Policy, then it is
   implicitly opting into the "no user recourse" approach, whereby all
   certificate errors or warnings cause a connection termination, with
   no chance to "fool" users into making the wrong decision and
   compromising themselves.

Comment 28

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Attachment: patch

Dao - would you be a good reviewer for this? Thanks.

Comment 29

[abcdzywx]

I see.  Thanks for clarifying this.  I suppose this standard reflects the unfortunate contemporary technological trend of dictating as much as possible to users and denying even advanced users the volition and self-determination that they should have.

In this case, my understanding is that allowing some kind of option to override a failing certificate would be a deviation from the protocol.  But I still think that an open-source application like Firefox should give such a capability to people, because there are definitely legitimate reasons for an end-user to have an override of a certificate error.  Clearly, in my case, I want/need the capability to use the web-debugging proxy Fiddler on any website.  In other cases, people might be testing their own development webservers and need to deal with self-signed certificates.  Oftentimes, applications will allow protocols to be deviated from when it makes sense to do so.  I think we should have some kind of option, even if it gives the end-user a stern warning like "You are deviating from the HSTS protocol by making this exception, are you ABSOLUTELY sure you want to do this?".  I'm not sure if there's any other way out, other than users having to download and hack the Firefox source code to override the certificate errors, which is rather excessive IMO.

Thank you for considering this.

Comment 30

[Frederik Braun [:freddy]]

For developers and testers who wish to intercept SSL traffic with a Proxy (like Fiddler, ZAP, Burp), there *is* a way to bypass these warnings.
If your intercepting proxy allows you to export the fake CA certificate that is used to create the fake certificates, then you can just import them in your browser and mark them as trusted (Preferences, Advanced, Certificates, View Certificates, Import). You won't get any more warnings.

I highly recommend creating a separate profile that contains the proxy settings and trusts the fake certificates. See here for more about this: https://support.mozilla.org/en-US/kb/profile-manager-create-and-remove-firefox-profiles

Comment 31

[abcdzywx]

I see.  I actually did find the option in Fiddler to export the fake certificate-authority certificate, and this does indeed get Firefox to trust Fiddler's certificates for any website.  Perhaps that is a more reliable solution overall than having to trust Fiddler's certificates for each individual website.

But what if someone was using a similar tool, had to trust self-signed certificates, and for some reason did not have access to the associated CA certificate?  Would they have to build their own CA certificate that trusts the given self-signed website certificates?  Is there a straightforward process to do this using, say, OpenSSL?  Could we still have an option in Firefox to bypass that requirement if someone simply wants to use a self-signed certificate on an HSTS website without having to go to the trouble to build a dummy CA certificate?

Comment 32

[Jim Mathies [:jimm]]

Comment on attachment 8468864 [details] [diff] [review]
patch

You could clean this up a bit by assigning the result of getCSSClass() once to a local variable and then check that in each ot the test cases. Other than that this looks ok to me.

Comment 33

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Comment on attachment 8468864 [details] [diff] [review]
patch

Let me know if you would prefer screenshots or a link to a try build or something. Thanks!

Comment 35

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Philipp, is there some way I can help facilitate this ui review? (e.g. before/after screenshots, having a try build for every platform?) This is a long-standing bug that I would like to check in before the merge next week.

Comment 36

[(Currently slow to respond) Philipp Sackl [:phlsa] (Firefox UX) please use needinfo]

Hey David, sorry this is taking so long.
Yes, before/after screenshots would make it a lot easier for me.

Comment 37

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Attachment: before.png

No worries - here's the before.

Comment 38

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Attachment: after.png

... and after.

Comment 39

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Attachment: non-hsts.png

This is what it looks like in both cases when the site isn't HSTS.

Comment 40

[(Currently slow to respond) Philipp Sackl [:phlsa] (Firefox UX) please use needinfo]

Comment on attachment 8468864 [details] [diff] [review]
patch

The copy works well within that setting.

I'd love to make this entire dialog more accessible in general at some point, but that's well outside the scope of this bug.

Comment 41

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Attachment: patch v2

Great - thanks!
I addressed a previous review comment before checking in:
https://hg.mozilla.org/integration/mozilla-inbound/rev/6753df2aec08

Comment 42

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/6753df2aec08

Comment 45

[Ralf Hauser]

see Bug 1334048