If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Firefox does not use more than one OCSP path within the AIA-attribute of a certificate

NEW
Unassigned

Status

NSS
Libraries
5 years ago
5 years ago

People

(Reporter: conny, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

5 years ago
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20100101 Firefox/15.0.1
Build ID: 20120905151427

Steps to reproduce:

Visited an SSL-site that has a certificate with multiple OCSP-paths within the AIA-extension of the certificate.


Actual results:

When Firefox attempts to validate the certificate  by means of OCSP it will only try the last OCSP-path specified within the certificate. If this path fails will not try other paths specified within the certificate.


Expected results:

Firefox should try all OCSP-paths that are specified within the AIA-extension of a certificate before it fails validation checking. Some CAs make use of multiple OCSP-responders with different paths for fault tolerance or in network isolation scenarios, with this behavior of Firefox you can only make use the last path specified in the certificates AIA-extension.
Assignee: nobody → nobody
Component: Untriaged → Libraries
Product: Firefox → NSS
Version: 15 Branch → unspecified

Comment 1

5 years ago
see use of CERT_GetOCSPAuthorityInfoAccessLocation
Status: UNCONFIRMED → NEW
Ever confirmed: true

Comment 2

5 years ago
Dear Kai,

I think you have right, the bug is there:

4416: for (i = 0; authInfoAccess[i] != NULL; i++) {

this for cycle gets only the latest AIA:OCSP... 
some randomness can solve here the to not overload the latest responder, if all of the AIA:OCSPs read out then stored, then select with a random nomber which of the URI should given back to the caller, but I think the correct way to modify the function here and give back a list for the caller...it sohuld be solve on upper level, maybe here: ocsp_GetOCSPStatusFromNetwork 

Those caller then needs to try each of the URIs, until its get its first valid OCSP response from one of URIs... and also here needed some randomnes too, because if there is no randomness, always the 1st will be selected if its working, it gets high loads, request goes on 2nd only when the first URI is dead.

Sorry to not give source code, but I am not familiar with coding rules here, and C is not my well used language...

Comment 3

5 years ago
Both Brian and I have discussed this in the context of the CA/Browser Forum and application best practices. I think we both have concerns about such failover behaviours, at least in the context of publicly-trusted CAs.

There have been a variety of discussions in the forum - always try in-order of the sequence, do a random sampling until no more are available, do one and then fail, etc.

I don't believe that the AIA should be used for failover or load balancing - it should represent equivalent URLs (equivalent in content, equivalent in availability, equivalent in guarantees). Which URL an application selects is application dependent, and I don't think it's necessary to try all URLs.

This is similar to how notions like DNS resolution and connection work (compare A records vs MX records, for example).

Comment 4

5 years ago
(In reply to Ryan Sleevi from comment #3)
> I don't believe that the AIA should be used for failover or load balancing -
> it should represent equivalent URLs (equivalent in content, equivalent in
> availability, equivalent in guarantees). Which URL an application selects is
> application dependent, and I don't think it's necessary to try all URLs.

I agree with you that you dont need to try all the URLs, and not should used as load balance, but take a look on some different scenario:

1. FFox uses the first available, when unavailable jumps to the next
For a high traffic site FFox leches the ocsp, then kills this responder with an overload... 
Results: 
-for website: unavailability
-for CA: customer calls, workaround
-for user: unavailable website, or long timeouts

2. FFox use randomly from the available, selects randomly the next if not working
For a high traffic site FFox tries different ocsp responders, so the same traffic can not kill that one responder... 
Results: 
-for website: more reliable connection than previous
-for CA: load balanced trough sites
-for user: available website, or long timeouts

3. Actual state
For a high traffic, FFox uses the last, if not working drops error...
Results: 
-for website: unavailability
-for CA: customer calls, workaround
-for user: unavailable website, or long timeouts

Its clear from these examples, to use only one is the worsest scenario (3), to jump to the next (1) is better, but the best is the randomize, because it not loads so higly...

An old example:
For a security software we found a few years ago, that all installed clients downloaded the CRLs at midnight. (it was timed to midnight.) that software was popular, so a lot of clients tried to leech the CRLs in the same minute. this was a DDoS for the CRLs and when the publisher of that software was contacted, they quickly randomized the downloads because they know, they cant get results if they kill the webserver.

this is why I think that the first available should used, and why the randomizing is important.

reagds. Viktor

Comment 5

5 years ago
Hello NSS team!

Red Hat's Identity Management team (FreeIPA project) would like to vouch for implementing the OCSP fail over feature to NSS/Firefox. We or working on building an identity management solution including Certificate management, aimed both at enterprise users and small admins.

OCSP failover ability is very important for us as the certificate the are signed by the FreeIPA server CA contains 2 AIA entries:
 * one URI containing a hostname of the CA who signed the certificate
 * one generic URI containing a single CNAME or a set of A records pointing to other configured FreeIPA CA replicas. This DNS name is either configured by FreeIPA or optionally by Administrator when FreeIPA does not control the DNS.

Example certificate published by FreeIPA server:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 17 (0x11)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: O=EXAMPLE.COM, CN=Certificate Authority
        Validity
            Not Before: Apr  8 10:16:15 2013 GMT
            Not After : Apr  9 10:16:15 2015 GMT
        Subject: O=EXAMPLE.COM, CN=testcert.example.com
        ...
        X509v3 extensions:
            X509v3 Authority Key Identifier: 
                keyid:9F:25:93:2F:20:2A:79:9A:A8:88:CF:CC:EB:D0:F5:43:E7:3B:B1:EE

            Authority Information Access: 
                OCSP - URI:http://ipa-ca.example.com/ca/ocsp
                OCSP - URI:http://server1.example.com/ca/ocsp

            X509v3 Key Usage: critical
                Digital Signature, Non Repudiation, Key Encipherment, Data Encipherment
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 CRL Distribution Points: 

                Full Name:
                  URI:http://ipa-ca.example.com/ipa/crl/MasterCRL.bin
                CRL Issuer:
                  DirName: O = ipaca, CN = Certificate Authority

                Full Name:
                  URI:http://server1.example.com/ipa/crl/MasterCRL.bin
                CRL Issuer:
                  DirName: O = ipaca, CN = Certificate Authority
        ...

This approach has 2 benefits:
* User has a fully working OCSP validation solution out of the box without a need to control and manage the ipa-ca.example.com DNS name
* Clients are still able validate the published certificate via the fail over if the original CA was decommissioned, either after a unrecoverable failure

Currently, when Firefox/NSS validates a web server controlled by the FreeIPA server, it only tries the hostname server1.example.com, but does not try to connect to the second AIA (ipa-ca.example.com) if the first AIA is down - resulting in a failure to connect to the web server in case of a strict OCSP validation.

Including the OCSP failover feature in NSS would help us to solve this use case requested by enterprise users. If you do not consider this option as a good default for Firefox, we would welcome a Firefox option that could be used to enable this feature.

Thank you,
Martin Kosek
(In reply to Martin Kosek from comment #5)
> Currently, when Firefox/NSS validates a web server controlled by the FreeIPA
> server, it only tries the hostname server1.example.com, but does not try to
> connect to the second AIA (ipa-ca.example.com) if the first AIA is down -
> resulting in a failure to connect to the web server in case of a strict OCSP
> validation.

Once Firefox supports OCSP stapling, all major browsers will support OCSP stapling and you can avoid this issue altogether in your product by simply having it staple the response. That's a better solution than making the fetching of OCSP statuses more complicated. In general, servers can no longer expect web browsers to fetch OCSP responses via HTTP. Chrome stopped doing it and there's an increasingly high likelihood that Firefox will too.

> Including the OCSP failover feature in NSS would help us to solve this use
> case requested by enterprise users. If you do not consider this option as a
> good default for Firefox, we would welcome a Firefox option that could be
> used to enable this feature.

It is very unlikely that we will make this behavior the default in Firefox or provide such an option.

Comment 7

5 years ago
For our purposes the problem is that the OCSP responder will not honor multiple OCSP AIAs. This is true whether stapling is used or not. Something, somewhere, has to make the OCSP request. It is at that point we'd like it to use all values. Whether that is the browser or not makes no difference to us.

We use a CA, dogtag, that has the ability to clone itself so the same CA can exist simultaneously on multiple servers. By using a CNAME for the OCSP AIA we can have a generic name available so that if one of the CA machines goes away all the certificates it has issued suddenly don't have a way to work (without having to do other DNS gymnastics). For purposes of stability we would still like the original server listed there for a chance at failover.

Comment 8

5 years ago
(In reply to Rob Crittenden from comment #7)
> For our purposes the problem is that the OCSP responder will not honor
> multiple OCSP AIAs. This is true whether stapling is used or not. Something,
> somewhere, has to make the OCSP request. 

In the case of OCSP stapling, it's the server software (on each server) that staples the OCSP response to the handshake.

This means, in that scenario, the server software would have to be try multiple requests.

So, in theory, if NSS knew how to try multiple responders, a server software based on NSS could enable that feature, regardless whether a NSS based client enables the feature.
(In reply to Rob Crittenden from comment #7)
> For our purposes the problem is that the OCSP responder will not honor
> multiple OCSP AIAs. This is true whether stapling is used or not. Something,
> somewhere, has to make the OCSP request. It is at that point we'd like it to
> use all values. Whether that is the browser or not makes no difference to us.

Yes, that makes sense too. But, also, it isn't good to make certificates larger by including too much stuff in them. There are performance costs to making the certificates bigger.

(In reply to Kai Engert (:kaie) from comment #8)
> So, in theory, if NSS knew how to try multiple responders, a server software
> based on NSS could enable that feature, regardless whether a NSS based
> client enables the feature.

Yes, that sounds reasonable to me. But, this bug is about Firefox, according to the summary. Should we change the summary to change the scope of the bug to be NSS.

Comment 10

5 years ago
Hello Brian and Kai,

We have been discussing this topic in our team and given the fraction to implement OCSP fail over in NSS as a default, we decided to rather workaround it on FreeIPA server side by defining just one general OCSP URI (http://ipa-ca.example.com/ca/ocsp) in certificates we issue. This will place a demand on our users to maintain this general URI, but in advance it will allow us to address OCSP validation for as many clients and platforms as possible.

Details and links to respective upstream tickets are in the freeipa-users thread dedicated to this issue:
https://www.redhat.com/archives/freeipa-users/2013-April/msg00088.html

Thanks,
Martin
You need to log in before you can comment on or make changes to this bug.