413997 - CERT_PKIXVerifyCert ignores FAIL_ON_MISSING_FRESH_INFO for CRLs

Status: RESOLVED DUPLICATE of bug 205434

(NSS :: Libraries, defect, P2)

Description

[Kai Engert [:KaiE:]]

This bug is about the following scenario:

- PSM contains CA root cert blessed for EV
- Firefox visits a https:// page
- the SSL server cert does NOT contain a OCSP AIA extension
- OCSP is enabled
- PSM calls into CERT_PKIXVerifyCert, requesting an OCSP check

I used a entrust root for testing, added to the external test_ev_roots.txt file, enabled using the environment variable.

Actual behavior:
  EV UI with green identity indicator is shown

Expected behavior:
  No EV UI, because no OCSP check was done.


More info:
Unfortunately we are not yet able to rely on dynamical CRL retrieval.
As a consequence, CRL availability is not sufficient.
For now, we require OCSP.


The bug is, we are not consistently requiring OCSP for EV.

Comment 1

[Kai Engert [:KaiE:]]

In theory PSM could get enhanced to test a server cert for the presence of a OCSP AIA extension, and if missing refuse EV status. But is that really the right approach?

I personally think this should be fixed at the NSS level.

Assigning to NSS.

Comment 2

[Frank Hecker]

Kai, could I get clarification on one point related to this? Let's suppose we have a CA root cert included in Firefox and marked as supporting issuance of EV certs, but the CA in question does not currently maintain an OCSP responder, and does not therefore include the OSCP AIA extension in EV SSL certs it issues. If the CA subsequently stands up an OCSP responder, what is required for that CA's EV certs to receive the EV UI in Firefox? Presumably the CA would have to include the OCSP AIA extension in any new SSL EV certs it issues, and would also have to reissue old certs to add the extension. Is there anything else required, either on the part of the CA or on our part?

Comment 3

[Kai Engert [:KaiE:]]

(In reply to comment #2)
> If the CA subsequently stands up an OCSP responder, what is required
> for that CA's EV certs to receive the EV UI in Firefox?

The old certs (lacking OCSP AIA) will not get EV UI.


> Presumably the CA would
> have to include the OCSP AIA extension in any new SSL EV certs it issues, and
> would also have to reissue old certs to add the extension.

Correct. The CA would have to issue new EV certs, now containing the OCSP AIA extension, that can be validated against their approved and included root and policy ID. Each server will have to have its configuration updated to use the new server certificate.

Comment 4

[Robert Relyea]

The problem is we do not have a flag that says 'require some revocation checking' in PKIX. Once we have that flag, then turning on that flag and the OCSP flag will yield the desired results.


(and map into the future when we can fetch CRL's).

Comment 5

[Kai Engert [:KaiE:]]

Attachment: Patch v1 for discussion

(In reply to comment #4)
> The problem is we do not have a flag that says 'require some revocation
> checking' in PKIX. Once we have that flag, then turning on that flag and the
> OCSP flag will yield the desired results.


Bob, could we use something like this?
I guess you might tell me that procparams is a better candidate to carry that flag, but what do you think about this approach?


However, it's more complicated.
While this change might be correct for EE certs, it might be bad to require it in general.

For example, the chain for paypal's includes two intermediates.
Intermediate A contains a OCSP AIA.
Intermediate B contains a CRLDP only.

With this change we might reject the chain because of the OCSP failure of an intermediate (once bug 414911 gets cleaned up).

So, in order to be smart, the flag transported in context/procparams would have to allow this special mode: "Require OCSP in the EE cert, but allow missing OCSP AIA in intermediates and roots".

IMHO, as a consequence the decision "ignore this OCSP failure" must be done elsewhere. When pkix_OcspChecker_Check can not complete an OCSP check (for whatever reasons, including missing URI), the caller should decide whether that's an error or whether it can be ignored.

The result of pkix_OcspChecker_Check should tell whether the cert was revoked or whether the URL was missing.

The chain building code that calls into pkix_OcspChecker_Check should make use of that error result.

That chain building code should inspect the flags that describe the revocation requirements.
Depending on the state of the flag
  (do not require OCSP
   or require OCSP in EE cert only
   or (maybe) require OCSP everywhere)
the chain building code should make smart decisions and ignore OCSP failures as allowed by the flag.

Comment 6

[Robert Relyea]

> However, it's more complicated.
> While this change might be correct for EE certs, it might be bad to require it
> in general.
> 
> For example, the chain for paypal's includes two intermediates.
> Intermediate A contains a OCSP AIA.
> Intermediate B contains a CRLDP only.
> 
> With this change we might reject the chain because of the OCSP failure of an
> intermediate (once bug 414911 gets cleaned up).

So I'm confused. If the OCSP request fails on Intermediate A, then we need to fail the chain.

If you mean Intermediate B, then yes, that is what this code is intended to do (since we didn't check the revocation status of Intermediate B).

In this case Intermediate B is replaces by the new EV root, so we would validate that.

> The result of pkix_OcspChecker_Check should tell whether the cert was revoked
> or whether the URL was missing.

Missing should return the state 'unknown', not revoked. There are 3 states you
can get back from a revocation check: unknown, revoked, good. 

We need flags to allow the application to control the results of these various checks:

1) revoked - should always fail the chain.
2) good - revocation is found and successful.
3) unknown - fall back to alternate revocations scheme. If no revocation status
is found then we either:
   1) succeed if revocation checks is not required.
   2) fail if revocation checks are required.

> So, in order to be smart, the flag transported in context/procparams would
> have to allow this special mode: "Require OCSP in the EE cert, but allow
> missing OCSP AIA in intermediates and roots".

What we have said (to date) is the we will require OCSP to chain to EE. If we determine that this is not tenable, we can rethink that. (you are asking for a require OCSP on the leaf.

BTW that semantic would not be implemented by the general libpkix library, only CERT_PKIXVerifyCert itself.

> Depending on the state of the flag
>  (do not require OCSP
>   or require OCSP in EE cert only
>   or (maybe) require OCSP everywhere)
> the chain building code should make smart decisions and ignore OCSP failures as
> allowed by the flag.

yes.

Comment 7

[Kai Engert [:KaiE:]]

> In this case Intermediate B is replaces by the new EV root, so we would
> validate that.

I was not aware. Thanks for clarifying.


> If the OCSP request fails on Intermediate A, then we need to
> fail the chain.

ok, makes sense now


> > The result of pkix_OcspChecker_Check should tell whether the cert was revoked
> > or whether the URL was missing.
> 
> Missing should return the state 'unknown', not revoked.

Agreed.


> There are 3 states you
> can get back from a revocation check: unknown, revoked, good. 

However, function pkix_OcspChecker_Check doesn't produce such a 3-state output.
It returns a result code, which can be success, or an error code.
As of today, it's the caller's responsibility to translate those codes into unknown or revoked.

Should that API get changed?
Should pkix_OcspChecker_Check decide whether the result is unknown/revoked/good?


> We need flags to allow the application to control the results of these various
> checks:
> 
> 1) revoked - should always fail the chain.
> 2) good - revocation is found and successful.
> 3) unknown - fall back to alternate revocations scheme. If no revocation status
> is found then we either:
>    1) succeed if revocation checks is not required.
>    2) fail if revocation checks are required.

I agree.
So the open questions are:
- what mechanism should be used to transport these flags to libpkix?
  (probably in proc-params)
- where should that code live that evaluates the flag and produces
  the unknown/good/revoked result?
  (probably inside pkix_OcspChecker_Check with an API change)
  

> > So, in order to be smart, the flag transported in context/procparams would
> > have to allow this special mode: "Require OCSP in the EE cert, but allow
> > missing OCSP AIA in intermediates and roots".
> 
> What we have said (to date) is the we will require OCSP to chain to EE. If we
> determine that this is not tenable, we can rethink that. (you are asking for a
> require OCSP on the leaf.

Sorry, I don't understand this sentence!

But I do understand your plans based on the API you proposed in bug 294531.

Yes, in the example above, I was asking for "require OCSP" on the leaf.

Regarding EV, I'm not sure what our requirement will be?
Either "require OCSP on the leaf" or "require OCSP on leaf and intermediates"?

We should make a decision and document it on
http://wiki.mozilla.org/CA:EV_Revocation_Checking

Comment 8

[Nelson Bolyard (seldom reads bugmail)]

> function pkix_OcspChecker_Check doesn't produce such a 3-state output.
> It returns a result code, which can be success, or an error code.
> As of today, it's the caller's responsibility to translate those codes into
> unknown or revoked.
>
> Should that API get changed?

I think maybe it should, yes.  I can't be certain without studying that code
a lot more than I'm prepared to do right now.  But success/failure doesn't
seem like the right set of responses for the test being performed.

> Should pkix_OcspChecker_Check decide whether the result is
> unknown/revoked/good?

I think there are several questions about the design of that API.  
The question you asked just above is one.  Another one is:
  Should pkix_OcspChecker_Check turn an "unknown" result into a "good" 
  result or a revoked result, according to some policy hints?  

Comment 9

[Alexei Volkov]

(In reply to comment #8)
> > function pkix_OcspChecker_Check doesn't produce such a 3-state output.
> > It returns a result code, which can be success, or an error code.
> > As of today, it's the caller's responsibility to translate those codes into
> > unknown or revoked.
> >
> > Should that API get changed?
IMO, pkix_OcspChecker_Check should return(but it currently does't) statuses in case 1. when check was complete: pass/fail, 2. all other cases(no AIA, no connection to a server and invalid cache): unknown. Higher level function, that is responsible for collecting rev statuses from all rev checkers, should check returned status and make a decision based on combination of input parameters.

> 
> > Should pkix_OcspChecker_Check decide whether the result is
> > unknown/revoked/good?
> 
> I think there are several questions about the design of that API.  
> The question you asked just above is one.  Another one is:
>   Should pkix_OcspChecker_Check turn an "unknown" result into a "good" 
>   result or a revoked result, according to some policy hints?  
Nelson, could you please elaborate, why it can be necessary to do such status change in ocsp checker besed on a particular policy? It seems to me, that the combination of revocation input flags proposed in bug 294531 and tree possible output statuses from ocsp checker(or any revocation checker in general) should be enough.

Comment 10

[Rob Stradling]

Will this bug be fixed in time for Firefox 3?
If "yes", please note that Mozilla will be putting Comodo, and potentially other CAs, in a rather awkward position:

We (the CAs) *must* include our OCSP Responder URL in the AIA extension of all the EV SSL certificates we issue, in order to receive the EV UI treatment in Firefox 3.
BUT...
We (the CAs) *cannot* include our OCSP Responder URL in the AIA extension of any SSL certificates we issue without causing Firefox 2 (with OCSP switched on) to display an error and refuse to navigate to the https sites that use those certificates.  See Bug #419030.

I can think of several options for avoiding, or at least decreasing the impact, of this problem:
  #1. Don't require OCSP for EV in Firefox 3.0.0.0.  Instead, allow either OCSP or CRL revocation checking for EV.  Then the affected CAs would be able to omit the AIA OCSP URL, still achieve the EV UI in FF3 (relying on a CRL), and not trigger Bug #419030 behaviour in FF2.
  #2. Fix Bug #419030 in time for Firefox 2.0.0.13, and launch Firefox 2.0.0.13 no later than Firefox 3.0.0.0.  Then at least the majority of Firefox 2.x users would upgrade and not be affected by Bug #419030.
  #3. Persuade all of the affected CAs (whoever they may be) to use delegated OCSP Response signing, rather than signing each OCSP Response with the same CA that issued each certificate.

I can't speak for any other CA that might be affected, but I can say that Comodo really do *not* want to have to implement #3.  We have invested considerable resources in developing a new OCSP Responder that is fully compliant with both RFC2560 and the recently published RFC5019.  We issue a lot of end-entity certificates so we anticipate a huge number of hits, because Vista/IE7 has, and Firefox 3 will have, revocation checking enabled by default.  We have deliberately opted to avoid using delegated OCSP Response signing, in order to keep the size of each OCSP Response (and therefore our bandwidth usage) to a minimum.  Our OCSP Responses are typically less than 500 bytes long, whereas delegated OCSP Response signing would increase the size to more like 2,000 bytes (because the OCSP Signing Certificate would need to be included in the OCSP Response).

IMHO...in an ideal world...
  - Mozilla should do option #1, because the CA/Browser Forum's EV Guidelines permit a valid EV SSL Certificate to include either a CRL or OCSP URL.
  - Mozilla should do option #2, especially as Bug #419030 was apparently fixed in the NSS trunk 9 months ago!
  - Comodo and any other affected CAs should not be required to do option #3 when their OCSP Responders are not at fault.

I'm not sure how anyone will respond to this message, but I hope that you can at least see things from our perspective.

Comment 11

[Nelson Bolyard (seldom reads bugmail)]

Rob, No one who works on NSS can answer the question of whether Mozilla
will take the fix for this bug in FF3, but it is our intention to provide
that fix.

This bug is the wrong place for your comment 10 about the FF2 issue. 
This is an NSS bug. The people who need to see your comment about the FF2 
issue are not NSS people but Mozilla people, who do not read NSS bugs.  
Bug 419030 is the right place.  Maybe also in mozilla.dev.tech.crypto 
or mozilla.dev.security

When you add your comment there, add an explanation of why you think 
problems with OCSP in FF2 are a major issue.  Do you have statistics to 
back the idea that many FF2 users actually use OCSP?  OCSP is still not
enabled by default in FF2.  But add that explanation in that other place
where you re-raise the FF2 issue.

Comment 12

[Rob Stradling]

(in reply to comment #11)
> ...Bug 419030 is the right place...

Thanks Nelson.  I'll raise the issue there.

One more thing...

(in reply to comment #0)
> For now, we require OCSP.

http://wiki.mozilla.org/CA:EV_Revocation_Checking asks the question: "Will Firefox 3 require successful OCSP transactions for each intermediate certificate, too?"

Is the NSS team able to clarify this point yet?
I'd really like to know if we're going to need to reissue (and distribute to our customers) any of our Intermediate Certificates that don't currently include AIA->OCSP URLs.  Thanks.

Comment 13

[Nelson Bolyard (seldom reads bugmail)]

Rob,
NSS provides the means whereby the calling application can specify whether
it wants revocation checking on just the leaf, or leaf and intermediates,
and whether it requires the check to actually be done, or only if the cert
contains the necessary extensions.  

Your question is: what will PSM (component of the Mozilla applications) ask
NSS to do, with respect to requiring revocation checking on intermediate CAs?  
That is not an NSS question.  It's not up to the NSS team to decide what the
applications that use NSS will do with it.

Comment 14

[Kai Engert [:KaiE:]]

The recent checkins to bug 294531 and 406755 answer your question.

+  CERTRevocationMethodIndex preferedRevMethods[1] = { 
+    cert_revocation_method_ocsp
+  };
+
+  PRUint64 revMethodFlags = 
+    CERT_REV_M_TEST_USING_THIS_METHOD
+    | CERT_REV_M_ALLOW_NETWORK_FETCHING
+    | CERT_REV_M_ALLOW_IMPLICIT_DEFAULT_SOURCE
+    | CERT_REV_M_REQUIRE_INFO_ON_MISSING_SOURCE
+    | CERT_REV_M_FAIL_ON_MISSING_FRESH_INFO
+    | CERT_REV_M_STOP_TESTING_ON_FRESH_INFO;
+
+  PRUint64 revMethodIndependentFlags = 
+    CERT_REV_MI_TEST_ALL_LOCAL_INFORMATION_FIRST
+    | CERT_REV_MI_REQUIRE_SOME_FRESH_INFO_AVAILABLE;
+
+  PRUint64 methodFlags[2];
+  methodFlags[cert_revocation_method_crl] = revMethodFlags;
+  methodFlags[cert_revocation_method_ocsp] = revMethodFlags;
+
+  CERTRevocationFlags rev;
+
+  rev.leafTests.number_of_defined_methods = cert_revocation_method_ocsp +1;
+  rev.leafTests.cert_rev_flags_per_method = methodFlags;
+  rev.leafTests.number_of_preferred_methods = 1;
+  rev.leafTests.preferred_methods = preferedRevMethods;
+  rev.leafTests.cert_rev_method_independent_flags =
+    revMethodIndependentFlags;
+
+  rev.chainTests.number_of_defined_methods = cert_revocation_method_ocsp +1;
+  rev.leafTests.cert_rev_flags_per_method = methodFlags;
+  rev.chainTests.number_of_preferred_methods = 1;
+  rev.chainTests.preferred_methods = preferedRevMethods;
+  rev.chainTests.cert_rev_method_independent_flags =
+    revMethodIndependentFlags;

See the code added with bug 294531 for comments on these flags.

Comment 16

[Kai Engert [:KaiE:]]

Sigh, now I completely messed it up :-) I apologize. Comment 15 is wrong.

We still use CERT_REV_M_REQUIRE_INFO_ON_MISSING_SOURCE.

We dropped CERT_REV_M_FAIL_ON_MISSING_FRESH_INFO.

See the patch in bug 423475.

Comment 17

[Kai Engert [:KaiE:]]

Today I tried to repeat the test that originally caused me to file this bug: Visiting a site where the server cert lacks the OCSP AIA. Unfortunately my original test case no longer works, because the server cert has been changed to include the OCSP AIA.

I should attempt to find a server that uses a server cert containing any policy OID, but no OCSP AIA, and bless that combination for EV in a test build.

Comment 18

[Nelson Bolyard (seldom reads bugmail)]

Shortly before FF3 RC0, we enhanced the definition of the API for  CERT_PKIXVerifyCert to have very complete control over revocation checking.
The next step is to implement that new specification. 
I think this bug is merely waiting for that implementation.

Comment 19

[Nelson Bolyard (seldom reads bugmail)]

I believe that this bug essentially requests that the new API defined in 
bug 294531 be fully implemented in NSS.  As such, I believe this bug is a 
duplicate of bug 205434, which requested exactly that, which was resolved 
as fixed about 5-6 weeks ago.  

Kai, do you agree?  If so, I propose to mark this bug as a duplicate of 
bug 205434.

Comment 20

[Rob Stradling]

Nelson, surely this bug was fixed in NSS 3.12.3?

See https://bugzilla.mozilla.org/show_bug.cgi?id=474606#c3 for example.

Comment 21

[Nelson Bolyard (seldom reads bugmail)]

Rob, correct me if I'm wrong here.

NSS 3.12.3 contains a facility by which an application can register a 
callback function to establish OCSP URLs for certs that don't have them, 
and a patch has been written to establish an OCSP URL for certs issued by 
CAs affiliated with your company.  (I'm not sure if that's checked in, or 
is held up by Boris's memory usage concerns.)  Together these two changes constitute a workaround for this bug, a workaround that benefits certs issued by your company's CAs.  

Are you suggesting that this bug is entirely fixed by that?
Are you suggesting that this bug has been entirely fixed by some other 
change? and if so, what change?

Comment 22

[Rob Stradling]

In reply to comment #21:

Nelson, IIUC this bug says that Firefox 3.0.x...
  - makes no attempt to perform an OCSP lookup when AIA->OCSP is not present in a certificate (which is fine).
  - is quite happy to display the EV UI when no attempt has been made to perform an OCSP lookup (which is bad).
  - should be fixed so that the EV UI can only be displayed when an OCSP lookup has been performed.

An example EV cert that is affected by this problem is the cert for https://addons.mozilla.org.  This cert does give the EV UI in Firefox 3.0.8, but it does *not* give the EV UI in mozilla-1.9.1 (Shiretoko, Firefox 3.1/3.5).  See bug #474606.

So I say that this bug is fixed in Shiretoko / NSS 3.12.3.  And in bug #474606 comment #9, Kaspar came to exactly the same conclusion:
"Bug 413997 is about a similar case, and I assume it's also addressed by this
NSS upgrade".

Now, as a result of this changed behaviour, we had to start thinking about how to restore the EV UI for certs that don't contain AIA->OCSP, but for which there does exist some source of revocation information...and that's when we first started talking about manually importing CRLs (bug #477244) and when I first proposed embedding some default OCSP Responder URLs (bug #483168 and bug #485052).

AFAIK, this bug is not fixed (and I presume it will not be fixed) in the FF3.0.x branch.

Comment 23

[Nelson Bolyard (seldom reads bugmail)]

Rob, one of the problems I have with the many EV-related bugs presently on 
file is that for each one, it is VERY difficult to answer these questions:

1) What SINGLE problem is this bug about?
2) What makes this bug different from any/all of the other EV-related bugs?
3) What exact change in symptomatic behavior is required to fix this bug?

I cannot answer any of those questions for this bug.  And note that 
"fix all outstanding EV issues" is not an acceptable answer to question 3.

The EV bugs tend to include the kitchen sink. They all seem to list all the EV-related problems, and includes examples from sites with different problems.  They tend to include several separate problems, and they overlap terribly.  

Now, at the top of comment 22, you made a clear definition of a problem,
which I will summarize as : 
   FF 30x displays EV UI without any revocation check
and you seem to acknowledge that that single problem has been fixed by NSS 3.12.x, but then at the end of the comment, you say that this bug is not 
fixed, which tells me that you think this bug is really about something else.  

So, Rob, please answer the questions I asked above (in this comment) for me
with respect to this bug.  Thanks.

Comment 24

[Wan-Teh Chang]

Rob said this bug is not fixed in the Firefox 3.0.x branch,
because the Firefox 3.0.x branch is still using NSS 3.12.2:

http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/client.mk&rev=1.391&mark=414#407

This bug is fixed in the Firefox trunk (mozilla-central)
and the upcoming Firefox release based on mozilla-1.9.1,
because they are using NSS 3.12.3.

Comment 25

[Rob Stradling]

In reply to comment #23:
> Rob, one of the problems I have with the many EV-related bugs presently on
> file is that for each one, it is VERY difficult to answer these questions:

Agreed.

> 1) What SINGLE problem is this bug about?

In my opinion, the single problem is accurately described by Kai in this bug's Description and again in comment #17.  

> 2) What makes this bug different from any/all of the other EV-related bugs?

I don't think I can answer that.  I'm sure that the single problem is already fixed in NSS 3.12.3 / Shiretoko, but I don't know exactly which bugs tracked this behaviour change.

> 3) What exact change in symptomatic behavior is required to fix this bug?

Exact change: Stop allowing the absence of AIA->OCSP in an EV cert to cause FF3.something to show the EV UI without performing any kind of revocation check.

> but then at the end of the comment, you say that this bug is not fixed, which
> tells me that you think this bug is really about something else.

Wan-Teh explained my fixed vs non-fixed in comment #24.

Nelson, by all means leave this bug open if you feel I've overlooked something, or if you believe that there's still a chance that the single problem may yet be fixed in the FF3.0.x branch.

Comment 26

[Nelson Bolyard (seldom reads bugmail)]

The previous subject of this bug:

  Incorrect EV behavior with server certs revocation checking

was effectively inclusive of all possible incorrect EV behaviors, and with 
such a broad definition, we cannot answer my questions from comment 23.
So, I'm changing the bug description to describe a single problem, using 
Rob's answers in comment 25 as my basis, to 

  CERT_PKIXVerifyCert ignores FAIL_ON_MISSING_FRESH_INFO for CRLs

I believe this was fixed as part of the work done for 
  Bug 205434 - Fully implement new libPKIX cert verification API
See 
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/security/nss/lib/libpkix/pkix/checker/pkix_crlchecker.c&rev=1.4&mark=449-453#448

So, I am marking this bug as a duplicate of bug 105434, as I proposed to do
in comment 19.