418644 - ocspclnt deficient for problem analysis

Status: RESOLVED INACTIVE

(NSS :: Tools, defect, P2)

Description

[Nelson Bolyard (seldom reads bugmail)]

ocspclnt is a tool for testing our OCSP client code.  It is both a code test 
vehicle, and a problem diagnosis tool.  One of its objectives is to help us identify problems that may be present in OCSP responses, but it does VERY poorly at that objective.  

Because of its deficiencies, I had to spend hours in a debugger to determine
what was wrong with the OCSP responder cert in a particular response.  
The tool COULD have made the diagnosis take mere seconds, but it did not.

The deficiencies include:
1) The tool provides no way to get just a binary OCSP response and write it 
to a binary file by itself, or to pretty print it.  The tool can pretty print
a binary response from a file, but not a response directly from a responder,
and it has no simple way to produce a file containing ONLY the binary response
from a responder.

When the tool gets a binary OCSP response from an OCSP responder, it will output that response in one of only these two forms:
a) a hexadecimal dump to stdout (-V and -S commands), or 
b) a hexadecimal dump, followed by a binary dump, to stdout (-R command)

It is possible to separate the binary response from the hex dump in the output, 
and then feed that binary response back into ocspclnt with the -P option.
This requires analyzing the output with a tool like "od -cx" to find the 
offset into the output where the binary response begins, and then using dd
to copy the binary response into a separate file, e.g. 
  dd if=response of=response.der bs=1 skip=NNNN
This procedure is tedious and time consuming, and frankly it is beyond the 
capabilities of most folks.

The tool really ought to offer the choice of outputting the OCSP response
in any one of these 3 forms:
a) binary output ONLY, or
b) hexadecimal dump (not very useful) or 
c) pretty printed output
and ideally it would have an option to write the output to a named file 
rather than only to stdout.

2. The tool ought to have an option to read input from a named file rather 
than only from stdin.

3. The tool ought to have the ability to parse the the certificate(s) from 
the OCSP response, and to optionally output them to separate files 
(separate from the OCSP response output) whether the binary response input comes from a responder or from a local file.

Because it lacks the ability to output the cert(s) separately, after separating
the binary OCSP response from the hex output with dd, it was necessary to 
repeat the procedure with dumpasn1 (to get the decimal offset of the cert)
and dd to separate out the certificate into a separate file.  

I would actually suggest that we drop the hex dump output entirely.  It's 
pretty useless.  The first column of the hex dump output seems to be an
offset of some kind, in hexadecimal.  But it doesn't start with zero, and 
it jumps around (is not continuous), so it's not useful for finding offsets
to use with dd.  That first column is just useless.

Comment 1

[Nelson Bolyard (seldom reads bugmail)]

Attachment: output of ocspclnt -R, containing both hex dump and binary response

Here is a sample output from the command ocspclnt -R
With this output file, one can use the following dd command to 
separate the binary output into its own file.
  dd if=response.bin of=response.der bs=1 skip=8538
Then one can use the following dd command to separate the responder's
signing cert out into its own file
  dd if=response.der of=respondercert.der bs=1 skip=587 count=767

Comment 2

[Nelson Bolyard (seldom reads bugmail)]

Today I discovered that the hex dump about which I complained in comment 0
is actually not in ocspclnt but rather is inside libPKIX.  

Any time libPKIX sends out http requests, whether for OCSP or for AIA, it
outputs a dump of the stuff to stdout.  This is a big problem for ocspclnt.

I will file a separate bug against libPKIX about the hex dumps, and mark it
as blocking this bug.  With that libPKIX problem fixed, the only part 
remaining in this bug will be to add the ability to pretty-print an ocsp 
response directly, without having to write it to a file first, and then 
pretty print the file.  

Comment 3

[Nelson Bolyard (seldom reads bugmail)]

Note: until bug 429716 is fixed, the undesired hex dump can be suppressed
with the environment variable SOCKETTRACE=off

Comment 4

[Nelson Bolyard (seldom reads bugmail)]

I'm assigning this bug to Mr. OCSP.

Here's the REAL PROBLEM:  It takes NSS developers HOURS, working in a 
debugger, to diagnose a SINGLE BAD OCSP RESPONSE.  We need better tools.

In order to completely diagnose and document a problem with a bad OCSP 
response, it is generally necessary to capture all of the following:
  - the complete cert chain for the cert whose status is being checked 
    via OCSP
  - the OCSP response
  - the cert of the OCSP response signer and the complete chain for that 
    cert.  This will often, but not always (especially not in error cases) 
    be a chain with most CA certs in common with the chain for the cert
    being checked via OCSP.
  - the OCSP request (maybe)

Here are some of the problems with our tools.

1) we have no tool that collects and outputs the OCSP responder's cert 
and its chain.  Ocspclnt should be able to output files for all the things
I listed above: the cert chain under test, the request, the response, the 
response signer's cert and the chain for that response signer's cert.

2) ocspclnt has an option to input a stored response and attempt to 
analyze it.  But that option typically produces incorrect results because
it does not have all the inputs it needs to correctly interpret the 
response.  It really needs to take as input all the things I listed above.
Without that input, ocspclnt cannot (in some cases) correctly:

2a) Find the response signer's cert, and determine if the signature on 
the response verifies using the public key in that cert.

2b) Determine that the response signer's cert is qualified to sign the 
response (that is, by virtue of being either the cert for the issuer of
the original cert under test, or being a delegated responder cert issued 
by the issuer of the original cert under test.

For evidence of this problem, see bug 449394 and/or bug 484211.  
The evidence in bug 449394 is plain.  A lot of effort was spent trying to
find out why the cert would not validate, and that effort ultimately led
to no solid evidence of a problem with the OCSP responder (though it should
have done so).  

Bug 484211 is actually a problem with an OCSP responder that sends back 
different responses at different times (about 3 different responses in all)
all of which have problems.  After spending hours in the debugger, I can 
see the problems with the OCSP responder's cert chain, but I cannot document
that easily because there is NO TOOL that will output that chain.  

This is a major issue for NSS.  NSS's credibility is on the line.  When a 
CA sends an invalid response, (but the CA denies that it is invalid), and 
we cannot document the problem, we appear incompetent.  This is AT LEAST 
as important to NSS's survival as libPKIX, IMO.

Comment 5

[Nelson Bolyard (seldom reads bugmail)]

Bug 449394 comment 7 and bug 449394 comment 32 both show ocspclnt displaying incorrect diagnosis of stored OCSP responses, due to having incomplete 
information with which to assess the responses.

Comment 6

[Julien Pierre]

I agree about the deficiency. We need to prioritize this work to a future release. Setting TFM to 3.12.5 for now.

Comment 7

[BugBot [:suhaib / :marco/ :calixte]]

In the process of migrating remaining bugs to the new severity system, the severity for this bug cannot be automatically determined. Please retriage this bug using the new severity system.