429716 - debug builds of libPKIX unconditionally dump socket traffic to stdout

Status: RESOLVED FIXED

(NSS :: Libraries, defect, P1)

Description

[Nelson Bolyard (seldom reads bugmail)]

This is P1 for 3.12.1

libPKIX has code that is compiled into all DEBUG builds that unconditionally
dumps all socket traffic (such as OCSP or AIA cert fetching) to stdout in hex.
You can see an example of this in bug 425847 comment 0 and the attachment to 
bug 418644.  This is the main cause of bug 418644.

This is evil for several reasons:

a) it uses stdio, rather than using NSPR, and so it makes NSS only buildable
on platforms that have stdio.

b) programs that write output to stdout have their output corrupted by this
unwanted dump.  For example, ocspclnt writes the OCSP response in binary to 
stdout.  That binary output is useless when it is corrupted with this hex
dump.

c) it is unconditional.  It should be controlled by some means, such as an 
environment variable, and should NOT be enabled by default.

To fix this bug, 
1) all printf, fprintf, puts, fputs calls must be removed from libPKIX, and replaced with NSPR IO function calls (I recommend using PRLog), and 
2) some means of controlling this output must be established, such as checking
for the presence of an environment variable.  I'd suggest having a variable
named PKIX_SOCKET_TRACE_FILE whose value is the name of the file to which the
NSPR PRLog output will be written.  Using stdout might be an option if (say)
the value of this variable is some magic value (say: stdout).

Comment 1

[Nelson Bolyard (seldom reads bugmail)]

Most (perhaps all) of the code that does this tracing of socket traffic to 
stdout is in file lib/libpkix/pkix_pl_nss/module/pkix_pl_socket.c 
but there are also a bunch of printf calls in widely used PKIX_DEBUG macros 
in lib/libpkix/pkix/util/pkix_tools.h 

All those uses of printf need to be changed to not use printf any more.
Using NSPR to print to stderr *MIGHT* be acceptable.

Comment 2

[Nelson Bolyard (seldom reads bugmail)]

I see that there is an environment variable named "SOCKETTRACE" that is 
supposed to control the socket tracing, 
http://bonsai.mozilla.org/cvsblame.cgi?file=/mozilla/security/nss/lib/libpkix/pkix_pl_nss/module/pkix_pl_socket.c&rev=1.4#793

But it is not used everywhere.  That is, it appears to me that some of the 
printf calls will print, even when SOCKETTRACE tells them not to.

Comment 3

[Nelson Bolyard (seldom reads bugmail)]

I may be wrong about SOCKETTRACE not disabling all the socket trace printfs.
I didn't see any unexpected printf activity when I ran ocspclnt with that
variable, e.g. 

NSS_ENABLE_PKIX_VERIFY=1 SOCKETTRACE=off vfyserv webmail.unicas.it

Simply changing the default value of the socketTraceFlag variable to 0
(false) would go a long way towards reducing the severity of this bug.

Comment 4

[Nelson Bolyard (seldom reads bugmail)]

Attachment: Turn socket tracing off by default

Alexei, please review.

Comment 5

[Nelson Bolyard (seldom reads bugmail)]

Checking in pkix_pl_socket.c; new revision: 1.5; previous revision: 1.4

Comment 6

[Nelson Bolyard (seldom reads bugmail)]

Updated 
http://developer.mozilla.org/en/docs/NSS_reference:NSS_environment_variables