818741 - Fix gcc compiler warnings "cast to pointer from integer of different size" in certvfy.c in 64-bit builds

Status: RESOLVED FIXED

(NSS :: Libraries, defect, P2)

Description

[Wan-Teh Chang]

Attachment: Patch

When doing a 64-bit build on Linux with gcc 4.4.3, I get the
following compiler warnings in certvfy.c:

gcc -o Linux2.6_x86_64_glibc_PTH_64_DBG.OBJ/certvfy.o -c -g -ansi -D_POSIX_SOURCE -D_BSD_SOURCE -D_XOPEN_SOURCE -fPIC -DLINUX2_1  -Wall -Werror-implicit-function-declaration -Wno-switch -pipe -DLINUX -Dlinux -DHAVE_STRERROR -DXP_UNIX -DDEBUG -UNDEBUG -DDEBUG_wtc -D_REENTRANT -DNSS_ENABLE_ECC -DNSS_ECC_MORE_THAN_SUITE_B -DUSE_UTIL_DIRECTLY -I../../../../dist/Linux2.6_x86_64_glibc_PTH_64_DBG.OBJ/include -I../../../../dist/public/nss -I../../../../dist/private/nss  certvfy.c
certvfy.c: In function ‘cert_VerifyCertChainOld’:
certvfy.c:504: warning: cast to pointer from integer of different size
certvfy.c:568: warning: cast to pointer from integer of different size
certvfy.c:596: warning: cast to pointer from integer of different size
certvfy.c:624: warning: cast to pointer from integer of different size
certvfy.c: In function ‘CERT_VerifyCACertForUsage’:
certvfy.c:839: warning: cast to pointer from integer of different size
certvfy.c:864: warning: cast to pointer from integer of different size
certvfy.c: In function ‘CERT_VerifyCertificate’:
certvfy.c:1160: warning: cast to pointer from integer of different size
certvfy.c:1167: warning: cast to pointer from integer of different size
certvfy.c:1176: warning: cast to pointer from integer of different size
certvfy.c: In function ‘CERT_VerifyCert’:
certvfy.c:1298: warning: cast to pointer from integer of different size
certvfy.c:1302: warning: cast to pointer from integer of different size
certvfy.c:1308: warning: cast to pointer from integer of different size

The attached patch fixes the warning. The fix works by using an intermediate
cast to PRWord, which is an integer type that is the same size as a pointer,
equivalent to the intptr_t type of C99.

I also changed the type of the |error| argument to |long| because NSPR
error code is a signed type. For example, PORT_GetError() returns int:
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/security/nss/lib/util/secport.h&rev=1.29&mark=75#75

The |error| argument is ultimately stored in the |error| field of the
CERTVerifyLogNodeStr structure, which is a |long|:
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/security/nss/lib/certdb/certt.h&rev=1.57&mark=734,736,738#729

Comment 1

[Wan-Teh Chang]

Patch checked in on the NSS trunk (NSS 3.14.1).

Checking in lib/certdb/certi.h;
/cvsroot/mozilla/security/nss/lib/certdb/certi.h,v  <--  certi.h
new revision: 1.38; previous revision: 1.37
done
Checking in lib/certhigh/certvfy.c;
/cvsroot/mozilla/security/nss/lib/certhigh/certvfy.c,v  <--  certvfy.c
new revision: 1.78; previous revision: 1.77
done