86125 - Unnecessary #if in mpi.c that breaks C preprocessors that don't understand LL constants

Status: RESOLVED FIXED

(NSS :: Libraries, defect, P2)

Description

[Wan-Teh Chang]

In mpi.c, we have the following:

#if DIGIT_MAX < 256
    if((res = s_mp_lshd(mp, 1)) != MP_OKAY)
      return res;
#else
    if((res = mp_mul_d(mp, 256, mp)) != MP_OKAY)
      return res;
#endif

On some platforms, mp_digit is an unsigned long long,
and DIGIT_MAX is a ULL constant.  This breaks some C
preprocessors that don't recognize long longs even
though the C compilers have no problem with them.

Looking at the code, I found that MP_DIGIT_MAX
(which DIGIT_MAX is a synonym for) is ULONG_MAX,
MP_ULONG_LONG_MAX, or UINT_MAX.  So it is at
least UINT_MAX.  Since C requires that int be
at least 16 bit long, UINT_MAX is greater than 256.
So #if DIGIT_MAX < 256 is always false.  Nelson,
is my reasoning correct?

This is my proposed patch.
Index: mpi.c
===================================================================
RCS file: /cvsroot/mozilla/security/nss/lib/freebl/mpi/mpi.c,v
retrieving revision 1.34
diff -u -r1.34 mpi.c
--- mpi.c       2001/05/31 00:12:18     1.34
+++ mpi.c       2001/06/15 19:03:02
@@ -2453,13 +2453,8 @@
 
   /* Read the rest of the digits */
   for(ix = 1; ix < len; ix++) {
-#if DIGIT_MAX < 256
-    if((res = s_mp_lshd(mp, 1)) != MP_OKAY)
-      return res;
-#else
     if((res = mp_mul_d(mp, 256, mp)) != MP_OKAY)
       return res;
-#endif
     if((res = mp_add_d(mp, ustr[ix], mp)) != MP_OKAY)
       return res;
   }

Comment 1

[Nelson Bolyard (seldom reads bugmail)]

This patch is fine with me.

Comment 2

[Wan-Teh Chang]

I checked in the patch on the trunk of NSS.