1384827 - tautological comparison in pqg.c triggering on clang/32-bits

Status: RESOLVED DUPLICATE of bug 1796308

(NSS :: Libraries, defect, P3)

Description

[Landry Breuil (:gaston)]

Due to -Werror,-Wtautological-constant-out-of-range-compare, nss build on OpenBSD/i386 fails with clang:

cc -o OpenBSD6.1_OPT.OBJ/OpenBSD_SINGLE_SHLIB/pqg.o -c -O -fPIC -DPIC -Di386 -Wall -Wno-switch -pipe -DOPENBSD -Wall -Qunused-arguments -Wno-parentheses-equality -W    no-array-bounds -Wno-unevaluated-expression -Werror -DXP_UNIX -DSHLIB_SUFFIX=\"so.48.0\" -DSHLIB_PREFIX=\"lib\" -DSHLIB_VERSION=\"3\" -DSOFTOKEN_SHLIB_VERSION=\"3\" -DRIJNDAEL_INCLUDE_TABLES -UDEBUG -DNDEBUG -pthread -DNSS_NO_INIT_SUPPORT -DUSE_UTIL_DIRECTLY -DNO_NSPR_10_SUPPORT -DSSL_DISABLE_DEPRECATED_CIPHER_SUITE_NAMES -DNSS_X8    6_OR_X64 -DNSS_X86 -DMP_API_COMPATIBLE -I/usr/local/include/nspr -I../../../dist/OpenBSD6.1_OPT.OBJ/include -I../../../dist/public/nss -I../../../dist/private/nss -Im    pi -Iecl -I/usr/local/include -O2 -pipe   pqg.c
pqg.c:345:16: error: comparison of constant 18446744073709551615 with expression of type 'unsigned long' is always true [-Werror,-Wtautological-constant-out-of-rang    e-compare]
    if (addend < MP_DIGIT_MAX) {
        ~~~~~~ ^ ~~~~~~~~~~~~
1 error generated.
gmake[3]: *** [../../coreconf/rules.mk:393: OpenBSD6.1_OPT.OBJ/OpenBSD_SINGLE_SHLIB/pqg.o] Error 1

https://hg.mozilla.org/projects/nss/annotate/046956d1bf47/security/nss/lib/freebl/pqg.c#l163 maybe the limit isnt the right one ?

Comment 1

[Landry Breuil (:gaston)]

Ew, that code is here since 17 years...

Comment 2

[Landry Breuil (:gaston)]

Giving a quick look at the code, comparing an unsigned long with a macro that is defined depending on the various typedefs size is a sure way to fail. I see that addend is casted all around to mp_digit, so either addend should be an mp_digit, or it should be compared to ULONG_MAX. Which is the best, i have no idea.

Comment 3

[Landry Breuil (:gaston)]

I went for the latter and fixed it with ULONG_MAX within our portstree:

http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/security/nss/patches/patch-nss_lib_freebl_pqg_c?rev=1.1&content-type=text/x-cvsweb-markup

If that solution is acceptable i'll do a proper patch for commit here.

Comment 4

[Landry Breuil (:gaston)]

fixed by https://hg.mozilla.org/projects/nss/rev/0447b84b691db7222d33c38fd595f04e1e2655ef/duplicate of bug #1796308

nss 3.85 builds fine on i386 without my patch.