I installed MS Visual Studio .NET 2003 from MSDN on my system and tried to build NSS. I got the following error : gmake: Leaving directory `C:/nss/39/mozilla/security/nss/cmd/shlibsign/mangl ' WIN954.0_DBG.OBJ/shlibsign -v -i ../../../../dist/WIN954.0_DBG.OBJ/lib/softokn3 dll Generating DSA Key Pair.... + Done(136) ? 2356 Floating point exception WIN954.0_DBG.OBJ/shlibsign gmake: *** [../../../../dist/WIN954.0_DBG.OBJ/lib/softokn3.chk] Error 136 gmake: Leaving directory `C:/nss/39/mozilla/security/nss/cmd/shlibsign' gmake: *** [libs] Error 2 gmake: Leaving directory `C:/nss/39/mozilla/security/nss/cmd' gmake: *** [libs] Error 2 The same tool works with the previous version of VC7 (2001).
Here is the stack : softokn3.dll!_s_mpv_div_2dx1d() Line 339 Asm softokn3.dll!mp_div(const mp_int * a=0x0012f934, const mp_int * b=0x0012f9d0, mp_int * q=0x00000000, mp_int * r=0x0012f934) Line 1004 + 0x11 C softokn3.dll!mp_mod(const mp_int * a=0x0012f934, const mp_int * m=0x0012f9d0, mp_int * c=0x0012f934) Line 1188 + 0x13 C > softokn3.dll!mpp_pprime(mp_int * a=0x0012f9d0, int nt=18) Line 331 + 0x11 C softokn3.dll!PQG_ParamGenSeedLen(unsigned int j=8, unsigned int seedBytes=128, PQGParamsStr * * pParams=0x0012fa38, PQGVerifyStr * * pVfy=0x0012fa3c) Line 474 + 0xb C softokn3.dll!PQG_ParamGen(unsigned int j=8, PQGParamsStr * * pParams=0x0012fa38, PQGVerifyStr * * pVfy=0x0012fa3c) Line 366 + 0x15 C softokn3.dll!nsc_parameter_gen(unsigned long key_type=1, PK11ObjectStr * key=0x00547f70) Line 2524 + 0x11 C softokn3.dll!NSC_GenerateKey(unsigned long hSession=1, CK_MECHANISM * pMechanism=0x0012fbbc, CK_ATTRIBUTE * pTemplate=0x0012fbd8, unsigned long ulCount=1, unsigned long * phKey=0x0012fbd4) Line 2891 + 0x10 C nss3.dll!PK11_PQG_ParamGenSeedLen(unsigned int j=8, unsigned int seedBytes=0, PQGParamsStr * * pParams=0x0012fd00, PQGVerifyStr * * pVfy=0x0012fd20) Line 105 + 0x31 C nss3.dll!PK11_PQG_ParamGen(unsigned int j=8, PQGParamsStr * * pParams=0x0012fd00, PQGVerifyStr * * pVfy=0x0012fd20) Line 187 + 0x13 C shlibsign.exe!main(int argc=4, char * * argv=0x00532dc0) Line 232 + 0x1a C shlibsign.exe!mainCRTStartup() Line 398 + 0xe C KERNEL32.DLL!77ea847c() The failure is in the assembly code in mpi_x86.asm , on a "div ebx". Perhaps the calling conventions changed in VC++ 2003 ?
I seem to recall that the floating point exception and the divide-by-zero exceptions are one and the same. Can you tell by looking at the caller whether it passed in a zero divisor or not? I'll be happy to come by and take a look if you want.
Had Nelson look at it. The problem was that mp_digit was considered to be a 64-bit integer rather than 32-bit using VC++ 2003 . Forcing it to 32-bits using a define worked and allowed the signing to pass, as well as all.sh .
Created attachment 128190 [details] [diff] [review] Patch to make mp_digit 32 bits with MS VC++ .net 2003
Comment on attachment 128190 [details] [diff] [review] Patch to make mp_digit 32 bits with MS VC++ .net 2003 Julien, to review this patch I'd like to know why mp_digit becomes 64-bit in VC++ 2003. Looking at mpi.h, I think this is because VC++ 2003 adds the support of the long long type. Could you verify this? mpi.h has three possible typedefs for mp_digit. Could you find out which typedef is used if this patch is not applied? I guess it's the second one: typedef unsigned long long mp_digit; and it is because one of the following macros is defined in VC++ 2003's header file <limits.h>: ULONG_LONG_MAX ULLONG_MAX ULONGLONG_MAX Thanks.
Yes, indeed, without the patch, mp_digit is defined as typedef unsigned long long mp_digit; This is caused by ULLONG_MAX being defined in limits.h : C:\MSVSNET2003\Vc7\include>grep -i ullong_max *.* limits.h:#define ULLONG_MAX 0xffffffffffffffff /*maximum unsigned __int64 value */
Comment on attachment 128190 [details] [diff] [review] Patch to make mp_digit 32 bits with MS VC++ .net 2003 Thank you, Julien. r=wtc.
Comment on attachment 128190 [details] [diff] [review] Patch to make mp_digit 32 bits with MS VC++ .net 2003 In addition to defining -DMP_USE_UINT_DIGIT we also need to define -DMP_NO_MP_WORD With both of those symbols defined, the same code is generated on both old and new MSVC compilers.
Created attachment 128201 [details] [diff] [review] update
Comment on attachment 128201 [details] [diff] [review] update sr=nelsonb
Checking in Makefile; /cvsroot/mozilla/security/nss/lib/freebl/Makefile,v <-- Makefile new revision: 1.49; previous revision: 1.48 done
Can a NSS person get this fix on the NSS_CLIENT_TAG?
Stephen, This bug was fixed on the tip of NSS - the yet unreleased NSS version 3.9. I have updated the target milestone to reflect this. The NSS_CLIENT_TAG is based on NSS version 3.8 . To get this patch into version 3.8 : - please open a separate bug with a target milestone of NSS 3.8 - attach the patch - cc me - send email to email@example.com for approval about the new bug, since the Mozilla trunk is currently closed Closing this bug.