Current lib/pk11wrap/secmodt.h defines CKM_INVALID_MECHANISM as a signed long constant: #define CKM_INVALID_MECHANISM 0xffffffffL This causes compiler warnings because CK_MECHANISM_TYPE is an unsigned long (CK_ULONG). Other macros that should be defined as unsigned long constants include: CKM_FAKE_RANDOM CKM_NETSCAPE_PBE_SHA1_DES_CBC
Created attachment 271786 [details] [diff] [review] Fix types for mechanism constants
Comment on attachment 271786 [details] [diff] [review] Fix types for mechanism constants r=nelson, provided you make one additional change. You want to change ALL the CKM_ values to have a UL suffix, right? You missed one. Please change it, too. >+#define CKM_NETSCAPE_PBE_MD5_HMAC_KEY_GEN 0x8000000aUL >+#define CKM_NETSCAPE_PBE_MD2_HMAC_KEY_GEN 0x8000000bUL > > #define CKM_TLS_PRF_GENERAL 0x80000373L That's the missed one.
Nelson, Thanks for the review. Yes, I intended to change them all. I actually looked at the CKM_TLS_PRF_GENERAL definition, but somehow forgot to include that change my patch. I checked this patch in on the trunk (with the change) : Checking in pk11wrap/secmodt.h; /cvsroot/mozilla/security/nss/lib/pk11wrap/secmodt.h,v <-- secmodt.h new revision: 1.33; previous revision: 1.32 done Checking in softoken/pkcs11n.h; /cvsroot/mozilla/security/nss/lib/softoken/pkcs11n.h,v <-- pkcs11n.h new revision: 1.17; previous revision: 1.16 done
We could also use no suffix, as is done in pkcs11t.h for the standard CKM_xxx macros: http://lxr.mozilla.org/mozilla1.8/source/security/nss/lib/softoken/pkcs11t.h#613 I remember hex constants are by default unsigned.