400404 - JSS and NSS use different salt sizes for PBEAlgorithms

Status: RESOLVED WONTFIX

(JSS Graveyard :: Library, defect, P2)

Description

[David Stutzman]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.8) Gecko/20071008 Firefox/2.0.0.8
Build Identifier: 4.2.5

CryptoStore.getEncryptedPrivateKeyInfo extracts an EPKI structure from the keydb using the PBEAlgorithm you pass in through a native call to NSS.  Of the 4 PBEAlgorithms that I can extract the key with (PBE_MD2_DES_CBC, PBE_MD5_DES_CBC, PBE_SHA1_DES_CBC, PBE_SHA1_DES3_CBC) the resulting EPKI has a salt size of 16 bytes.  Calling getSaltLength on the 4 algorithms report the following salt sizes:
PBE_MD2_DES_CBC - 8 bytes
PBE_MD5_DES_CBC - 8 bytes
PBE_SHA1_DES_CBC - 8 bytes
PBE_SHA1_DES3_CBC - 20 bytes

I'll put in the attachments a file of dumpasn1 output on each of the EPKI structures showing the salt of 16 bytes for each and in the filename the size returned by getSaltLength

Reproducible: Always

Steps to Reproduce:
1. Export EPKI using getEncryptedPrivateKeyInfo with various PBEAlgorithms
2. write bytes to file, inspect with asn1 decoder
3. compare values of PBEAlgorithm.getSaltLength to size in EPKI structure
Actual Results:  
salt of 16 bytes for all 4 PBEAlgorithms tested

Expected Results:  
appropriate salt size (either 8 or 20 bytes) as returned by PBEAlgorithm.getSaltLength

I'm using NSS 3.11.7, NSPR 4.6.7 and JSS 4.2.5

Comment 1

[David Stutzman]

Attachment: asn1 decoding of each of the EPKIs

Comment 2

[David Stutzman]

Attachment: the 4 EPKIs with 4 different algorithms

the salt-size-should-be-# was obtained with PBEAlgorithm.getSaltLength()

Comment 3

[David Stutzman]

If you want PKCS12s I can give you those either with decrypting the key and having it reencrypted via createEncryptedPrivateKeyBag which will result in the key being encrypted using PBE_SHA1_DES3_CBC and a salt of 20 which works fine with the other toolkits or I can not decrypt the key and just insert the EPKI that getEncryptedPrivateKeyInfo returns directly into a SafeBag and then most toolkits won't be able to read the PKCS12 files at all.

Comment 4

[David Stutzman]

I mentioned I only used 4 of the possible PBEAlgorithms, here are the issues I
have with the other 4:

PBEAlgorithm.PBE_SHA1_RC2_128_CBC:
getEncryptedPrivateKeyInfo dumps the Java VM:
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0378fedd, pid=5428, tid=5072
#
# Java VM: Java HotSpot(TM) Client VM (1.6.0_02-b06 mixed mode, sharing)
# Problematic frame:
# C  [nss3.dll+0x6fedd]

PBE_SHA1_RC2_40_CBC:
I get the EPKI out, trying to decrypt it results in the following exception:
java.security.InvalidAlgorithmParameterException: RC2/CBC/NoPadding cannot use
a null parameter
        at org.mozilla.jss.pkcs11.PK11Cipher.checkParams(PK11Cipher.java:253)
        at org.mozilla.jss.pkcs11.PK11Cipher.initDecrypt(PK11Cipher.java:139)
        at
org.mozilla.jss.pkix.primitive.EncryptedPrivateKeyInfo.decrypt(EncryptedPrivateKeyInfo.java:247)

Here's an ASN1 dump of the key (again with 16 bytes salt):
   0  682: SEQUENCE {
   4   36:   SEQUENCE {
   6   10:     OBJECT IDENTIFIER
         :       pbeWithSHAAnd40BitRC2-CBC (1 2 840 113549 1 12 1 6)
  18   22:     SEQUENCE {
  20   16:       OCTET STRING 09 C9 0F 12 75 6A 9B 02 75 4F D6 93 59 2F F5 5D
  38    2:       INTEGER 2048
         :       }
         :     }
  42  640:   OCTET STRING
         :     13 10 D0 CC 99 E9 0D 50 1E 4D A3 56 2B 4D E3 98
         :     65 82 DA 8E 8F EB 5A 11 9C E0 CC 8B 03 BA 4D EE
         :     7E 2C 33 6A 15 96 DC EF 42 8B E8 E2 9C DE A4 16
         :     81 EF DD 9B 0E 95 BE 1C A8 AA 16 D3 1E 05 60 9C
         :     C4 D1 8C 1C D6 83 61 8C 71 33 17 4A EE EB 0D 33
         :     9F BD DC 55 37 51 33 9A E1 FA C7 CC 2C 40 35 F2
         :     EA FB 9A 35 A6 9D 4E 64 27 BC EA 9F 6C 43 11 00
         :     7C B3 4D AD 43 40 BF F6 7E 99 6B A5 F3 DC 73 7F
         :             [ Another 512 bytes skipped ]
         :   }

PBE_SHA1_RC4_128 and PBE_SHA1_RC4_40:
I get the EPKI out, trying to decrypt it results in the following exception:
java.lang.NullPointerException
        at
org.mozilla.jss.pkix.primitive.EncryptedPrivateKeyInfo.decrypt(EncryptedPrivateKeyInfo.java:239)

Here's an ASN1 dump of both keys (again with 16 bytes salt):
   0  677: SEQUENCE {
   4   36:   SEQUENCE {
   6   10:     OBJECT IDENTIFIER
         :       pbeWithSHAAnd128BitRC4 (1 2 840 113549 1 12 1 1)
  18   22:     SEQUENCE {
  20   16:       OCTET STRING 66 3F 8C 75 4C D4 FD 6C C3 FA 60 EE 7A 4F 13 5D
  38    2:       INTEGER 2048
         :       }
         :     }
  42  635:   OCTET STRING
         :     6E 19 80 D8 A6 EC 3A 15 3B 0D 2B DB D6 51 CA 53
         :     37 EC BE 49 E3 1D A6 7C DE F7 E6 AB 9B 09 F7 F7
         :     A4 76 E7 F9 D2 2A 9B 4F 0E 57 86 16 ED CC 22 BB
         :     C4 2E 7E 07 1A F6 56 8F C0 A0 31 C1 1D F3 E7 DE
         :     E5 E0 E3 AB B9 5C 2D EE BB E5 79 B8 E6 FE 97 CB
         :     D8 EB EA AE 21 A7 B9 22 19 4E F3 A6 78 87 8B CD
         :     50 BE 2D 1C D7 E2 2F DD 6F C5 77 92 F7 2E DC 3E
         :     07 08 7B 45 E7 82 90 21 8B 8C C4 DD 36 7D 73 E7
         :             [ Another 507 bytes skipped ]
         :   }

   0  676: SEQUENCE {
   4   36:   SEQUENCE {
   6   10:     OBJECT IDENTIFIER
         :       pbeWithSHAAnd40BitRC4 (1 2 840 113549 1 12 1 2)
  18   22:     SEQUENCE {
  20   16:       OCTET STRING 3A A9 1F 8E 0A 3F B1 56 54 BE A2 D3 D1 DB 0D E5
  38    2:       INTEGER 2048
         :       }
         :     }
  42  634:   OCTET STRING
         :     E0 38 24 C8 8F 47 1B 17 52 4E B5 A8 26 E9 7D 72
         :     A5 21 6F 67 18 37 FB 29 25 59 9A C2 CB 5E AF 8D
         :     73 F8 DB 45 61 82 DF A8 BD 09 7E 90 2F 60 AA 17
         :     78 51 11 6A 1E 7D 32 F0 AE A7 E8 1F 2A 49 E1 9D
         :     66 0B 72 18 27 67 DF 66 FE 5E 36 CB 2C 7B 54 77
         :     D2 9D F5 79 E7 0E 27 5B 1C DC 4B 12 2F FB F2 C4
         :     E3 D3 E2 5A 28 97 F6 18 86 0E 96 82 D1 2F D7 D6
         :     80 DD 66 10 2E 8B 63 DE B2 B7 44 EE FA 9E 44 26
         :             [ Another 506 bytes skipped ]
         :   }

Comment 5

[David Stutzman]

I did some investigation. There's a #define here that sets the salt length to 16:
http://mxr.mozilla.org/security/source/security/nss/lib/pk11wrap/pk11pbe.c#272

I followed the calls from JSS through to NSS and on this codepath, NULL is passed in for the salt param of sec_pkcs5_create_pbe_parameter so it goes through the else here:
http://mxr.mozilla.org/security/source/security/nss/lib/pk11wrap/pk11pbe.c#303
and sets a random salt of size DEFAULT_SALT_LENGTH.

Unfortunately, changing this hardcoded 16 to a hardcoded 20 didn't help out my decrypting the EPKI with either Java or OpenSSL.  Both complain about the padding of the final block.  I'm not sure where to go from here.

And this is probably just an NSS issue now and not JSS.

Comment 6

[BMO Automation]

JSS development has moved from the Mozilla community to the Dogtag PKI community. Please re-file this bug at https://github.com/dogtagpki/jss if it is still relevant. Thank you!