A certain Netscape user regularly receives encrypted SMIME email from his coworkers who are users of Lotus Notes. He can read these encrypted emails with no problems using Communicator 4.7x on Windows or Linux. But using newer Netscape and mozilla browsers, he cannot read these encrypted emails. Mozilla reports: "There are unknown problems with this encrypted message." (sigh.) When he attempts to read them using NSS's cmsutil test program, it reports: "cmsutil: problem decoding: security library: improperly formatted DER-encoded message." He compared encrypted messages generated by several SMIME email programs and found a crucial difference between the messages generated by Lotus Notes and those generated by other programs. The RecipientInfos SET in the Enveloped Data type is indefinite-length encoded by Notes and definite length encoded by others. He altered one message from Notes, changing the RecipientInfos SET to definite length encoding and then mozilla was able to read the message. He sent me both the original and altered copies for me to test with. I will attach them to this bug. RecipientInfos is permitted to be indefinite length encoded. The decoding failure in NSS's fault. It is a regression in libSMIME as compared to the old libPKCS7. I have been debugging his unaltered message using cmsutil in NSS 3.8 and have found several problems. The RecipientInfos SET contains a single RecipientInfo. The decoder decodes that correctly, using NSSCMSRecipientInfoTemplate, then returns to the parent state. It sees that it is parsing a SET, so it goes to parse the next NSSCMSRecipientInfoTemplate, and then discovers the EndOfContents bytes. But by that time, it thinks it's parsing the CHOICEs inside the NSSCMSRecipientInfoTemplate, and it concludes that it has found no item that matches any of those CHOICEs. I conclude that the EndOfContents bytes are being found at the wrong point in the decoding, but I'm not sure yet how to do it better. In the old PKCS7 library, the RecipientInfoTemplate was not a CHOICE. Rather, it was very similar to the NSSCMSKeyTransRecipientInfoTemplate. I suspect it is the new combination of a SET of a CHOICE that causes the problem.
I asked the reporter to detemine which Netscape/mozilla clients can, and which cannot, read encrypted emails from Lotus notes. Here is his answer: > Netscape 4.79 Linux success > Netscape 7.0 Linux fails > Netscape 7.0 Windows fails > Mozilla 1.0.1 Linux fails > Mozilla 1.3 Linux fails > NSS 3.4.1 Linux fails > Outlook Express 6.0 success > openssl 0.9.6 Linux success
This bug appears to now be fixed in NSS by revisions 1.20-1.22 of file nss/lib/util/secasn1d.c. Don't know how soon the mozilla browser will pick this up though. I'm tentatively marking this fixed. However, testing of the ASN.1 BER decoder continues.
Comment on attachment 121959 [details] [diff] [review] Patch for NSS_3_8_BRANCH Bob, could you review Nelson's fix for this bug? This patch is the diffs between rev. 1.21 and rev. 1.23 adapted for the NSS_3_8_BRANCH. (We can't apply the diffs from the trunk to the NSS_3_8_BRANCH directly because Nelson also made some white space changes on the trunk to fix the coding style.)
Comment on attachment 121959 [details] [diff] [review] Patch for NSS_3_8_BRANCH As much as I can understand give that the ASN1 decoder is black magic;).
Created attachment 122538 [details] [diff] [review] Updated patch to NSS 3.8 branch This patch incorporates the changes from revisions 1.22-1.24 on the trunk.
Wan-Teh, If you agree, I'll check this patch into the 3.8 branch.
Comment on attachment 122538 [details] [diff] [review] Updated patch to NSS 3.8 branch Nelson, you can check in this patch on the 3.8 branch. The only change I would suggest is to use the old 2-space indentation level in the "hunk" of your patch at -2187,6 +2205,26.
Comment on attachment 122538 [details] [diff] [review] Updated patch to NSS 3.8 branch Requesting review. I hope to get this in mozilla 1.4b
Comment on attachment 122538 [details] [diff] [review] Updated patch to NSS 3.8 branch Requesting approval for Moz 1.4b. Note, SR not needed for NSS code.
Comment on attachment 122538 [details] [diff] [review] Updated patch to NSS 3.8 branch a=sspitzer so you are just going to back port this fix from the nss trunk to the nss 3.8 branch?
Re comment 11, yes, that's exactly what patch attachment 122538 [details] [diff] [review] is, a back port from the trunk to the branch.
Moved NSS_CLIENT_TAG for nss/lib/util/secasn1d.c from rev 1.19 to rev 188.8.131.52 for Mozilla 1.4
Comment on attachment 122538 [details] [diff] [review] Updated patch to NSS 3.8 branch Nelson, could you check in this patch on the NSS_3_7_BRANCH?
Done. Patch applied to 3.7 branch.