Beginning on October 25th, 2016, Persona will no longer be an option for authentication on BMO. For more details see Persona Deprecated.
Last Comment Bug 202384 - mozilla can't read encrypted mail from Lotus Notes
: mozilla can't read encrypted mail from Lotus Notes
Product: NSS
Classification: Components
Component: Libraries (show other bugs)
: 3.8
: All All
: P1 normal (vote)
: 3.8.1
Assigned To: Wan-Teh Chang
: Bishakha Banerjee
Depends on:
  Show dependency treegraph
Reported: 2003-04-17 02:30 PDT by Nelson Bolyard (seldom reads bugmail)
Modified: 2003-05-14 10:29 PDT (History)
2 users (show)
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---

Patch for NSS_3_8_BRANCH (2.73 KB, patch)
2003-04-28 16:56 PDT, Wan-Teh Chang
rrelyea: review+
Details | Diff | Splinter Review
Updated patch to NSS 3.8 branch (2.97 KB, patch)
2003-05-05 20:44 PDT, Nelson Bolyard (seldom reads bugmail)
julien.pierre: review+
sspitzer: approval1.4+
Details | Diff | Splinter Review

Description Nelson Bolyard (seldom reads bugmail) 2003-04-17 02:30:24 PDT
A certain Netscape user regularly receives encrypted SMIME email from 
his coworkers who are users of Lotus Notes.  He can read these encrypted
emails with no problems using Communicator 4.7x on Windows or Linux.
But using newer Netscape and mozilla browsers, he cannot read these 
encrypted emails.  Mozilla reports: 
"There are unknown problems with this encrypted message."  (sigh.)

When he attempts to read them using NSS's cmsutil test program, it reports:  
"cmsutil: problem decoding: security library: improperly formatted
DER-encoded message."

He compared encrypted messages generated by several SMIME email programs
and found a crucial difference between the messages generated by Lotus
Notes and those generated by other programs.  The RecipientInfos SET 
in the Enveloped Data type is indefinite-length encoded by Notes and 
definite length encoded by others.  He altered one message from Notes,
changing the RecipientInfos SET to definite length encoding and then 
mozilla was able to read the message.  He sent me both the original and
altered copies for me to test with.  I will attach them to this bug.

RecipientInfos is permitted to be indefinite length encoded.  
The decoding failure in NSS's fault.  It is a regression in libSMIME 
as compared to the old libPKCS7.  

I have been debugging his unaltered message using cmsutil in NSS 3.8 
and have found several problems.  
The RecipientInfos SET contains a single RecipientInfo.  The decoder
decodes that correctly, using NSSCMSRecipientInfoTemplate, then 
returns to the parent state.  It sees that it is parsing a SET, so
it goes to parse the next NSSCMSRecipientInfoTemplate, and then 
discovers the EndOfContents bytes.  But by that time, it thinks it's
parsing the CHOICEs inside the NSSCMSRecipientInfoTemplate, and it 
concludes that it has found no item that matches any of those CHOICEs.
I conclude that the EndOfContents bytes are being found at the wrong 
point in the decoding, but I'm not sure yet how to do it better.

In the old PKCS7 library, the RecipientInfoTemplate was not a CHOICE.
Rather, it was very similar to the NSSCMSKeyTransRecipientInfoTemplate.
I suspect it is the new combination of a SET of a CHOICE that causes 
the problem.
Comment 1 Nelson Bolyard (seldom reads bugmail) 2003-04-18 11:53:13 PDT
I asked the reporter to detemine which Netscape/mozilla clients can, and 
which cannot, read encrypted emails from Lotus notes.  Here is his answer:

> Netscape 4.79 Linux               success
> Netscape 7.0 Linux                fails
> Netscape 7.0 Windows              fails
> Mozilla 1.0.1 Linux               fails
> Mozilla 1.3 Linux                 fails
> NSS 3.4.1 Linux                   fails
> Outlook Express 6.0               success
> openssl 0.9.6 Linux               success
Comment 2 Nelson Bolyard (seldom reads bugmail) 2003-04-25 21:51:09 PDT
This bug appears to now be fixed in NSS by revisions 1.20-1.22 of file
Don't know how soon the mozilla browser will pick this up though.

I'm tentatively marking this fixed.  
However, testing of the ASN.1 BER decoder continues.
Comment 3 Wan-Teh Chang 2003-04-28 16:56:41 PDT
Created attachment 121959 [details] [diff] [review]
Patch for NSS_3_8_BRANCH
Comment 4 Wan-Teh Chang 2003-04-28 17:00:20 PDT
Comment on attachment 121959 [details] [diff] [review]
Patch for NSS_3_8_BRANCH

Bob, could you review Nelson's fix for this bug?

This patch is the diffs between rev. 1.21 and rev. 1.23
adapted for the NSS_3_8_BRANCH.  (We can't apply the diffs
from the trunk to the NSS_3_8_BRANCH directly because
Nelson also made some white space changes on the trunk to
fix the coding style.)
Comment 5 Robert Relyea 2003-04-28 18:17:10 PDT
Comment on attachment 121959 [details] [diff] [review]
Patch for NSS_3_8_BRANCH

As much as I can understand give that the ASN1 decoder is black magic;).
Comment 6 Nelson Bolyard (seldom reads bugmail) 2003-05-05 20:44:13 PDT
Created attachment 122538 [details] [diff] [review]
Updated patch to NSS 3.8 branch

This patch incorporates the changes from revisions 1.22-1.24 on the trunk.
Comment 7 Nelson Bolyard (seldom reads bugmail) 2003-05-05 20:45:32 PDT
Wan-Teh,  If you agree, I'll check this patch into the 3.8 branch.
Comment 8 Wan-Teh Chang 2003-05-06 17:38:15 PDT
Comment on attachment 122538 [details] [diff] [review]
Updated patch to NSS 3.8 branch

Nelson, you can check in this patch on the 3.8 branch.
The only change I would suggest is to use the old 2-space
indentation level in the "hunk" of your patch at
-2187,6 +2205,26.
Comment 9 Nelson Bolyard (seldom reads bugmail) 2003-05-06 19:25:50 PDT
Comment on attachment 122538 [details] [diff] [review]
Updated patch to NSS 3.8 branch

Requesting review.  I hope to get this in mozilla 1.4b
Comment 10 Nelson Bolyard (seldom reads bugmail) 2003-05-07 22:02:15 PDT
Comment on attachment 122538 [details] [diff] [review]
Updated patch to NSS 3.8 branch

Requesting approval for Moz 1.4b.  Note, SR not needed for NSS code.
Comment 11 (not reading, please use instead) 2003-05-08 06:57:05 PDT
Comment on attachment 122538 [details] [diff] [review]
Updated patch to NSS 3.8 branch


so you are just going to back port this fix from the nss trunk to the nss 3.8
Comment 12 Nelson Bolyard (seldom reads bugmail) 2003-05-08 13:29:40 PDT
Re comment 11, yes, that's exactly what patch attachment 122538 [details] [diff] [review] is, 
a back port from the trunk to the branch.
Comment 13 Nelson Bolyard (seldom reads bugmail) 2003-05-12 20:00:30 PDT
Moved NSS_CLIENT_TAG for nss/lib/util/secasn1d.c from rev 1.19 to rev
for Mozilla 1.4
Comment 14 Wan-Teh Chang 2003-05-13 18:59:08 PDT
Comment on attachment 122538 [details] [diff] [review]
Updated patch to NSS 3.8 branch

Nelson, could you check in this patch on the NSS_3_7_BRANCH?
Comment 15 Nelson Bolyard (seldom reads bugmail) 2003-05-13 21:53:47 PDT
Done.  Patch applied to 3.7 branch.

Note You need to log in before you can comment on or make changes to this bug.