Open Bug 95262 Opened 23 years ago Updated 2 years ago

OE 5 can't read sign/encrypt message created by smime perl script

Categories

(NSS :: Tools, defect, P3)

Product:
NSS
Component:
Tools
Version:
3.2.1
Type:
defect

Tracking

(Not tracked)

People

(Reporter: alam, Unassigned)

References

Details

Attachments

(3 files)

table showing test results
3.28 KB, text/html
Details
new perl smime script
19.23 KB, text/plain
Details
Modified test input file.
1.97 KB, text/plain
Details 
Build Used:  20010810 solaris 2.6
smime tool infor: http://mozilla.org/projects/security/pki/nss/smime/

In Outlook Express 5, It can read signed message as well as encrypted
message sent by the smime script, but NOT able to read a message with both
signature and encryption. It displayed the following error in the message pane: 

-----------------------------
Message could not be displayed
Outlook Express encountered an unexpected problem while displaying this message. 
Check your computer for low memory or low disk space and try again.
-----------------------------

(I am confirmed that there is no memory or disk space problem in my NT box)

In Netscape Messenger, all three cases (sign, encrypt, sign/encrypt) work fine
with the smime script.

I also tested sending a signed/encrypted message from Netscape Messenger to 
Outlook Express, and it just work fine. (OE can read the signed/encrypted 
message).
Antonio,
If you filed this bug after sending the two email messages that I saw today, I
can report that using Netscape 4.76 with PSM 1.4 installed, they came empty and
with an invalid signature.

Note that PSM 1.4 reports that the message has no digital signature because it
"does not include the sender's digital signature" and is encrypted, but can't be
decrypted (PSM 1.4 reports that the key is not in may db)  For the record, I am
able to send myself a encrypted and signed message from a win n4.76 no psm, and
read it with a 4.76 with psm on solaris.

Stephane,
I didn't use the smime script to send the message yesterday's morning regarding 
the "sectest".  Probably your case is different issue. thx.
Assigned the bug to Julien.
Assignee: ian.mcgreer → jpierre
Status: UNCONFIRMED → NEW
Ever confirmed: true
Priority: -- → P1
Target Milestone: --- → 3.4
Blocks: 74157
Julien, the s/mime scripts and toolkit is what we use for the browser. Thus it's
very likely that the browser will sign and encrypt in a way that will cause OE
to fail to process these emails.
When do you think you'll have time to look at this?
Stephane,

I will look into this sometime tomorrow.
Which version of Outlook Express is it exactly ? 5.0 ? 5.5 ?
I downloaded IE from Microsoft but it includes Outlook Express 6.


Antonio,

I also need more information about the test case you used. What exactly did you 
do to send the message using the S/MIME toolkit ?

Thanks.

I have reproduced the "out of memory or disk space" error on OE6. However, I 
can't get the output from the smime script to work, regardless of whether I 
sign, encrypt, or both, even with Communicator. The message never decrypts and 
the signature always fails to verify. This is with the current codebase in NSS 
3.4 . I am going to try with an older tree.

Julien,

I forgot to tell you that you should first debug
this with the stable NSS_3_3_BRANCH.
Wan-Teh,

I just tried on the 3.3 branch, but I'm still getting the same problems. I must 
be doing something wrong when sending the message. Antonio, you really need to 
tell me more about your testcase : exact command line you use with the script, 
content of your input message, cert used.

Julien, sorry for the late response, since I am busy with other testing.

Here is what I did:

Setup:  You need to have two certdb:  One for the sender and one for the 
recipient (eg. stest1 and stest2)

1. Use smime.pl to compose a S/MIME msg (eg. signed message):

#cat /u/alam/temp/mail.txt | perl smime.pl -S "smime test1's iplanet.com ID" -p 
"netscape" -d "/u/alam/Cert/smimetest1" > /tmp/smime.msg

where /u/alam/temp/mail.txt is an ordinary RFC822 message, and the directory 
/u/alam/Cert/smimetest1/ contain the cert db for user "smime test1"

2. Send the newly create S/MIME message to a recipient:

#cat /tmp/smime.msg |mail stest2@netscape.com

where stest2@netscape.com is the mail recipient
Note: I used Sendmail daemon from a solaris box to send the mail message.

3. Using OE 5.5, setup a mail account with stest2 certdb (needed this if you 
want to read encrypted message)

4. Try read the S/MIME message. You should get an error if the message is both 
sign/encrypted as I described above.
Using your mail.txt file, I was able to reproduce the problem.
Here are the cases I tested :

1) signed 
works in both communicator and OE 6

2) encrypted
works in both communicator and OE 6

3) encrypted and signed
works only in communicator
OE6 gets:
"Outlook Express encountered an unexpected problem while displaying this
message. Check your computer for low memory or low disk space and try again."

I think this is exactly what you saw, right ?

Yes.
This is exactly the problem I had.
I think the problems are related to the script and the header parsing and CR LF 
handling.

When I ran the S/MIME test from OS/2 as opposed to Solaris, I was able to send 
myself an e-mail that decrypted and verified successfuly in Outlook. But it 
was showing one of the headers in the body. The CRLF handling on OS/2 is 
different (with perl and piping). However the same encrypted/signed e-mail from 
my OS/2 smime client does not verify in Communicator, it only verifies in 
outlook. So this is sort of a reverse problem that you had.

I will write a more thorough S/MIME test in straight C that doesn't depend on 
external cat / perl piping and deals with headers better.
Another thing that would be useful would be to turn off the random number 
generator and make the resulting email completely predictable. This way it would 
be possible to compare the S/MIME output from one run to another and detect the 
CR/LF and header problems much faster. Is there a way to do that in NSS ?

Are we seeing a similar problem in the experimental browser builds with S/MIME 
or is it only with the smime script ?

Julien, I tried to read the S/MIME messages using the experimental build sent 
from the smime toolkit, but it seems that they can not be displayed correctly 
(either sign or encrypt).

S/MIME messages (sign/encrypt/both) sent from communicator 4.7x can be displayed 
in the experimental build with no problem.
Antonio,

What are you referring to when talking about "the experimental build sent 
from the smime toolkit" ?
Are you trying in a 6.2 browser with experimental support or NT ?
I'm trying to get the perl scripts out of the picture to see if the problem is 
in the library or the scripts. I am afraid at this point that there are 
problems in both, unfortunately.

I meant to ask if you tried running a 6.2 beta browser with experimental 
support for S/MIME in it.

Yes.
Forget about the perl scripts from the toolkit for a second.

If you send S/MIME encrypted/signed/encrypted+signed mail from the 6.2 
S/MIME build, can Outlook read them ?
Same question when sending from Outlook and reading in the 6.2 S/MIME build.


Curently we are not able to send S/MIME message from N6.2, but only receiving.
I will try to use OE to send S/MIME message to N6.2 to see what happen...
Thanks, Antonio, please let me know the results.

The reason I'm wary of the perl scripts is that I have gotten a series of 
inconsistent result depending on where I run them . I am piping the output of 
the smime encoding script back into the smime script with the decode option.

On Solaris, it works in all 3 cases - that is to say, the smime script verifies 
the signature on a signed message, decrypts an encrypted message, and both 
decrypts and verifies the signature on a signed/encrypted message. All 3 
e-mails are readable in Communicator. Only the first two e-mails are readable by 
outlook though, not the third one - as you observed.

On NT however, the decode script never works.

With a signed message, the following gets returned by the smime decode script :
cmsutil: failed to decode message.
cmsutil: problem decoding: security library: improperly formatted DER-encoded me
ssage.
ERROR: no content type header found in MIME entity

The signed email sent from the smime script on NT does not verify in 
Communicator, and also causes OE6 to display the out of memory error.

With an encrypted e-mail, the decode script runs cmsutil.exe which crashes in 
the ASN.1 decoder in secasn1d.c at line 1500 . Sending the encrypted email to OE 
causes it to display the encrypted email without a body but without any error 
... Communicator shows it with "invalid encryption".

With an encrypted/signed e-mail, the decode script runs cmsutil.exe which 
crashes in the ASN.1 decoder in secasn1d.c at line 1500 (same crash as for 
encrypted). Outlook shows it only as an encrypted message, but without error, 
and Communicator shows it with "invalid encryption".

I'm not going to post the OS/2 results since we don't know if that ever worked, 
but the tests show already that the code behaves very differently on Solaris 
than on NT. Basically 8 out of 9 cases work correctly on Solaris, and 0 out of 9 
cases work on NT.

With help from Christian, I was able to make some fixes into the perl script to 
deal with some of the CR/LF platform issues, using "binmode" after every 
perl call to open.

The results are now much more consistent accross platforms.
Namely, all emails decrypt and decode in Communicator regardless of the 
originating platform where the smime perl script was run. The signed & encrypted 
case however still doesn't work in Outlook and still produces the "out of 
memory" error as reported in this defect
. I think there might still be a cr lf issue remaining in the script when 
cmsutil is called twice for the signed & encrypted case.

I now have successfully sent S/MIME encoded & signed e-mails that can be read in 
both Communicator and Outlook 6.2 !!!

What I did was do two passes of the script - the first one to sign, and then 
taking the output of that, doing a second pass to encrypt, then e-mailing the 
result. Everything works when I do that in both mail clients. I can't use the 
smime script to test the decoding of that however, it really doesn't seem to 
like to do both steps in one shot, and doing two decode script steps also 
doesn't work, but that's again a perl script issue.

My conclusion is definitely a perl script issue, and should not happen in 
Communicator 6.2 with the C libraries. I will try to get the perl script fixed 
to handle all those cases correctly.


OK, so the plan of action for this fix is :
1) track down the ASN.1 crash when invalid data is being fed to it and fix it to 
make it more robust . This is necessary to get a robust browser with S/MIME
2) fix the CR LF issues in the S/MIME perl script so that signing, encrypting 
and signing + encrypting all work . This is working in my tree now with 2 
passes for the last case, but also needs to work in a single pass
3) fix the S/MIME perl script so that the single-pass verify/decrypt works. It 
must work whether the email was generated by single-pass or 2 passes of the 
smime script. Right now it doesn't work. This is another CR LF issue to deal 
with in the script.

I think the first two items would be a top priorities for now, since I will use 
the client browser (communicator, OE, and mozilla) to  decrypt/verify the S/MIME 
messages.

There is a problem when try to use Outlook Express to send S/MIME message to 
N6.2.  See bug 105474 for detail
Antonia,

Please try your test again using the new attached perl smime script. It should 
work for all cases of encode/decode on any platform now.
Attached file new perl smime script — 

    
    

    
  
Tried the new smime perl script from a solaris box, and now Outlook Express can
read the sign/encrypt message sent from the script.

However, the N6.2 S/MIME experimental build failed to decode all kind of S/MIME
messages (sign/encrypt/both) sent from the script.  

This is different issue and it may/may not related to the smime script, because
the experimental build also has problem reading S/MIME msg sent from Outlook
express (bug 105474). 

I have open a new bug 105774 for this.


    
    

    
  
Look like this bug is not completly fixed yet.

When I tried to both sign/encrypt a message that contain a MIME attachment (eg. 
a jpg file) using the toolkit, Outlook express complained that "The Message has 
been tampered with".  It worked fine in Communicator 4.7x though.

I ran the smime script in the solaris platform with the sendmail daemon.

You can find the input file that I used to test in: /u/alam/QA/jpg.in

    
    

    
  
Antonio,

I think your test file is invalid. There are some mail headers that span 
multiple lines, which is not allowed. Apparently you did a simple copy/paste 
from Communicator "view source". If you fix the mail headers in the message I 
think it will work better.


    
    

    
  
Which mail header you think has the problem?

I only see the Content-Type header has span to the next line:

Content-Type: multipart/mixed;
 boundary="------------A6BACAE8C10EFFC08EC85F4D"

But, I think it is fine.

Anyway, I tried to put it in one line and also remove some unnecessary header, 
but the problem still existed.

I will attached the modified input file.


    
    

    
  


  

    
    

    
  
Look at your content-type and content-disposition in the attachment. Please fix 
them to be on one line. I still think it might be causing a problem.
I can't test this right now because Outlook just hangs on me when trying to 
fetch mail.

    
    

    
  
Same problem with all headers in one line.

Content of updated input file:

----------------------------------------
From: stest1@red.iplanet.com
MIME-Version: 1.0
To: stest2@red.iplanet.com
Subject: Sign s/mime testing sample (with no span headers)
Content-Type: multipart/mixed; boundary="------------A6BACAE8C10EFFC08EC85F4D"

This is a multi-part message in MIME format.
--------------A6BACAE8C10EFFC08EC85F4D
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

JPG attached

--------------A6BACAE8C10EFFC08EC85F4D
Content-Type: image/jpeg; name="Ms.jpg"
Content-Transfer-Encoding: base64
Content-Disposition: inline; filename="Ms.jpg"

/9j/4AAQSkZJRgABAgIAAAAAAAD/4QAKUElDAAEyMgD/wAARCAAgACADAREAAhEBAxEB/9sA
hAAZERIVEg8ZFRQVHBoZHSU+KCUiIiVMNjktPlpPX11ZT1dVZHCPeWRqh2tVV32qfoeUmaCi
oGB4sL2vnLuPnaCaARocHCUgJUkoKEmaZ1dnmpqampqampqampqampqampqampqampqampqa
mpqampqampqampqampqampqampqampr/xAGiAAABBQEBAQEBAQAAAAAAAAAAAQIDBAUGBwgJ
CgsQAAIBAwMCBAMFBQQEAAABfQECAwAEEQUSITFBBhNRYQcicRQygZGhCCNCscEVUtHwJDNi
coIJChYXGBkaJSYnKCkqNDU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6
g4SFhoeIiYqSk5SVlpeYmZqio6Slpqeoqaqys7S1tre4ubrCw8TFxsfIycrS09TV1tfY2drh
4uPk5ebn6Onq8fLz9PX29/j5+gEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoLEQACAQIE
BAMEBwUEBAABAncAAQIDEQQFITEGEkFRB2FxEyIygQgUQpGhscEJIzNS8BVictEKFiQ04SXx
FxgZGiYnKCkqNTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqCg4SFhoeI
iYqSk5SVlpeYmZqio6Slpqeoqaqys7S1tre4ubrCw8TFxsfIycrS09TV1tfY2dri4+Tl5ufo
6ery8/T19vf4+fr/2gAMAwEAAhEDEQA/AOnkdYo2kdgqICzE9gKAONGoXbXkt7DM0LStu2dV
28AAjoeB1rGVS0j0qWCU6SbdmzWs/EseVj1BBC56SJyh/qK1i+ZaHHWoTou0jcjkSVA8bq6N
yGU5BpmJl60Wu2j0yJ9rSgvK2M7UHtkdTgfnWNeqqUOYqO5nXFjFYaXO1yiSSsxEcqrynGFy
ewyM/jjnv50Krq1FyOy6rudTqSnU572f5GHK8Uqlw5DAYUev4V7ND29CajBau33G2IlQrwc3
Lbb/AIYsaM15/aEUFlO8XmH58cgDucHjpXoY2lSgk0rNnlRbZu290La8uH1FXtprh8qZPubQ
DtUMCRwOvTk183jaVWbTSukbRaMi4iubeeT7ScTyKQ7Bgd6ng9O3pnHt04uEoSj7uy/A5ajn
Btt7mYqs0mzADdxjgV9BHEQw+Hi4O/z+8G0lc6jwpYeVbveyD55vlT2Uf4n+QrkrVnWnzM2S
sjekjSVCkiK6NwVYZBrEZiXvhtG+exl8lgMCN8snc4HcckmocEyJQUtzI/s67e9jsZoWiaU4
LdRtHUg9+P5iojTtIyjStK7OyjRYo1jRQqIAqgdgK2Og/9k=
--------------A6BACAE8C10EFFC08EC85F4D--


    
    

    
  
Antonio,

After uninstalling/reinstalling outlook and still having it hang, I finally 
found a way to work around the Outlook hang (for those who care : hit "cancel" 
when prompted for password. Wait ... Then click on the inbox, and then on 
"send/receive all" .. then type the password).

I reproduced your problem with the JPG attachment in signed/encrypted mode. OE 
does complain about tampering, but Communicator is fine with it. I also checked 
a simple signature on it and obtained the same results. Only encrypted without 
signature works OK with your attachment. Do you get the same thing ?


    
    

    
  
Yes.

    
    

    
  
Any update on this?

    
    

    
  
Not yet. I was out friday so I haven't been able to look at it, but will pursue
it further this week.

    
    

    
  
Antonio,

I still have not figured the cause of this failure with the attachment, but I'm
suspecting some more CR LF issues in the perl script. Can you try with the
browser  builds to send yourself signed/encrypt messages and check them in
Outlook ? That would at least isolate if the problem is in the library or perl
script.

Thanks.


    
    

    
  
Tried to used 2001-11-20 Windows comercial trunk build to send a sign/encrypt 
message with a jpg attachment, and it can be read back from Outlook Express 5.5

This seems to be a perl script problem again.

    
    

    
  
Lowering priority to P2 since it is only an issue in the script rather than the
library.

Priority: P1 → P2

    
    

    
  
Julien,
I agree this is a P2 bug, but fixing it will definitely speed up the S/MIME test 
effort. Do you have any estimate on how long you need to take to have this fix? 
Thx.

    
    

    
  
Antonio, I am not very knowledgeable about Perl and there is no debugger for it,
so it is taking me a very long time to fix issues in it.
I have considered in the past rewriting the script in C, and this is what I
might end up doing. But for now I must focus on higher priority bugs. I know how
important it is to have a working test tool but unfortunately it will take some
more time.



    
    

    
  
Just to confirm, this is definitely a problem only in the perl script, not with
the NSS S/MIME implementation. I just sent myself emails with attachments from
the current Mozilla on OS/2 no less - with signature, encryption, and both
encryption and signatures, and all decrypted/verified successfully both in
Outlook and Communicator.


    
    

    
  
Changed the QA contact to Bishakha.
QA Contact: sonja.mirtitsch → bishakhabanerjee

    
    

    
  
Set target milestone to NSS 3.5.
Target Milestone: 3.4 → 3.5

    
  
Target Milestone: 3.5 → 3.6

    
  
Target Milestone: 3.6 → Future

    
    

    
  
Filtering out all the platform-specific perl interpreter problems with CR LF
will require a rewrite of the perl script to C. The perl script essentailly
deals with the MIME headers, while the C tool cmsutil deals with the signature.
The MIME perl code should be rewritten and merged into the C program.

Summary: OE 5 can't read sign/encrypt message created by smime script → OE 5 can't read sign/encrypt message created by smime perl script

    
  
Hardware: Sun → All
QA Contact: bishakhabanerjee → jason.m.reid

    
  
OS: Solaris → All

    
  
Priority: P2 → P3
Target Milestone: Future → ---
QA Contact: jason.m.reid → tools

    
  
Assignee: julien.pierre.bugs → nobody

    
  
Severity: normal → S3

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: