192007 - Content-Type header line which is > 78, S/MIME Structure rejected by MessageWall

Status: RESOLVED WORKSFORME

(MailNews Core :: Security: S/MIME, defect)

Description

[Wilmack Sanchez]

User-Agent:       Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows XP) Opera 7.0  [en]
Build Identifier: Mozilla/5.0  (Windows; U; WindowsNT 5.1; en-US; rv:1.0.2) Gecko/20021120 Netscape/7.01

 Our company has chosen NS 7.0 as his mail program standard. surprisingly  we 
have had a problem when we sent mail to one of our partners. They have a program 
(www.messagewall.org) which review the structure of the mail searching virus, 
worms, etc.  When we sign digitally our messages they rejected it based on a 
supposed problem with the structure of the boundary. The problem doesn't exist 
if we send the message using Outlook express but it also appears if we use NS 
4.5-4.7. It seems that the problem  is that NS use a structure like this:




boundary=------------ms010905060608070601030006;




and Outlook use a structure like this




boundary="----=_NextPart_000_000F_01C2CCF2.931A6E10";




MessageWall said that NS is violating the S/MIME standard, so that can't do 
anything in his program to avoid this problem.




Could you help me?  I need to confirm if this is true and I need some kind of 
patch or If it isn't true and they have to modify his program.


Reproducible: Always

Steps to Reproduce:
1.Get a Personal Certificate


2. Sign your messages to a company which has MessageWall


3. You'll receive a mail rejecting your former message.

Actual Results:  
This is the mail you received:




Received: from [192.168.110.1] by certicamara.com (MessageWall 1.0.8)


 with SMTP; Tue, 04 Feb 2003 20:29:41 +0000


Date: Tue, 04 Feb 2003 20:29:42 +0000 (GMT)


From: [e-mail]


Subject: Undeliverable mail


To: undisclosed-recipients: ;


Message-id: <[e-mail]>


MIME-version: 1.0


Content-type: multipart/mixed; boundary="Boundary_(ID_UZvA0hgoK2b3+9NOF45YAg)"


X-MessageWall-Score: 0 (certicamara.com)






--Boundary_(ID_UZvA0hgoK2b3+9NOF45YAg)


Content-type: text/plain; charset=us-ascii


Content-transfer-encoding: 7BIT




Attached is your mail message which could not be delivered, or may have 
encountered some trouble in the delivery process.


mail could not be delivered to all of its recipients






--Boundary_(ID_UZvA0hgoK2b3+9NOF45YAg)


Content-type: message/rfc822




Received: from  ([200.75.44.205]) by fw.certicamara.com; Tue,


 04 Feb 2003 15:17:17 -0500 (COT)


Received: from etb.com.co ([172.18.53.81])


 by etb-web.etb.com.co (iPlanet Messaging Server 5.1 HotFix 1.9 (built Dec  3


 2002)) with ESMTPA id <[e-mail]> for


 [e-mail]; Tue, 04 Feb 2003 15:26:51 -0500 (GMT)


Date: Tue, 04 Feb 2003 15:21:29 -0500


From: wilmsanc <[e-mail]>


Subject: firmado con attachemet


To: camoa <[e-mail]>


Message-id: <[e-mail]>


MIME-version: 1.0


Content-type: multipart/signed; protocol="application/x-pkcs7-signature";


 micalg=sha1; boundary=------------ms000407030103090904070603


X-Accept-Language: en-us, en


User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2)


 Gecko/20021120 Netscape/7.01




This is a cryptographically signed message in MIME format.




--------------ms000407030103090904070603


Content-Type: multipart/mixed;


 boundary="------------060109070402070303000605"


Expected Results:  
It seems that mozilla should usea tructru  like boundary="---- "; and not 
boundary=-----;

Comment 1

[Mitchell Stoltz (not reading bugmail)]

->PSM S/MIME

Comment 2

[Kai Engert [:KaiE:]]

JF, what's your opinion, are our MIME multipart boundaries invalid?

Comment 3

[Jean-Francois Ducarroz]

I review RFC2046 (section 5.1.1) and I don't think so Mozilla bondary are in
violation of the RFC as the "-" characters is not a gramatical element which can
impact mime headers. But like said the RFC:

"WARNING TO IMPLEMENTORS:  The grammar for parameters on the Content-
   type field is such that it is often necessary to enclose the boundary
   parameter values in quotes on the Content-type line.  This is not
   always necessary, but never hurts. Implementors should be sure to
   study the grammar carefully in order to avoid producing invalid
   Content-type fields."

we could put the boundary between double quote to prevent any confusion...

Comment 4

[(not reading, please use [e-mail] instead)]

confirming based on ducarroz's comments.

The places in the code where we generate the boundary are:

C:\trees\trunk\mozilla\mailnews\compose\src\nsMsgCompUtils.cpp(746):*mime_make_separator(const
char *prefix)
C:\trees\trunk\mozilla\mailnews\extensions\smime\src\nsMsgComposeSecure.cpp(170):*mime_make_separator(const
char *prefix)

Comment 5

[(not reading, please use [e-mail] instead)]

I was going to whip up a patch, but it looks like there is code that should do
the right thing.

I need to get in there and debug.

I'll continue to investigate.

Comment 6

[(not reading, please use [e-mail] instead)]

The trunk actually appears to do the right thing.

we are quoting the boundary.

I'll attach a sample email.

Comment 7

[(not reading, please use [e-mail] instead)]

Attachment: signed email

Comment 8

[(not reading, please use [e-mail] instead)]

Wilmack Sanchez, can you try with a recent trunk build?

Comment 9

[Wilmack Sanchez]

I tried with Mozilla 1.3b, but it doesn't work. I'll attach the file

Comment 10

[Wilmack Sanchez]

Attachment: mail message

Comment 11

[(not reading, please use [e-mail] instead)]

notice that in Wilmack's message he has the following in his Content-type header:

boundary=------------ms010406080207000108040405; micalg=sha1;

where I have:

micalg=sha1; boundary="------------ms080707000607070105020406"

I'm not sure why his are in a different order

also, he has "Content-type:" and I have "Content-Type:"

the code that generates this is 
http://lxr.mozilla.org/mozilla/source/mailnews/extensions/smime/src/nsMsgComposeSecure.cpp#1051

I'm trying to see if there is some second code path?

Comment 12

[(not reading, please use [e-mail] instead)]

I'm not finding that "Content-type" in the source, except in some import (from
outlook, eudora, etc) code.

is it possible that your SMTP server (or MTA) is altering your headers?

my headers:

Return-Path: <[e-mail]>
Received: from netscape.com ([10.169.192.107]) by judge.mcom.com
          (Netscape Messaging Server 4.15) with ESMTP id HABMZ101.84P for
          <[e-mail]>; Fri, 14 Feb 2003 15:09:01 -0800 
Message-ID: <[e-mail]>
Date: Fri, 14 Feb 2003 15:07:45 -0800
From: [e-mail] (Seth Spitzer)
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3b) Gecko/20030214
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: [e-mail]
Subject: test
Content-Type: multipart/signed; protocol="application/x-pkcs7-signature";
micalg=sha1; boundary="------------ms050701030800070609030008"

his:

Date: Fri, 14 Feb 2003 15:15:48 -0500
From: wilmsanc <[e-mail]>
Subject: Boundary test
To: wilmsanc <[e-mail]>
Message-id: <[e-mail]>
MIME-version: 1.0
Content-type: multipart/signed;
 boundary=------------ms010406080207000108040405; micalg=sha1;
 protocol="application/x-pkcs7-signature"
X-Accept-Language: en-us, en
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.3b)
 Gecko/20030214

can you do a view source of a signed message from your Sent folder, and see if
the headers look the same as they do when you get the message in your inbox?

the only difference I see when I do that is that the message in my sent folder
doesn't have the "Return-Path:" and the "Received:" headers

Comment 13

[Jean-Francois Ducarroz]

sure, the following is illegal and will cause problem:
  boundary=------------ms010406080207000108040405; micalg=sha1;

to be valid, it should have been:
  boundary="------------ms010406080207000108040405"; micalg=sha1;

You should look at the source of the message saved in your sent folder. This is
exaclty what we sent to the smtp server. If it differs from the source of the
message received, somebody between the SMTP server and the POP/IMAP server is
changing the message!

Comment 14

[Wilmack Sanchez]

Attachment: This is the mail saved directly from mozilla

Yes. The original mail has the right format

Comment 15

[Wilmack Sanchez]

Attachment: This is the mail I received.

Now It seems that may problem is related with my iPlanet Messaging Server 5.1
HotFix 1.9 (built Dec  3 2002).

Comment 16

[(not reading, please use [e-mail] instead)]

john might know about this issue where the Content-Type header is getting
altered by iPlanet Messaging Server 5.1

Comment 17

[John G. Myers]

I respectfully disagree with comment #13 and the claim of MessageWall.  The
unquoted boundary is legal per the ABNF in RFC 2045 section 5.1.

I have no knowledge of that particular hotfix, but I will mention this issue to
contacts in Sun.

Comment 18

[Chris Newman]

I suspect the problem is a combination of a minor bug in Mozilla with a major
bug in MessageWall.

It appears Mozilla is generating a Content-Type header line which is > 78
characters long.  This is not advisable according to the standards:
   http://www.apps.ietf.org/rfc/rfc2822.html#sec-2.1.1

The iPlanet Messaging Server MTA reformats the Content-Type header so it is
standards compliant and minimally quoted.  The boundary marker which Outlook
generates contains an "=" which requires double-quotes.  The boundary marker
generated by Mozilla does not require double-quotes, so they are removed.

Then MessageWall gets a 100% standards compliant message (with unquoted boundary
marker parameter) and rejects it.  They need to fix their software.

Here's the relevant part of the MIME spec (RFC 2045) ABNF which shows when
double-quotes are unnecessary:

  parameter := attribute "=" value

  value := token / quoted-string

  token := 1*<any (US-ASCII) CHAR except SPACE, CTLs,
              or tspecials>

  tspecials :=  "(" / ")" / "<" / ">" / "@" /
                "," / ";" / ":" / "\" / <">
                "/" / "[" / "]" / "?" / "="
                ; Must be in quoted-string,
                ; to use within parameter values

Comment 19

[Wilmack Sanchez]

Attachment: This is the mail I received using Outlook Express 6.0 and the same messaging server

I was thinking and if the problem is IPlanet, why I don't have problems if I
sent my mails using Outlook Express 6.0?

Comment 20

[Wilmack Sanchez]

Attachment: This is the mail saved directly from Outlook Express 6.0

Comment 21

[Wilmack Sanchez]

This is what MessageWall software reports: 
{0} (0) SERVER/STATUS: connection from 192.168.110.1:35391
{0} (0) SMTP/STATUS: Message start: Relay
{0} (0) SMTP/INFO: From: [e-mail]
{0} (0) SMTP/INFO: To: [e-mail]
{0} (0) SMTP/STATUS: received message of 5362 bytes
{0} (0) MIME/REJECT: unable to find boundary
{0} (0) SMTP/FATAL: client QUIT

And this is the answers of the MessageWall Support people:

<flamingcow> ok, that means that the message has an invalid MIME
structure and it can't find the boundary line
<flamingcow> since it can't check the mail contents, it rejects the mail
<camoa> it is a digitally signed message from a netscape client...
<camoa> 4.7 
<flamingcow> the digital signature software is probably mucking up the
line endings or something
<camoa> and when it comes from a 7.0 it rejects it for Bare LF
<flamingcow> you can direct complaints to the application developers for
violating the SMTP rfcs
<camoa> is there any way to select which reviews it does?
<camoa> hehehehehe
<flamingcow> those are basic parsing items
<flamingcow> it can't do any checking whatsoever if the mail it that
badly formed
<flamingcow> so it just rejects it
<camoa> i Know, but the problem is it's a inquiry from an important
client 
<camoa> anyway not to rejectit?
<flamingcow> nope; that is not and will never be an option
<camoa> in configurations?
<flamingcow> i don't support software that violates simple SMTP rules
like that
<flamingcow> dude, i don't know what to tell you
<flamingcow> in order to do proper mail filtering, you have to be
willing to reject the completely broken mail that can't even be filtered
<camoa> i know, i know 

Comment 22

[(not reading, please use [e-mail] instead)]

thanks to jgmyers and chris.newman for the info.

from chris newman:

>I suspect the problem is a combination of a minor bug in Mozilla with a major
>bug in MessageWall.
>
>It appears Mozilla is generating a Content-Type header line which is > 78
>characters long.  This is not advisable according to the standards:
>   http://www.apps.ietf.org/rfc/rfc2822.html#sec-2.1.1

for the mozilla side, I'll morph this bug into that.

I think on the mozilla side, we can easily do the right thing.

my gut tells me we do have code that does the proper header folding, except this
one particular header is manually created.

I'll look into it, but no plans to do this right away (working on 1.3 blockers
and bigger 1.4 alpha issues)

Comment 23

[John G. Myers]

Re comment #21, where is the evidence for this claim of "bare LF"?  I see
nothing to support a claim that Mozilla is mucking up line endings.  The
evidence seems to point to MessageWall not handling unquoted boundary parameters
as required by the MIME specification.

Outlook Express uses a boundary value which includes a tspecial, so such values
must be kept quoted per the MIME specification.

Comment 24

[Chris Newman]

A good way to get a second opinion if a message is valid is to go to this web form:

   http://www.apps.ietf.org/msglint.html

I observe there are no errors with MIME parsing of the message embedded in
attachment 114501 [details] (it generates a number of warnings, but none of them relevant
to this issue).

Comment 25

[Wilmack Sanchez]

Attachment: Inbox mail reviewed by http://www.apps.ietf.org/msglint.html

This is the answer I get when I reviewed mozilla mail (attachment 114501 [details]) using
http://www.apps.ietf.org/msglint.html

Comment 26

[Wilmack Sanchez]

Attachment: Sent mail reviewed by http://www.apps.ietf.org/msglint.htmll

This is the answer I got from http://www.apps.ietf.org/msglint.html  when I
reviewed attachment 114501 [details]

Comment 27

[Carlos Ospina]

hello, 

We've made a test using NS 7.0 and we've digitally signed the message without 
any problem, the message was correctly delivered.

We're Using Postfix as our Mail Server.

Comment 28

[Carlos Ospina]

PS: and of course it went through messagewall.

Comment 29

[(not reading, please use [e-mail] instead)]

sorry for the spam.  making bugzilla reflect reality as I'm not working on these bugs.  filter on FOOBARCHEESE to remove these in bulk.

Comment 30

[Nelson Bolyard (seldom reads bugmail)]

Mozilla's S/MIME messages now properly quote boundaries.