149272 - 8-bit messages transformed to 7-bit: Invalidates signature

Status: VERIFIED INVALID

(MailNews Core :: Security: S/MIME, defect, P1)

Description

[bugzilla]

Hi,

(this is for Mozilla 1.0RC3)

when I extract the signature attachment of a signed mail to a file
sig.pkcs7 and add
-----BEGIN PKCS7---- and ... END ... and feed it to openssl (0.9.6d)

    openssl pkcs7 -print_certs -noout < sig.pkcs7

it displays the senders certificate twice instead of the senders
certificate and its root certificate. Signing with mutt/OpenSSL works
and the attachment contains the certificate of the signer and its root.

Also, signing works only for new mails; if I reply using mutt and reply
to this with mozilla the signature of the second reply is invalid.

If needed I can provide the mails via mail.

Regards,
Thomas

Comment 1

[bugzilla]

Two corrections:
- with mutt/OpenSSL I can configure whether or not to have the root
  certificate in the signature, but including the signer's cert twice
  should be fixed
- the signature is also invalid for a new mail if it contains an
  attachment, so it could also be an MTA which modifies the contents of
  the attachment and thus makes the signature invalid

Comment 2

[Charles Rosendahl]

Some questions for you - when you say that it is invalid, are you seeing a
broken signature icon or an icon with a question mark?  If you see the question
mark, this is a workaround for verificationon demand from IMAP that has been
fixed in the latest builds.  If you see an outright broken signature icon, then
we can continue.

The second issue related to the two certs is a separate issue which might or
might not be a bug.  Netscape includes the encryption cert in all signed
messages, so it might be duplicating the cert in all circumstances.  A dual key
cert would require the presence of two certs, whereas a dual use cert might
appear as duplicated.

Let me know,
charles

Comment 3

[Charles Rosendahl]

We also have an open bug that is to be fixed by RTM to include the CA chain in
signed messages.

Comment 4

[bugzilla]

Issue signature invalid:
  In mozilla I see a broken signature icon (the broken pencil with
  a white "x" in a read circle) and OpenSSL says:
    38402:error:21071065:PKCS7 routines:PKCS7_signatureVerify:digest failure:pk7_doit.c:808:
    38402:error:21075069:PKCS7 routines:PKCS7_verify:signature failure:pk7_smime.c:256:

  So the problem is either when signing (mozilla) or during transmission
  (MTA). I can investigate this tomorrow, only.

Issue duplicate certs:
  You are right. If the cert with the encryption key is included even
  for non-encrypted mails then one could say it is a feature that 2
  certs are included.
  Maybe it it is possible to check whether the 2 certs are identical.

  My cert contains only one key; some attributes:
      Data:
          Signature Algorithm: md5WithRSAEncryption
          Subject Public Key Info:
              Public Key Algorithm: rsaEncryption
              RSA Public Key: (2048 bit)
          X509v3 extensions:
              Netscape Cert Type: critical
                  SSL Client, S/MIME, Object Signing
              X509v3 Basic Constraints: critical
                  CA:FALSE
      Signature Algorithm: md5WithRSAEncryption

Issue bug to include the CA cert:
  Maybe it is best to make it configurable. RFC 2312, 2632 don't require
  it but recommend it e.g. for DSA certs.

Comment 5

[bugzilla]

I just had the chance for a test: the sender activated the option
to use quoted-printable for 8-bit characters and a second reply to
a previously sent message arrived intact!
So it most probably _is_ some MTA.

Comment 6

[Charles Rosendahl]

I'm with you now.  I changed my character encoding for composing messages to
UTF-8 and replied to a mail message and signed it.  Sending it through the
Netscape Mail Server to my netscape account, the message arrives intact and
correctly signed.  Sending it to my AOL account and retrieving it through the
AOL mail server gateway, the message arrives with a broken signature.

Most likely the message encoding is being converted, or not.  technically if the
message encoding is altered, then the message has been altered, and it _is_
invalid (technically)  - Need to look at the RFC and see what they say about
character encoding transformations generated during transport.

Need to look at this more carefully - doubtful it can be fixed by RTM as it
seems to be dependent on the MTA, but will note this as something we need to
look into.

Kai, Stephane - your thoughts would be greatly welcomed.

Confirming bug.

Comment 7

[Charles Rosendahl]

The question is:  Are there any drawbacks to forcing 'Quoted printable' under
the hood when a user sends/replies/forwards a signed message containing 8-bit
characters?  If not, this is probably the best solution to the problem.

Excerpt of a conversation with Kai:

carosendahl (12:08:31 PM): well not all users, just users of western alphabet
plus special chars.
kaie147 (12:09:52 PM): You said it only happens with some mail servers?
carosendahl (12:11:41 PM): True - but it can happen during transport on the
relays in addition to the destination mail server. The AOL mail gateways (!)
have this problem. An awful lot of mail servers out there, a lot of them fairly
old and kludgey...
carosendahl (12:13:49 PM): The quick fix would be to force quoted printable if
the user is using an 8 bit charset when sending signed messages.
kaie147 (12:22:23 PM): hmm, can you write that suggestion into a bug and cc me
and ducarroz ?

Comment 8

[Charles Rosendahl]

*** Bug 152318 has been marked as a duplicate of this bug. ***

Comment 9

[Kai Engert (:KaiE:)]

I agree the signer's certificate should not get included twice in messages we send.

I suspect this error is introduced because we support dual-key-certificates,
i.e. separate certificates for signing and the prefered encryption certificate
of the sender.

It seems reasonable to avoid including the cert twice, if a user is not using
dual-key certificates and uses the same cert for both purposes.


Re the encoding issue:

You say it depends on the mail server whether the arriving message is valid or not.

Charles, could you help and prepare a test message?
If I understand you correctly, you say, you are able to send and receive
messages that look broken depending on the mail server.

Could you please do the following:

Prepare a single message. Send it to multiple recipients at the same time. Send
it to recipient A, so that it will be correct, and in addition, send it to
recipient B so that it will appear broken.

Please use "view message source" for each of both received messages. Please save
them to a file and attach both here.

This should enable us to see what the exact differences of the encodings done by
the servers are. We can then judge wether the servers are broken or whether we
are sending incorrect data.

Comment 10

[Charles Rosendahl]

Attachment: Signed message sent through NS Mail Server

Has a good signature

Comment 11

[Charles Rosendahl]

Attachment: Through AOL Mail Gateway

Viewed within Mach 5 Browser, has a bad signature.  The content however is
identical to the content in the netscape mail account message.

Comment 12

[Kai Engert (:KaiE:)]

Charles, thanks for the sample messages.

The contents of the attached messages are not identical. I used a diff program,
and the AOL mail gateway introduced a change to the wrapping of the inner message.

I was also able to reproduce the same problem myself, sending the same message
to both my netscape and aol account at the same time.

The AOL gateway is reformatting the inner contents of the message.
This is not allowed.
Our signature verification correctly reports the signature as broken.


But I do not see how this problem is related to the original problem.
If I understand correctly, [e-mail] reports a different problem?

Comment 13

[Charles Rosendahl]

This bug is now related to comment 4, comment 5, and comment 6.  The cert issue
is no longer part of this bug.

If you were to use quoted printable now under edit->prefs->composition and check
the quote option, you will find that the messages now arrive intact and the
signature validates.  Essentially if you send it out in 7 bit, it arrives in
7-bit.  If you send it out in 8-bit, it is altered and the charset is 7-bit ascii.

Older/Poorly implemented mail servers incorrectly alter the message contents by
reducing the charset from 8-bit to 7-bit on a misguided notion of
compression/data delivery speed - most likely a legacy throwback to when charges
were accrued on the amount of data delivered in number of bytes (a la EDI).

Comment 14

[Kai Engert (:KaiE:)]

I can't reproduce problems using simple messages and 8 bit messages.

I composed a message to myself, containing non-ascii 8 bit characters (צה�).
I sent it signed to @netscape and @aol using quoted printable.
I sent it signed to @netscape and @aol "as is".
All four messages are displayed with a correct signature for me.


If you say the problem is related to 8bit characters being contained in
messages, succesfully arriving @netscape but not @aol, I would have expected the
test messages you attached to this bug contained 8bit characters. However, I
just checked, they do not, all chars in both messages are plain 7bit.


If using quoted printable fixed the problem, then I simply guess, the mail
server you saw the problems on did not reformat the message when you used quoted
printable.


In addition, I have a message, when I forward that message and sign it, it
arrives @aol with an invalid signature, whether I have used quoted printable or not.


I'm resolving this as invalid. S/Mime signatures only work if MTA do not alter
the messages they are transfering.

Please reopen if you think I'm wrong.

Comment 15

[Charles Rosendahl]

Agreed - If mail is being altered during transport, then it isn't a bug in the
client, but rather the servers.

The workaround might be to use quoted printable, but according to Kai, that
isn't a 100% guarantee either.