800877 - 8BITMIME keyword ignored, BODY=8BITMIME absent in 8bit transfers.

Status: RESOLVED FIXED

(MailNews Core :: Networking: SMTP, defect)

Description

[Alexei]

Attachment: nsemail-1.eml

User Agent: Mozilla/5.0 (Windows NT 6.1; rv:15.0) Gecko/20100101 Firefox/15.0.1
Build ID: 20120905151427

Steps to reproduce:

1)Compose and send signed message
2)Catch file %temp%\nsemail-1.eml while mozilla ask me password to SMTP server
3) Continue send to SMTP
4) Get message from Mail Delivery Subsystem <[e-mail]> subj:"Returned mail: see transcript for details", with details "DATA
<<< 554 5.6.0 Message contains NUL characters
554 5.0.0 Service unavailable (Message contains NUL characters)"



Actual results:

Mozilla composed message with UTF8 encoded text (it is 8bit chars!): "Это сообщение в формате MIME подписано с использованием криптографии."
between headers and the first mime boundary. This text is Russian equivalent of phrase "This is a cryptographically signed message in MIME format.".
Then sendmail cuts highest bit in 8-bit chars and one of the chars (0x80) become 0x0 and then sendmail claims than "Message contains NUL characters".
Here is dump of eml file (at 0x181 body begins and it is 8bit even with mail.strictly_mime=true):
C:\Users\ak\AppData\Local\Temp\11111\nsemail-1.eml
00000000F0: 74 3A 20 65 65 65 0D 0A │ 43 6F 6E 74 65 6E 74 2D  t: eee♪◙Content-
0000000100: 54 79 70 65 3A 20 6D 75 │ 6C 74 69 70 61 72 74 2F  Type: multipart/
0000000110: 73 69 67 6E 65 64 3B 20 │ 70 72 6F 74 6F 63 6F 6C  signed; protocol
0000000120: 3D 22 61 70 70 6C 69 63 │ 61 74 69 6F 6E 2F 70 6B  ="application/pk
0000000130: 63 73 37 2D 73 69 67 6E │ 61 74 75 72 65 22 3B 20  cs7-signature";
0000000140: 6D 69 63 61 6C 67 3D 73 │ 68 61 31 3B 20 62 6F 75  micalg=sha1; bou
0000000150: 6E 64 61 72 79 3D 22 2D │ 2D 2D 2D 2D 2D 2D 2D 2D  ndary="---------
0000000160: 2D 2D 2D 6D 73 30 37 30 │ 37 30 31 30 31 30 38 30  ---ms07070101080
0000000170: 31 30 31 30 36 30 32 30 │ 32 30 33 30 38 22 0D 0A  1010602020308"♪◙
0000000180: 0D 0A D0 AD D1 82 D0 BE │ 20 D1 81 D0 BE D0 BE D0  ♪◙Э›т›о› с›о›о›б
0000000190: B1 D1 89 D0 B5 D0 BD D0 │ B8 D0 B5 20 D0 B2 20 D1  ›щ›е›н›и›е› в› ф
00000001A0: 84 D0 BE D1 80 D0 BC D0 │ B0 D1 82 D0 B5 20 4D 49  ›о›р›м›а›т›е› MI
00000001B0: 4D 45 20 D0 BF D0 BE D0 │ B4 D0 BF D0 B8 D1 81 D0  ME п›о›д›п›и›с›а
00000001C0: B0 D0 BD D0 BE 20 D1 81 │ 20 D0 B8 D1 81 D0 BF D0  ›н›о› с› и›с›п›о
00000001D0: BE D0 BB D1 8C D0 B7 D0 │ BE D0 B2 D0 B0 D0 BD D0  ›л›ь›з›о›в›а›н›и
00000001E0: B8 D0 B5 D0 BC 20 D0 BA │ D1 80 D0 B8 D0 BF D1 82  ›е›м› к›р›и›п›т›
00000001F0: D0 BE D0 B3 D1 80 D0 B0 │ D1 84 D0 B8 D0 B8 2E 0D  о›г›р›а›ф›и›и›.♪
0000000200: 0A 0D 0A 2D 2D 2D 2D 2D │ 2D 2D 2D 2D 2D 2D 2D 2D  ◙♪◙-------------
0000000210: 2D 6D 73 30 37 30 37 30 │ 31 30 31 30 38 30 31 30  -ms0707010108010
0000000220: 31 30 36 30 32 30 32 30 │ 33 30 38 0D 0A 43 6F 6E  10602020308♪◙Con
0000000230: 74 65 6E 74 2D 54 79 70 │ 65 3A 20 74 65 78 74 2F  tent-Type: text/
0000000240: 70 6C 61 69 6E 3B 20 63 │ 68 61 72 73 65 74 3D 55  plain; charset=U
0000000250: 54 46 2D 38 3B 20 66 6F │ 72 6D 61 74 3D 66 6C 6F  TF-8; format=flo
0000000260: 77 65 64 0D 0A 43 6F 6E │ 74 65 6E 74 2D 54 72 61  wed♪◙Content-Tra
0000000270: 6E 73 66 65 72 2D 45 6E │ 63 6F 64 69 6E 67 3A 20  nsfer-Encoding:
0000000280: 71 75 6F 74 65 64 2D 70 │ 72 69 6E 74 61 62 6C 65  quoted-printable
0000000290: 0D 0A 0D 0A 0D 0A 0D 0A │ 0D 0A 2D 2D 2D 2D 2D 2D  ♪◙♪◙♪◙♪◙♪◙------
00000002A0: 2D 2D 2D 2D 2D 2D 2D 2D │ 6D 73 30 37 30 37 30 31  --------ms070701


Expected results:

This message should be english or encoded with QP, but I am not sure for 100%.
In english version of program we see normal 7bit chars:
Subject: test
Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms020303050703060102010503"

This is a cryptographically signed message in MIME format.

--------------ms020303050703060102010503
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: quoted-printable

Comment 1

[Alexei]

Attachment: Message with cuted highest bit returns bt sendmail in such kind

Comment 2

[Alexei]

Attachment: English version creates message without 8bit chars

Comment 3

[Nikolay Shopik]

Hi Alexei,
Could you provide SMTP log?
https://wiki.mozilla.org/MailNews:Logging
Message could use 8-bit symbols if smtp server anounce such capability [8BITMIME], sometimes SMTP server anounce it but won't accept such messages.

Comment 4

[Alexei]

Server puts capability flag 8BITMIME as I see in logs, but client should ask for such capability with  BODY=8BITMIME clause at the "MAIL FROM:"
As we can see thunderbird does not.

This I have read in http://tools.ietf.org/html/rfc1652 page#1,topic#3 :If the server SMTP responds with code 250 to the EHLO
   command, and the response includes the EHLO keyword value 8BITMIME,
   then the server SMTP is indicating that it supports the extended MAIL
   command and will accept MIME messages containing arbitrary octet-
   aligned material.

   The extended MAIL command is issued by a client SMTP when it wishes
   to transmit a content body consisting of a MIME message containing
   arbitrary lines of octet-aligned material. The syntax for this
   command is identical to the MAIL command in [1], except that a BODY
   parameter must appear after the address.  Only one BODY parameter may
   be used in a single MAIL command.

Comment 5

[Alexei]

Attachment: SMTP log

Comment 6

[Nikolay Shopik]

Clearly something borked here, IIRC could be regression. confirming

Comment 7

[Nikolay Shopik]

Seems OS independ as I see same behaviour on Linux with 15.0.1 english release even when not using signed mails, just cyrillic letters.

Comment 8

[Nikolay Shopik]

More investigation show when MTA doesn't announce 8BITMIME, thunderbird still  send 8bit body.

Comment 9

[Masatoshi Kimura [:emk]]

This is a regression from bug 507804. The message was not localizable before the bug.

Comment 10

[Masatoshi Kimura [:emk]]

Attachment: Change the message back to a hard-coded English string

- It doesn't make sense to encode the message because the message will be displayed only if the MUA doesn't support MIME.
- We need to consider what to do in case the server doesn't support 8BITMIME.
In short, we can't use non-ASCII characters here.

Comment 11

[Mark Banner (:standard8)]

Comment on attachment 671049 [details] [diff] [review]
Change the message back to a hard-coded English string

Review of attachment 671049 [details] [diff] [review]:
-----------------------------------------------------------------

This is reasonable, but not quite r+... we need a comment as I mention below.

::: mailnews/extensions/smime/src/nsMsgComposeSecure.cpp
@@ +24,5 @@
>  #define MK_MIME_ERROR_WRITING_FILE -1
>  
>  #define SMIME_STRBUNDLE_URL "chrome://messenger/locale/am-smime.properties"
>  
> +static const char crypto_multipart_blurb[] = "This is a cryptographically signed message in MIME format.";

This needs a comment explaining why we're hard-coding the english string.

Comment 12

[Masatoshi Kimura [:emk]]

Attachment: patch v2

Added a comment explaining why the english string is hard-coded.

Comment 13

[Mark Banner (:standard8)]

Comment on attachment 676369 [details] [diff] [review]
patch v2

Review of attachment 676369 [details] [diff] [review]:
-----------------------------------------------------------------

Great, thanks.

Comment 14

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/comm-central/rev/5b9f2c237c3f