Last Comment Bug 800877 - 8BITMIME keyword ignored, BODY=8BITMIME absent in 8bit transfers.
: 8BITMIME keyword ignored, BODY=8BITMIME absent in 8bit transfers.
Status: RESOLVED FIXED
: regression
Product: MailNews Core
Classification: Components
Component: Networking: SMTP (show other bugs)
: 16
: All All
: -- major (vote)
: Thunderbird 19.0
Assigned To: Masatoshi Kimura [:emk]
:
Mentors:
Depends on:
Blocks: 507804
  Show dependency treegraph
 
Reported: 2012-10-12 04:54 PDT by Alexei
Modified: 2012-10-30 07:13 PDT (History)
5 users (show)
ryanvm: in‑testsuite-
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
nsemail-1.eml (4.25 KB, application/octet-stream)
2012-10-12 04:54 PDT, Alexei
no flags Details
Message with cuted highest bit returns bt sendmail in such kind (4.61 KB, text/plain)
2012-10-12 05:09 PDT, Alexei
no flags Details
English version creates message without 8bit chars (4.17 KB, text/plain)
2012-10-12 05:11 PDT, Alexei
no flags Details
SMTP log (1.98 KB, text/plain)
2012-10-12 05:53 PDT, Alexei
no flags Details
Change the message back to a hard-coded English string (5.77 KB, patch)
2012-10-13 00:52 PDT, Masatoshi Kimura [:emk]
standard8: feedback+
Details | Diff | Review
patch v2 (6.03 KB, patch)
2012-10-29 15:47 PDT, Masatoshi Kimura [:emk]
standard8: review+
Details | Diff | Review

Description Alexei 2012-10-12 04:54:41 PDT
Created attachment 670766 [details]
nsemail-1.eml

User Agent: Mozilla/5.0 (Windows NT 6.1; rv:15.0) Gecko/20100101 Firefox/15.0.1
Build ID: 20120905151427

Steps to reproduce:

1)Compose and send signed message
2)Catch file %temp%\nsemail-1.eml while mozilla ask me password to SMTP server
3) Continue send to SMTP
4) Get message from Mail Delivery Subsystem <MAILER-DAEMON@lps.komi.ru> subj:"Returned mail: see transcript for details", with details "DATA
<<< 554 5.6.0 Message contains NUL characters
554 5.0.0 Service unavailable (Message contains NUL characters)"



Actual results:

Mozilla composed message with UTF8 encoded text (it is 8bit chars!): "Это сообщение в формате MIME подписано с использованием криптографии."
between headers and the first mime boundary. This text is Russian equivalent of phrase "This is a cryptographically signed message in MIME format.".
Then sendmail cuts highest bit in 8-bit chars and one of the chars (0x80) become 0x0 and then sendmail claims than "Message contains NUL characters".
Here is dump of eml file (at 0x181 body begins and it is 8bit even with mail.strictly_mime=true):
C:\Users\ak\AppData\Local\Temp\11111\nsemail-1.eml
00000000F0: 74 3A 20 65 65 65 0D 0A │ 43 6F 6E 74 65 6E 74 2D  t: eee♪◙Content-
0000000100: 54 79 70 65 3A 20 6D 75 │ 6C 74 69 70 61 72 74 2F  Type: multipart/
0000000110: 73 69 67 6E 65 64 3B 20 │ 70 72 6F 74 6F 63 6F 6C  signed; protocol
0000000120: 3D 22 61 70 70 6C 69 63 │ 61 74 69 6F 6E 2F 70 6B  ="application/pk
0000000130: 63 73 37 2D 73 69 67 6E │ 61 74 75 72 65 22 3B 20  cs7-signature";
0000000140: 6D 69 63 61 6C 67 3D 73 │ 68 61 31 3B 20 62 6F 75  micalg=sha1; bou
0000000150: 6E 64 61 72 79 3D 22 2D │ 2D 2D 2D 2D 2D 2D 2D 2D  ndary="---------
0000000160: 2D 2D 2D 6D 73 30 37 30 │ 37 30 31 30 31 30 38 30  ---ms07070101080
0000000170: 31 30 31 30 36 30 32 30 │ 32 30 33 30 38 22 0D 0A  1010602020308"♪◙
0000000180: 0D 0A D0 AD D1 82 D0 BE │ 20 D1 81 D0 BE D0 BE D0  ♪◙Э›т›о› с›о›о›б
0000000190: B1 D1 89 D0 B5 D0 BD D0 │ B8 D0 B5 20 D0 B2 20 D1  ›щ›е›н›и›е› в› ф
00000001A0: 84 D0 BE D1 80 D0 BC D0 │ B0 D1 82 D0 B5 20 4D 49  ›о›р›м›а›т›е› MI
00000001B0: 4D 45 20 D0 BF D0 BE D0 │ B4 D0 BF D0 B8 D1 81 D0  ME п›о›д›п›и›с›а
00000001C0: B0 D0 BD D0 BE 20 D1 81 │ 20 D0 B8 D1 81 D0 BF D0  ›н›о› с› и›с›п›о
00000001D0: BE D0 BB D1 8C D0 B7 D0 │ BE D0 B2 D0 B0 D0 BD D0  ›л›ь›з›о›в›а›н›и
00000001E0: B8 D0 B5 D0 BC 20 D0 BA │ D1 80 D0 B8 D0 BF D1 82  ›е›м› к›р›и›п›т›
00000001F0: D0 BE D0 B3 D1 80 D0 B0 │ D1 84 D0 B8 D0 B8 2E 0D  о›г›р›а›ф›и›и›.♪
0000000200: 0A 0D 0A 2D 2D 2D 2D 2D │ 2D 2D 2D 2D 2D 2D 2D 2D  ◙♪◙-------------
0000000210: 2D 6D 73 30 37 30 37 30 │ 31 30 31 30 38 30 31 30  -ms0707010108010
0000000220: 31 30 36 30 32 30 32 30 │ 33 30 38 0D 0A 43 6F 6E  10602020308♪◙Con
0000000230: 74 65 6E 74 2D 54 79 70 │ 65 3A 20 74 65 78 74 2F  tent-Type: text/
0000000240: 70 6C 61 69 6E 3B 20 63 │ 68 61 72 73 65 74 3D 55  plain; charset=U
0000000250: 54 46 2D 38 3B 20 66 6F │ 72 6D 61 74 3D 66 6C 6F  TF-8; format=flo
0000000260: 77 65 64 0D 0A 43 6F 6E │ 74 65 6E 74 2D 54 72 61  wed♪◙Content-Tra
0000000270: 6E 73 66 65 72 2D 45 6E │ 63 6F 64 69 6E 67 3A 20  nsfer-Encoding:
0000000280: 71 75 6F 74 65 64 2D 70 │ 72 69 6E 74 61 62 6C 65  quoted-printable
0000000290: 0D 0A 0D 0A 0D 0A 0D 0A │ 0D 0A 2D 2D 2D 2D 2D 2D  ♪◙♪◙♪◙♪◙♪◙------
00000002A0: 2D 2D 2D 2D 2D 2D 2D 2D │ 6D 73 30 37 30 37 30 31  --------ms070701


Expected results:

This message should be english or encoded with QP, but I am not sure for 100%.
In english version of program we see normal 7bit chars:
Subject: test
Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms020303050703060102010503"

This is a cryptographically signed message in MIME format.

--------------ms020303050703060102010503
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: quoted-printable
Comment 1 Alexei 2012-10-12 05:09:25 PDT
Created attachment 670768 [details]
Message with cuted highest bit returns bt sendmail in such kind
Comment 2 Alexei 2012-10-12 05:11:06 PDT
Created attachment 670769 [details]
English version creates message without 8bit chars
Comment 3 Nikolay Shopik 2012-10-12 05:15:08 PDT
Hi Alexei,
Could you provide SMTP log?
https://wiki.mozilla.org/MailNews:Logging
Message could use 8-bit symbols if smtp server anounce such capability [8BITMIME], sometimes SMTP server anounce it but won't accept such messages.
Comment 4 Alexei 2012-10-12 05:52:33 PDT
Server puts capability flag 8BITMIME as I see in logs, but client should ask for such capability with  BODY=8BITMIME clause at the "MAIL FROM:"
As we can see thunderbird does not.

This I have read in http://tools.ietf.org/html/rfc1652 page#1,topic#3 :If the server SMTP responds with code 250 to the EHLO
   command, and the response includes the EHLO keyword value 8BITMIME,
   then the server SMTP is indicating that it supports the extended MAIL
   command and will accept MIME messages containing arbitrary octet-
   aligned material.

   The extended MAIL command is issued by a client SMTP when it wishes
   to transmit a content body consisting of a MIME message containing
   arbitrary lines of octet-aligned material. The syntax for this
   command is identical to the MAIL command in [1], except that a BODY
   parameter must appear after the address.  Only one BODY parameter may
   be used in a single MAIL command.
Comment 5 Alexei 2012-10-12 05:53:36 PDT
Created attachment 670775 [details]
SMTP log
Comment 6 Nikolay Shopik 2012-10-12 06:11:22 PDT
Clearly something borked here, IIRC could be regression. confirming
Comment 7 Nikolay Shopik 2012-10-12 06:27:12 PDT
Seems OS independ as I see same behaviour on Linux with 15.0.1 english release even when not using signed mails, just cyrillic letters.
Comment 8 Nikolay Shopik 2012-10-12 06:33:12 PDT
More investigation show when MTA doesn't announce 8BITMIME, thunderbird still  send 8bit body.
Comment 9 Masatoshi Kimura [:emk] 2012-10-12 12:45:58 PDT
This is a regression from bug 507804. The message was not localizable before the bug.
Comment 10 Masatoshi Kimura [:emk] 2012-10-13 00:52:35 PDT
Created attachment 671049 [details] [diff] [review]
Change the message back to a hard-coded English string

- It doesn't make sense to encode the message because the message will be displayed only if the MUA doesn't support MIME.
- We need to consider what to do in case the server doesn't support 8BITMIME.
In short, we can't use non-ASCII characters here.
Comment 11 Mark Banner (:standard8) 2012-10-29 15:38:55 PDT
Comment on attachment 671049 [details] [diff] [review]
Change the message back to a hard-coded English string

Review of attachment 671049 [details] [diff] [review]:
-----------------------------------------------------------------

This is reasonable, but not quite r+... we need a comment as I mention below.

::: mailnews/extensions/smime/src/nsMsgComposeSecure.cpp
@@ +24,5 @@
>  #define MK_MIME_ERROR_WRITING_FILE -1
>  
>  #define SMIME_STRBUNDLE_URL "chrome://messenger/locale/am-smime.properties"
>  
> +static const char crypto_multipart_blurb[] = "This is a cryptographically signed message in MIME format.";

This needs a comment explaining why we're hard-coding the english string.
Comment 12 Masatoshi Kimura [:emk] 2012-10-29 15:47:04 PDT
Created attachment 676369 [details] [diff] [review]
patch v2

Added a comment explaining why the english string is hard-coded.
Comment 13 Mark Banner (:standard8) 2012-10-30 03:46:30 PDT
Comment on attachment 676369 [details] [diff] [review]
patch v2

Review of attachment 676369 [details] [diff] [review]:
-----------------------------------------------------------------

Great, thanks.
Comment 14 Ryan VanderMeulen [:RyanVM] 2012-10-30 07:13:49 PDT
https://hg.mozilla.org/comm-central/rev/5b9f2c237c3f

Note You need to log in before you can comment on or make changes to this bug.