User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.94 Safari/537.36 Steps to reproduce: I sent a message in Plain Text containing NONASCII characters and long lines. Actual results: TB inserted hard line breaks in the long lines. The message received displays as expected in TB, but NOT in Outlook Express, where it has inappropriate HARD breaks in the long lines. Expected results: In a flowed text, the line breaks in long lines must depend on the width of the viewport OF THE RECEIVER, hence MUST NOT be hard-coded by the sender. Details and suggestion follow. Versailles, Sun 02 Jun 2013 22:47:00 +0200
(1) I set Thunderbird 17.0.6 with: in "TB17 > Tools > Options > Advanced > General > Advanced Configuration > Config Editor... > I'll be careful, I promise": - in "Search: strictly": mail.strictly_mime;true - in "Search: flowed": mailnews.display.disable_format_flowed_support;false mailnews.send_plaintext_flowed;true (2a) I went "TB17 > Write (New Message) > Options" with: - "Format > Plain Text Only" - "Character Encoding > Western (ISO-8859-1)" (2b) I included in that message a long line (245 chars), supposed (since in Plain Text) to be a paragraph (2c) I inserted a few NONASCII characters (« À CURAÇAO et à SÃO PAULO, $1+£1=1.43€ ± 2‰ ») which apparently caused TB encoding to switch from its usual 7bit default to quoted-printable (2d) which made the message sent with: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable (3a) When that message gets received in TB, its Subject and Body are displayed correctly, including the long line: THERE was once upon a time a widow who had two daughters. The eldest was so much like her in the face and humor that whoever looked upon the daughter saw the mother. They were both so disagreeable and so proud that there was no living with them. (3b) But when received in Outlook Express (and probably in any email client or web mail applying exactly the specifications of "quoted-printable"), the long lines are wrongly wrapped (at about 72 char): THERE was once upon a time a widow who had two daughters. The eldest was so much like her in the face and humor that whoever looked upon the daughter saw the mother. They were both so disagreeable and so proud that there was no living with them. (4a) Inspecting (in both TB and OE) the MIME source of the message received (using "Notepad++ > View > Show Symbol > Show All Characters"): THERE was once upon a time a widow who had two daughters. The eldest was =CRLF CRLF so much like her in the face and humor that whoever looked upon the=20CRLF daughter saw the mother. They were both so disagreeable and so proud=20CRLF that there was no living with them. (4b) Analysis: The MIME source of the received message contains 4 "CRLF" (Carriage Return + Line Feed), that duly insert breaks *in that MIME source*. The 1st one is escaped (by an "=" right before it), but the 3 others are NOT, hence are conveyed down to the email client that will therefore create breaks *in the rendered message*, excepted when read in TB, that intended it that way. Logically, specifying "format=flowed" should imply that the eventual breaks in RENDERING long lines are decided by the RECEIVER's client, depending on its viewport. So when Transfer-Encoding that "flowed" part with "quoted-printable", sure TB has to insert breaks in the *encoding* text (the one in the MIME source), but these breaks must ALL be escaped (i.e. preceded by a "=" sign), so they DO NOT create a break in the TO-BE-RENDERED text. Surely TB developers did that properly at the formatting stage (when applying "flowed"), but at the Transfer-Encoding stage (when applying "quoted-printable") they apparently inadvertently inserted the breaks. This is due IMO to the unnecessarily complicate specifications of QP ( http://en.wikipedia.org/wiki/Quoted-printable#Quoted-printable_encoding ), that go to lengths imposing to encode trailing invisible chars, something that IMO should NOT be addressed at all in the QP specs, but only recommended at the writer's stage. One consequence of this complication is the habit of most HTML editors to encode their trailing spaces (into "=20"), without bothering following or not with a " =" (to maintain the space and escape the coming CRLF), since in HTML source, outside quotes, any series of spaces, Tabs and CRLFs, can be increased or reduced, it will produce the same result as long as you leave at least one (space or tab or CRLF). TB developers seem to have NOT noticed that their "=20", being NOT followed by a "=", does NOT escape the following CRLF, which then will break the rendered line, causing the exact contrary of the very goal of "flowed". Now let's notice that TB developers at least kept the encoding and decoding consistent, i.e. kept TB decoding correctly the messages encoded by TB. (5) Suggested fix: When Transfer-Encoding with "quoted-printable" a part formatted with "flowed", TB MUST NOT add any non-escaped CRLF; that is, to break the line as required *in the encoding MIME source*, the sequence that TB adds should NOT be " =CRLFCRLF" or "=20CRLF" but "=20=CRLF". Versailles, Sun 02 Jun 2013 22:49:30 +0200
(In reply to Michel Merlin from comment #1) > (2b) I included in that message a long line (245 chars), supposed (since in > Plain Text) to be a paragraph In format=flowed encoding, this would be spread of several lines, yes. > (2c) I inserted a few NONASCII characters (« À CURAÇAO et à SÃO PAULO, > $1+£1=1.43€ ± 2‰ ») > which apparently caused TB encoding to switch from its usual 7bit default to > quoted-printable > > (2d) which made the message sent with: > > Content-Type: text/plain; charset=windows-1252; format=flowed > Content-Transfer-Encoding: quoted-printable That's the expected behavior. If non-ASCII characters are included, messages are sent with plain-text parts of 8bit Content-Transfer-Encoding, or quoted-printable with mail.strictly_mime set. > (3b) But when received in Outlook Express (and probably in any email client > or web mail applying exactly the specifications of "quoted-printable"), the > long lines are wrongly wrapped (at about 72 char): Microsoft Outlook does not obey the format=flowed specification, they render line breaks as they are. In fact, they came up with their own "standard" for non-QP encoding which adds a space in lines which are /not/ to be connected with the next line to form a paragraph, thus the exact opposite of what RFC 3676 instructs. For QP-encoded messages, Microsoft products add a simple continuation character "=CRLF" to indicate "soft" line breaks rather than "=20CRLF" as instructed by the standard. In a compliant client, this causes bug 387687. You'll see "=20" at the end of lines in Microsoft-generated messages which are supposed to be "hard" line breaks, again the opposite of the IETF specification. To emphasize, this is a bug in Microsoft's implementation disregarding RFC 3676, nothing Thunderbird can do about it. > THERE was once upon a time a widow who had two daughters. The eldest was =CRLF > CRLF I would expect "=20CRLF" in this line unless there was a hard line break. When encoding accented characters, they have to be escaped, thus exceeding the line limit for QP encoding, hence causing the first line ending with a continuation character "=CRLF"; the line continues, and at the end of the typed line there is a soft line break followed by a new-line indicator, "=20CLRF" in QP encoding. > so much like her in the face and humor that whoever looked upon the=20CRLF > daughter saw the mother. They were both so disagreeable and so proud=20CRLF > that there was no living with them. These are correctly encoded RFC 3676 "soft" line breaks in the first two line, followed by a "hard" line break (just "CRLF") at the end of the third line. > When Transfer-Encoding with "quoted-printable" a part formatted with > "flowed", TB MUST NOT add any non-escaped CRLF; that is, to break the line > as required *in the encoding MIME source*, the sequence that TB adds should > NOT be " =CRLFCRLF" or "=20CRLF" but "=20=CRLF". Unless I'm mistaken, the encoding of "=20=CRLF" would no longer indicate a soft line break in accordance with RFC 3676 but simply be the equivalent of " =CRLF", thus a space followed by a contrinuation line. Thus, you'd end up with bug 387687 at the recipient's end again, thus would be the same flawed encoding as Microsoft. In summary, this does not seem to be a viable proposal.
Component: Untriaged → MIME
Product: Thunderbird → MailNews Core
TB is the only email client to fail wrapping (or "flowing") properly long lines of plain text "rsx11m" Mon 03 Jun 2013 04:10:04 GMT, I spent a great deal of care to post a lot of precise technical info and test. This bug report was an accessory to my research and tests (that are going on and that I will post when they are ready) that I started after the posts (last being the kind one of yours http://forums.mozillazine.org/viewtopic.php?f=28&t=1697835#p12894089 of Sat 01 Jun 2013 13:15:37 GMT, for which I thank you) we exchanged under "Use 7 bit encoding for UTF-8 messages". But here in your reply you show from the start your determination to finish, no matter the evidences, with a pre-decided "this does not seem to be a viable proposal". So I think it would be useless to read and answer again on the technical level again. I would simply notice a few things (apparently "details" for you): - Technics can only progress with open minds. When some start to forbid newcomers to bring new opinions or ideas, progress stalls, and even becomes impossible. Said Amadou Hampâté Bâ: A full calabash holds no new water (in plain English: who knows all learns nothing). Forum "I-know-all" gurus (who usually know little), while feeling helping progress, actually prevent it. - Microsoft products and "standards" were chosen by 1B people with their dearly earned money. By how many people, and with how much effort or money, were chosen W3C "standards" (that BTW enjoy just the same amount and nature of political or marketing support)? Particularly, the "Content-Type: format=flowed" one? Do you really ignore that standards are NOT equal, and that "quoted-printable" is much better accepted and much more used by the public, than "flowed"? Have you really missed how strongly these 2 standards conflict? - Have you really missed that "indent characters" (usually "> ") are dumb, useless, and have been dropped by me starting in ~2003 (and then by many everywhere on the web, including the MS gurus who bashed me for this)? see my final explanation in https://groups.google.com/forum/?fromgroups#!topic/microsoft.public.outlookexpress.general/16TJae88hBI §A1 "Reasons", and myriads examples like https://groups.google.com/d/msg/microsoft.public.windows.inetexplorer.ie6_outlookexpress/bDPyHHnFieU/OkwI8FL5CvUJ "Switched back to OE from OL" of Tue 21 Dec 2004 11:07:55 GMT or https://groups.google.com/d/msg/microsoft.public.win2000.multimedia/63hCTsueSpU/jgQheSSPF4sJ "Troubleshooting Links to messages" of Wed 29 Oct 2003 21:20:35 GMT. - Have you really missed that wrapping (or flowing) only makes sense if done according to the viewport width available UNDER THE USER'S EYES, hence if done by the RECEIVER's email client? and that since about everyone (excepted TB pack) understands this, about every text editor (even tiniest and simplest as Notepad) has, for decades, displayed plain text long lines properly, leaving to the human reader the choice of "flowed" or not? - Have you really missed that, as firstname.lastname@example.org wrote in his https://bugzilla.mozilla.org/show_bug.cgi?id=387687#c6 post of 2010-01-27 23:20:58 GMT in the bug discussion that you linked (and that I had already visited of course), TB is the only email client dumb enough to ignore the basics of information handling, the obvious best solutions, the needs and wishes of the majority of users, while MS, Apple, and others, handle it right? I switched from CompuServe to Netscape, then to OE (~1997), then to Outlook (and back immediately, ~1998), then I tried very hard to TB, but TB is still too lacking so far, and seeing how bugs are addressed, will only replace others when their organized sabotage has gone further than now. In conclusion if something did "not seem to be a viable proposal", it would be IMO rather TB or its pack's rebuttal of others and of reality. Sorry for being frank, but being as polite as my usual habit costs time, and I feel I spent enough already, at least for now. Versailles, Mon 03 Jun 2013 15:37:00 +0200
Interesting, that's an unexpectedly hefty response. I'm wondering where you see the rudeness or ignorance in my post, but anyway. I was a bit short given that it was late and we had some discussion already in the MZ forum thread (which was mostly about URL encoding having the advantage in QP that they can be split across lines without breaking the URL). I was also a bit disappointed as it appeared to me that you didn't quite understand how the format=flowed wrapping in the QP context works, given that I spent a good deal of time too explaining it over in the forum. > TB is the only email client to fail wrapping (or "flowing") properly long > lines of plain text This is where we disagree. The format=flowed concept goes on top of the QP concept, where CTE=QP implies that logical lines can be spread across multiple encoded lines using the continuation '=' character. They remain single lines, though. In format=flowed, the receiving client is free to rewrap the lines as indicated by SP+CR+LF at the end. Encoding that in QP yields the "=20CRLF" as described and is not a "hard" line break. These are multiple logical lines forming a paragraph that is allowed to rewrap when displayed on the recipient's end. If the receiving client or a user configuration doesn't apply that, it's a choice that the respective vendor or user has made. BTW: Thunderbird-generated flowed-format messages wrap just fine in the Gmail web interface and apparently also with Apple Mail. > But here in your reply you show from the start your determination to finish, > no matter the evidences, with a pre-decided "this does not seem to be a > viable proposal". Obviously I've read your entire post before replying. Thus, after getting to your proposal to essentially abandon flowed formatting and replacing it with the single-line QP approach, I figured that you will have a hard time convincing the developers to do so, hence the "not viable" comment (and personally, I don't think it's the right thing to do, especially given that format=flowed would still be needed for 7bit and 8bit CTEs). At that time the direction of my response was clear, yes, but not before reading your analysis. As you may have noticed, I've triaged this into the MIME component so that it gets the attention of the respective people maintaining that module.
Isn't this a duplicate of the previously mentinoed bug 387687?
No, it's the opposite - the bug opener wants us to disregard RFC 3676 "soft" line breaks and create essentially QP-wrapped single lines per paragraph, thus *prompting* bug 387687 in the first place. Thus, while certainly related, it's not a duplicate.
You need to log in before you can comment on or make changes to this bug.