1690957 - Outgoing messages are wrapped to what seems to be just one word

Status: RESOLVED FIXED

(MailNews Core :: Backend, defect)

Description

[Nazar Mokrynskyi]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:87.0) Gecko/20100101 Firefox/87.0

Steps to reproduce:

1. Insert following text into an email (paste without formatting):
   "Why might someone disable word and line wrap?

Word and line wrap may be distracting in other web and email clients as it introduces random line breaks throughout the email. See images below to see the difference between word and line wrap enabled compared to disabled."
2) Send it
3) Go to Sent folder (or check on receiving side

Actual results:

Sent text looks like this:
"Word and
line wrap
may be
distracting
in other
web and
email
clients as
it
introduces
random
line
breaks
throughout
the email.
See images
below to
see the
difference
between
word and
line wrap
enabled
compared
to
disabled."

Expected results:

This didn't happen before latest update to 86.0b1. I didn't change wrapping settings for ages and even intentionally disabled to test, didn't help either.

Comment 1

[Thomas D. (:thomas8)]

jcramner, can you comment?
Shouldn't format=flowed always be displayed as flowed?
And why is the content-transfer-encoding different between Sent folder and received message?

Hi Nazar,

I cannot reproduce this in your extreme form on 86.0b1 (32-bit).

Sent message has:

Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

So that displays correctly (flowed) without any line breaks.

Received message has:

Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: quoted-printable

Indeed that is displayed with linebreaks, which is undesirable.

Comment 2

[Joshua Cranmer [:jcranmer]]

From some of the emails I have seen, TB has had a problem flowing format=flowed text properly, but looking at message headers, it's not coming from TB 86.0b1.

(In reply to Thomas D. (:thomas8) from comment #1)

Shouldn't format=flowed always be displayed as flowed?
And why is the content-transfer-encoding different between Sent folder and received message?

Outgoing SMTP servers can change the bodies of the messages as appropriate, and Content-Transfer-Encoding is one of the easier ones to do so. (In fact, technically you're required to convert 8bit bodies if the next MTA down the line doesn't support it but in practice I believe everyone assumes everyone supports 8bit anyways).

Going back to the first question, format=flowed requires that the line have trailing whitespace to flow the text appropriately. The translation from 7-bit to quoted-printable could well kill trailing whitespace. Actually, unencoded trailing whitespace in quoted-printable is supposed to be stripped by the client, so a bad CTE-conversion job that only looks for = to be quoted could well miss the need to encode the trailing whitespace. The bug in that would case would be with the MTA.

That said, my suspicion is that we'd be more tolerant of bad servers if we just simply stopped emitting 7-bit and went with quoted-printable or base64 exclusively--using 7-bit/8-bit also brings up concerns around CRLF-versus-LF line endings.

Comment 3

[Klaus B.]

Thomas, how did you manage to get CTE: QP on the received message? Which provider does that? MTAs shouldn't really mess with the message content, but painfully we know that some do, even in an very incompetent fashion messing up all highbit characters, see bug 1435903 and
https://wiki.mozilla.org/User:Jorgk/8-bit_bytes_and_e-mail_corruption_at_Verizon,_Yahoo,_etc.

Note that format=flowed and CTE: QP are (almost) virtually exclusive, see bug 1689804 comment #18, last paragraph.

EDIT:

This didn't happen before latest update to 86.0b1.

TB 86 uses the new JS send module, so Ping is your man here.

Comment 4

[Nazar Mokrynskyi]

TB 86 uses the new JS send module, so Ping is you man here.

I'm not sure what "Ping" is...

Is there a workaround? All my emails are like that now and I can't downgrade Thunderbird, it refuses to start.

Comment 5

[Klaus B.]

Ping is the person who implemented the JS send module which is active in the latest beta replacing the old C++ backend. You can switch it off with pref mailnews.send.jsmodule. You can also downgrade using -allow-downgrade on the command line/shortcut that starts TB. It appears that this bug isn't receiving any attention. That said, there is the suspicion that some MTA/provider actually changes the e-mail format in transit, but it would still be good to know that change in TB caused this.

Comment 6

[Nazar Mokrynskyi]

Flipping "mailnews.send.jsmodule" to false helped, thanks!
I'm having this issue both with GMail and IRedMai (Postfix) server.

Comment 7

[Klaus B.]

There really aren't enough details to do anything with this issue. I tried sending your text "Word and line wrap may be distracting in other web and email clients as it introduces random line breaks throughout the email. See images below to see the difference between word and line wrap enabled compared to disabled." and it goes out just fine. So in order to do anything here, you'd need to provide whole lot more details and also a message that got wrapped in an undesired way, at best from your Sent Folder (which should have the message without any MTA/provider interference).
We need to know whether the message was sent as plain text, it appears that way, which charset was used, whether it's format=flowed, whether you have some special mailnews.wraplength set. Does it happen if you disable all add-ons?

Comment 8

[Nazar Mokrynskyi]

Oh, I though there was enough info.
Attaching 2 files. One is how message is saved in drafts, the other one is how it looks like when sent/received after this.

Comment 9

[Nazar Mokrynskyi]

Attachment: draft.txt

Comment 10

[Nazar Mokrynskyi]

Attachment: sent.txt

Comment 11

[Ping Chen [:rnons] (away)]

I can reproduce when I set mailnews.wraplength to 0, do you happen to have the same pref value?

I will make a fix soon.

Comment 12

[Nazar Mokrynskyi]

Yes, I did have mailnews.wraplength set to 0, though I do not remember doing it.
Thanks!

Comment 13

[Ping Chen [:rnons] (away)]

Attachment: Bug 1690957 - Fix text wrapping in MessageSend.jsm when mailnews.wraplength is 0. r=mkmelin

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/5950078d065f
Fix text wrapping in MessageSend.jsm when mailnews.wraplength is 0. r=mkmelin