167634 - text isn't wrapped when sending email

Status: RESOLVED INVALID

(MailNews Core :: Composition, defect)

Description

[Max Nokhrin]

When I compose email, the auto-wrap feature works perfectly. However, when I
send the message, all wrapping is lost and the text auto-wraps to screen width.
Executing Rewrap before sending fixes this issue, however, text should be sent
as it is when composing.

Comment 1

[Matthias Versen [:Matti]]

*** Bug 167635 has been marked as a duplicate of this bug. ***

Comment 2

[Matthias Versen [:Matti]]

BUILD ID ?

Comment 3

[Max Nokhrin]

Sorry :)

Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1) Gecko/20020826

Comment 4

[Todd Cohn]

Reporter, is this still happening?  Can you test on a newer build?

Comment 5

[Max Nokhrin]

Yup. I'm using BUILD Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.2b)
Gecko/20021016 at this moment. 

Comment 6

[julian]

I can confirm exactly this behaviour on Mozilla/5.0 (Windows; U; Windows NT 5.0;
en-US; rv:1.2) Gecko/20021126.

Yes, I need to upgrade to 1.2.1. But as this bug is around for a long time, and
not even confirmed here, I guess it's still unfixed.

I consider this the second most annoying issue in Mail. The most annoying being
the incredible slowness of the search facility, of course.

So please mod this bug up.

Comment 7

[Todd Cohn]

Confirming per reporters. 

Comment 8

[Max Nokhrin]

I have 1.2.1 installed now. It's still there...

Comment 9

[Markus Gebhard]

The Problem also exists in 1.3a and 1.3b. Currently I am using
  Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3b) Gecko/20030210
and it still is in there. It is deterministic so it should not be hard to
reproduce it. 

There are two workarounds:
 - Perform "rewrap" action before sending
 - "Send later" -> "Edit As New" -> "Send" (and delete duplicate)

To me (and some others) this problem is the most annyoing one in Mozilla.

My settings for mail:
 Composition: Wrap text messages at 72 characters
  (use quoted printable MIME ecncoding setting does not matter)
 Send format: 'Convert the message to plain text'

Comment 10

[Max Nokhrin]

Yes, I have 1.3b installed now and the problem still exists. The workaround is
too long for me to really bother :) 

Do you still want a before-and-after screenshot?

Comment 11

[Martin Heusel]

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030529
No wrapping in outgoing Mail :(

Comment 12

[Max Nokhrin]

Windows XP Mozilla 1.4b, still there.

Comment 13

[julian]

I can no longer confirm this bug.
Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030529

I'm not sure if it really is fixed, so I won't close.
Anyone else?

Thanks to all volunteers for their great work!

Mozilla really is a fine product.

Comment 14

[julian]

What an insidious bug. After I noticed that Markus uses the same build as I do
(only on X instead of Windows) and still experiences the problem, I decided to
investigate further. I'm doing so for two long hours now and I've been misguided
a couple of times, but I think I finally know the real problem now. Surprise:
it's Mozilla's Mail Viewer, not the Compositor!

For those in a hurry: 

This bug can be closed, wrapping of outgoing mail works fine. (Please confirm)

The fact that Mozilla concatenates text lines in plaintext mails is probably
responsible for people (perhaps including me) believing that there is something
wrong with wrapping. Not sure if this applies to the whole lifetime of this bug,
but this seems to be the case in 1.4.

Read on for details.

The following tested on Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4)
Gecko/20030529. Please confirm that you can reproduce this behaviour.

1. Outgoing Mail

- Use Edit->Preferences->Mail->Composition and set the wrap width to your
liking. Note that this setting doesn't affect any Compose windows already open.

- Compose and send a mail to yourself. Write a couple of paragraphs without
inserting hard breaks.

- Use "View->Message Source" on either the "Sent" message or the one you'll find
in your Inbox, and you should see the body properly formatted using the wrap
width you specified. So, there is no bug in outgoing mail wrapping, at least no
longer!


2. Incoming Mail

- Make sure that "Preferences->Mail->Message Display->Wrap text to fit window
width" is disabled (!!)

- For testing purposes, send a mail to yourself with a couple of paragraphs
wrapped at only 20 or 30 characters and open it using Mozilla's Mail viewer. 

- Play around with the right window border, and watch how Mozilla wraps the
short lines in your test mail according to the width of the message window.

- You should notice the following "feature": if the Window is wide enough,
Mozilla tries to concatenate consecutive lines of text when they are not
separated by an empty line!

Comment 15

[Max Nokhrin]

Confirming as per Comment 14. The weird thing is, though, that other people's
messages ARE NOT "unwrapped" - only the ones I send. Very interesting. Anyone
else confirming this? 

Comment 16

[Max Nokhrin]

Err... not "confirming" but "seeing"... :)

Comment 17

[Joergen Ramskov]

I'm seeing this too. I tried wrapping the text at 30 characters and then send a
message to myself. It did wrap the text while composing, but when it when I
recieved it, it didn't show as wrapped.

I then tried to send a similar message to an address using MS Outlook and it
arrives wrapped there.

Using "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030529"

Comment 18

[Martin Heusel]

Julian is right, the mails are correctly wrapped but expanded in the mail viewer.
The mail source viewer shows them ok.

Comment 19

[Max Nokhrin]

I guess then that this bug has to be reassigned to those working on the viewer,
not the composer.

Regarding Comment 14, I don't think that "Wrap text to window width" has to be
unchecked. From what I can tell from that description, it should only wrap text
to fit width when needed, ie when there are no line breaks in the text. It
shouldn't "unwrap" text that is wrapped. Is that a bug?

Comment 20

[julian]

Max is right, this option doesn't affect the behaviour. I was emphasizing this
only because one would expect that Mozilla leaves the text layout alone when not
told to wrap lines.

I think the Mail Viewer *removing* linefeeds is definitely a bug. No text viewer
does something similar on purpose, correct me if I'm wrong.

Now that Max, Joergen and Martin have confirmed my findings and we seem to agree
that this behaviour is at least "strange", we should either mark this bug as
invalid and open a new one, or mark it as a duplicate of the real bug, if such a
bug is already filed. I'll browse Bugzilla later to see if this is the case.

Comment 21

[julian]

Found it - we have been fooled by bug #101877 which seems to be known since late
2001. I wanted to mark this bug as a duplicate, but only the owner can do that.
Jean-Francois? 

I'm going to add a pointer to this bug to #101877 now.

Comment 22

[Mathias Nicolajsen Kjærgaard]

I think this Feature/Bug has to do with Content-Type: format=flowed
More info on this can be found here:
http://www.ietf.org/rfc/rfc2646.txt
http://www.eudora.com/techsupport/kb/1626hq.html

If "format=flowed" is removed from the content-type, the viewer shows the
message with "correct" wrappings.
I actually think that the composer and the viewer work correctly, but maybe it
should be possible to disable "format=flowed" in the composer?

Comment 23

[Max Nokhrin]

It seems that this is an invalid bug, but I will wait another few days for
anyone to make a point as to why this bug isn't invalid. If no comments are
added, I'll mark this bug as invalid. 

Regarding Comment 22, I think that this comment should also be posted under bug
101877 as well as it seems to refer more to that bug rather than this one.

Comment 24

[julian]

Comment 22 explains everything. I was misguided again. Thanks Mathias, this
should finally end the guessing!

However, let me add that Mozilla's current behaviour is NOT correct. Let me try
to put this together, I think I get it now.

This text too long? Brief summary follows in next comment.

It's in fact a bug in outgoing mail, which doesn't implement RFC2646 correctly.
It claims to be compatible by declaring the "format=flowed" header, but it wraps
to the user specified width and at the same time adds a blank to the end of each
line - therefore rendering all lines "flowed" in terms of RFC2646, though they
have been pre-wrapped. And pre-wrapping doesn't seem to be the intention of the
flowed format.

An easy way to verify the fact that Mozilla appends a blank to each wrapped
(non-wrapped too?) line is by looking at the source of a sent message and
selecting a couple of lines with the mouse - the inverted text will show a blank
at the end of each line. That's not the only proof I have though, see below.

The viewer component is correct in interpreting these lines as flowed lines
(because the header value is present and the lines end in a blank) and combining
them to paragraphs.

The relevant quotes from the RFC are:

"If the line ends in one or more spaces, the line is flowed. Otherwise it is
fixed." (4.2)

and

"A Format=Flowed message consists of zero or more paragraphs, each containing
one or more flowed lines followed by one fixed line.  The usual case is a series
of flowed text lines with blank (empty) fixed lines between them." (4.2)

and

"Consecutive [flowed] lines with the same quoting depth are considered one
paragraph and are reformatted together." (4.5)

Therefore, incoming is OK and the solution for outgoing is obvious: choose fixed
format by removing the trailing blank and/or by omitting the header, or be
damned and choose flowed format by not wrapping altogether and leaving
everything else as-is (make sure to append that blank though). 

The best solution would be to let the user choose of course. I can imagine one
radio button "Flowed format (no wrap)" and one radio button "Fixed format" which
enables the text field for entering the desired wrap width.

I prefer fixed format. Besides, this bug would be pointless if we were talking
flowed. But I guess there are reasons for flowed format as well, why else would
I use this fscking small HTML textbox.

BTW that also explains why the odd behaviour only occurs with mails generated by
Mozilla, as Max noticed. Seems like our foes implement RFC2646 better in this
respect... until now! <evil laugh>

I have checked my facts. Qmail users can inject the demo attached by doing
'qmail-inject youraccount < rfc2646demo.txt'. Don't know how it works with
Sendmail et al. Note that when you remove the "format=flowed" header, you'll see
that Mozilla Mail Viewer regards this change and doesn't interpret "flowed"
lines any longer. Again, it seems the viewer is absolutely correct.

I'll notify bug 101877 as well - thanks to Mathias' hint, it looks like "our"
bug is now more relevant than "theirs". Ok, just kidding.

Say, isn't this another demonstration of what huge a difference a tiny, even
invisible blank can make?

Comment 25

[julian]

This bug, as well as bug 101877 can be solved simply by stripping all whitespace
at the end of each line of outgoing mail. More information in comment 24.

Comment 26

[julian]

Attachment: Demonstrates that Mozilla Mail Viewer handles "flowed lines" correctly.

Qmail users can send this test mail using the following command:

qmail-inject youraccount < rfc2646demo.txt

Comment 27

[Max Nokhrin]

Very interesting. I hate these small bugs that don't seem to be bugs but really
are. They're usually very difficult to find but have amazingly simple solutions :)

Comment 28

[julian]

Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030612
Outgoing mail still mixes flowed and fixed format.

Comment 29

[julian]

Oh my... I'm only adding more and more confusion to this discussion instead of
helping to clarify the situation. After re-reading RFC 2646, it turns out I've
been mistaken once again! Fourth time or so. Go ahead and laugh at me.

Actually, now it looks to me like Mathias was right in the first place: both the
Composer and the Viewer are correct, though the Viewer doesn't expose the ideal
behaviour.

In particular, I failed to read the following quote from section 4.2:

"A series of one or more flowed lines followed by one fixed line is considered a
paragraph, and MAY be flowed (wrapped *AND UNWRAPPED*) as appropriate on display
and in the construction of new messages (see section 4.5)." (emphasis added)

and, more importantly, the following from section 4.1:

"A generating agent SHOULD:
 1.  Ensure ALL lines (fixed *AND FLOWED*) are 79 characters or fewer in length
[...]" (emphasis added)

So I was wrong when saying that in flowed format, lines shouldn't be wrapped at
all. Composer seems to be correct in regarding the user-specified wrap column
even for format=flowed messages (which makes a lot of sense for downward
compatibility.)

Furthermore, I was assuming that the viewer is correct in concatenating two
consecutive flowed lines to one line when there is enough room, and leaving the
lines as-is when the window width isn't enough.

However, while this behaviour doesn't directly violate the RFC, it's not the
ideal behaviour. The ideal (and also compliant) behaviour would be to reorganize
"flowed" paragraphs regardless of where "soft breaks" (i.e. SP CRLF sequences)
are found in the paragraph. 

In short, it looks like nothing is really broken, but Viewer should ideally
reorganize flowed paragraphs freely. And as Mathias already said, you should
ideally be able to switch between format=fixed and format=flowed in Composer.

I justed wanted to point out this my mistake. It's probably better if I shut up
now before I have to backup once again, so I'll leave the discussion for now. I
*think* I should now understand the whole situation well enough to write a
detailed "problem/solution" comment, but I thought so three times before so I'll
refrain unless someone explicitly asks me to do so.

Sorry for this mess.

Comment 30

[julian]

See bug 168420. Seems like there's a LOT to say about format=flowed.

I still think something is wrong with the system, probably the viewer. It's kind
of unnatural that the viewer concatenates only full text lines, not on the word
level. I think that's not how RFC2646 is meant.

Comment 31

[Mike Cowperthwaite]

I'm resolving this bug as Invalid, per Max Nokhrin's comment 23.

People voting for this bug are invited to check out bug 168420, which includes a 
list of the bugs that *do* exist with f=f; if one of those strikes your fancy, 
move your vote to that.  (If not, I've got a couple of pet bugs that I'd like to 
see gain a few more votes... :)

Julian: I really do not know what you're talking about in comment 30, when you 
say "the viewer concatenates only full text lines, not on the word level."  The 
viewer concatenates lines (for a f=f message, and with f=f display enabled) when 
the first line ends in a space, no matter how long either line is.  As far as 
I've been able to tell with my testing, the viewing of f=f messages works as it 
should; the problems I'm aware of are glitches in mail composition.