165077 - quoted text not separated by empty line from reply when sending text/plain

Status: VERIFIED WORKSFORME

(MailNews Core :: Composition, defect)

Description

[Brendan Eich [:brendan]]

This drives me batty.  If I reply to a message, depending on how the body of
that message is quoted, when I enter my reply text below a quoted excerpt of the
body to which I'm replying, even though I see an extra newline (making an empty
line) separating the quoted excerpt from my newly-entered reply text, when I
send as plain text, the empty line often isn't there.

From view message source I've discerned the difference.  The quote and reply run
together vertically (are missing that extra newline I crave) when the quoted
text ends line so:

>blah blah blah. 
>
reply reply reply

Note the extra space (one or more) after the period ('blah blah blah. ' if I may
quote the excerpt).  The quote and reply do not run together if there is no
trailing space.

Viewing message source, I also find a bug that I've never noticed because of how
Mozilla shows text/plain mail.  Why should there be an extra > on its own line
just before the 'reply reply reply' line?  If I were reading the message with
BSD Mail or another glass tty interface, I'd be pretty annoyed at that useless
'>' line.

/be

Comment 1

[Brendan Eich [:brendan]]

Ok, calling in editor good folks who've helped me sort out other bugs.  If this
is misassigned, please help get it reassigned.  I haven't heard a peep from JFD.

/be

Comment 2

[Akkana Peck]

Brendan: You're seeing format=flowed in action (RFC 2646).  The space at the end
of the line is a "soft line break" which indicates that the following line can
be joined to it if width permits.  It's a mail thing, not a general editor
thing, and there's a mail pref to disable f=f (maybe separate prefs for send and
receive?  I'm not sure).  Mail compose (either JF or Varada) is probably the
right owner.

There may be a related issue of what you see in the compose window not being
what you get -- perhaps the block around the quoted lines (currently a special
div) is showing extra space around it, and perhaps should have special CSS in
the mail compose window to not show any extra vertical whitespace.

Re the extra blank line at the end of the quoted text: I had a paragraph typed
agreeing with you that extra vertical whitespace was annoying and comparing it
to the contentious mail compose bug arguing whether there should be a blank line
between "someone wrote:" and the first quoted line; but then I realized that
maybe you're talking about your own replies in a message you're composing.  In
other words, you have
> a line
> 
> another line
and you put the caret at the end of the quoted blank line and hit return, and
you want the editor to notice that the line at which the quote is split is a
quoted but otherwise blank line, and delete that whole line during the split. 
If you're indeed asking for that, that would be a mail edit rules RFE, which you
should probably file to jfrancis, but cc me if you file it.

Comment 3

[Brendan Eich [:brendan]]

akkana: I'm not asking for a special edit rule, but that would be nice. I'll
leave it to you to file. I'm happy (from decades of habit) putting the cursor at
the end of a quoted line and hitting enter twice and typing my reply to that
quoted text.  But with Mozilla (and I can't believe this is due to an RFC), I
too often end up sending something that, when read in Mozilla, runs the quoted
text right up against my reply, without any blank line separating the two.

That seems like a bug.

Also, if my recipient read using a glass-tty Mail-like program, wouldn't she be
rightly annoyed at the extra > on its own line, running up against my reply?  I
did not type that, nor did I break the quote after the empty line (in the way
you surmised I may have done, which would want a special edit rule).

/be

Comment 4

[Brendan Eich [:brendan]]

format=flowed may be specified well, don't get me wrong.  What I'm saying is
that when Mozilla breaks a quoted body text chunk due to insertion, it should
not let the presence or absence of a space at the end of that quoted line make a
difference in what recipients see (or saved mail perusers see in their Sent
folders).  Nor should that extra > on its own line ugly up the works for glass
tty interface mail user agents.

/be

Comment 5

[Ben Bucksch (:BenB)]

I agree with brendan.

If I create an interleaving comment, I should be able to easily create a
vertical space between quote and my comment (usually by hitting return twice, as
brendan described). That space should sho up in editor, and the same amount of
space I see in the editor should end up in the plaintext version of the
document. If I see the quote, then vertical space (without quote bar), then my
comment, this is what should end up in the f=f mail:
=======
> quote

my comment
=======
Note the absence of the trailing space.

So, from what I understand, unless some other part (editor, parser etc.) is
screwing up here big time, this is a bug in the nsPlaintextSerializer. In fact,
I think we had that bug or a similar one before already.

What Akk wrote about f=f is also correct. The mail is indeed rendered correctly,
the trailing space brendan noticed tells the interpreter to remove the linebreak
and thus the empty line within the quote. Because there is no empty line after
the quote, you see none.

Comment 6

[Trevor Harmon]

I've seen the exact same problem that Brendan reported, and I can confirm it in
Mozilla 1.3b (on Linux). However, I cannot come up with a test case for it. I
thought that I could send mail to myself and then reply to it to reproduce the
problem, but I just can't figure out how to do that. I've tried all kinds of
options with HTML, plain text, single lines and multiple lines, but I can only
reproduce the bug on mail that I've already received. Anybody have a testcase?

Comment 7

[Brendan Eich [:brendan]]

JFD, are you gonna work on this?  If not, maybe someone willing and able can
take it for 1.4a?

/be

Comment 8

[Jean-Francois Ducarroz]

no, I won't be working on this. Reassign to ssu

Comment 9

[Ben Bucksch (:BenB)]

Trevor Harmon, Mozilla never sends real plaintext, only format=flowed (unless
you hack prefs.js). Try sending yourself an email with Netscape Messenger 4.x.

Comment 10

[Mike Cowperthwaite]

(In reply to comment #3)
> Also, if my recipient read using a glass-tty Mail-like program, wouldn't she
> be rightly annoyed at the extra > on its own line, running up against my
> reply?  I did not type that, nor did I break the quote after the empty line

So does this part of the symptom still occur for you, Brendan?  I'm interpreting 
this as meaning that, when you originally hit the Reply button, the compose 
window initialized with something like:
  > some text some text some text some text
  > blah blah blah. |Furthermore furthermore furthermore

And that you put the cursor where the | is above, and hit return twice.  In your 
compose window, you would then see something like:
  > some text some text some text some text
  > blah blah blah. 

  |
  Furthermore furthermore furthermore

with the cursor right before the first "Furthermore."  (That remainder of the 
quoted line is affected by bug 161968, but that's another issue.)  That "blah 
blah blah. " ends with a space, as stated.

However, what I'm seeing with 1.6/1.7 (and in fact, in 1.3, which I just fired 
up on a clunky old machine) is that there is no quote prefix added to the empty 
line immediatley after the quote, immediately above the cursor; neither in the 
compose window nor in the sent mail.  Since the subsequent line is not quoted, 
it does not get reflowed onto the end of the quote despite the trailing space.

Comment 11

[Ben Bucksch (:BenB)]

In 1.4 and 1.7beta, I can reproduce it with the following setup:

Reproduction:
- HTML composer (default)
- Sending format=flowed (default)
- Pref: 'Automatically quote and reply below quoted text' (default)
- Replying to real plain text (not format=flowed)
- (d) Just type a few words after the composer comes up
  (i.e. at the first line after the quote, no need to hit return anywhere)
  It looks like there is about one empty line between (not part of) the quote
  and my reply.
- Go (with the cursor) somewhere inside the quote, hit return to
  "break" the quote and enter some text to "interleave" a comment.
  Do it 2 times:
  (a) once breaking *after* a space in the quote (in the middle of a line)
  (b) once breaking *before* a space in the quote (in the middle of a line)
  (c) once breaking at the end of a line (without a trailing space)
  In both cases, it (often*) looks like there is about one empty line between
  (not part of) quote and reply, both above and under the reply.
- Send to self

* The HTML composer also behaves wierd, first putting a spurious empty line
between lines *inside* the quote (where I didn't do anything yet) and then
removing it again, but that is subject for another bug.

Original msg:
-----start-------
(a) some quoted text
(b) some quoted text
(c) some quoted text
(d) some quoted text
------end--------



Expected result (source, format=flowed):
-----start-------
Somebody wrote:

> (a) some quoted

(a) some reply

> text
> (b) some quoted

(b) some reply

>  text
> (c) some quoted text

(c) some reply

> (d) some quoted text

(d) some reply
------end--------

Expected result (display):
-----start-------
Somebody wrote:

| (a) some quoted

(a) some reply

| text
| (b) some quoted

(b) some reply

|  text
| (c) some quoted text

(c) some reply

| (d) some quoted text

(d) some reply
------end--------



Actual result (source):
-----start-------
Somebody wrote:
>(a) some quoted 
>
(a) some reply

>text
>(b) some quoted
>
(b) some reply

> text
>(c) some quoted text
>  
>
(c) some reply

>(d) some quoted text
>  
>
(d) some reply
------end--------

Actual result (display):
-----start-------
Somebody wrote:
| (a) some quoted
(a) some reply

| text
| (b) some quoted
|
(b) some reply

| text
| (c) some quoted text
|  
|
(c) some reply

| (d) some quoted text
|  
|
(d) some reply
------end--------



There are actually *2" empty lines after the message now, if I break the quote
at the end of the line and at the end of the msg, I never noticed that.

Please also note the number of spaces, also the trailing ones (visible when you
mark the next).

There are several bugs here, probably:
- Making the empty line after a quote a part of the quote instead of
  a part of the reply
- Not trimming the space at the end of the line.
- Incorrectly *un*-"stuffing" the line beginning with a space
  inside the quote
All bugs are during the sending process, the display is correct (at least it
seems so).

Comment 12

[Ben Bucksch (:BenB)]

> - Incorrectly *un*-"stuffing" the line beginning with a space
>   inside the quote
> All bugs are during the sending process, the display is correct (at least it
> seems so).

Eh, no, the last one (the unstuffing) is a bug during display time. You see it
case (b), there should be a space after the reply, before the quoted "text".
This one a minor bug, hardly noticably in practice.

Comment 13

[Mike Cowperthwaite]

(In reply to comment #11)
Nice work, Ben.  This also helped me figure out what was going on in bug 200854, 
which occurs when replying to a non-f=f message using the HTML editor and 
sending as text.

> There are actually *2" empty lines after the message now, if I break the quote
> at the end of the line and at the end of the msg, I never noticed that.

"After the message"?  Do you mean "after the quoted text" here?
Note that even in the plain-text composer, there are empty (quoted) lines 
appended to the quoted text at the end of the message -- bug 144998.

> Please also note the number of spaces, also the trailing ones (visible when
> you mark the [t]ext).

The only addenda to your "actual results" that I see:  (a)There are three empty 
lines between (d)quote and (d)reply; and the empty quote line immediately 
following (c)quote has two trailing spaces, as does the *2nd* empty quote line 
after (d)quote.  All other empty quote lines consist only of ">".


> There are several bugs here, probably:
> - Making the empty line after a quote a part of the quote instead of
>   a part of the reply

I would say that this symptom is the basic problem in this bug

> - Not trimming the space at the end of the line.

When composing f=f for plain text, quoted lines don't have the terminal space 
trimmed either.  Generally, in quotes, you *want* to preserve the space so that 
the quote will reflow.  If that space is going to be trimmed, it probably should 
occur in composition, when the user types the Enter key mid-quote.

(There is a known problem with the HTML->plain converter not trimming all spaces 
before an explicit line break in the new text: bug 125928.)

> - Incorrectly *un*-"stuffing" the line beginning with a space
>   inside the quote

The symptom from bug 200854 applies here: All of the quoted lines are prefixed 
with ">" instead of "> " -- no space -- except for the one instance where the 
line break occurred before a space.  That quoted line therefore begins with "> " 
as expected for f=f, and so there is nothing to unstuff.

Comment 14

[Ben Bucksch (:BenB)]

> Generally, in quotes, you *want* to preserve the space so that 
> the quote will reflow.

The reflow-handling is done before the HTML editor is filled with the quote -
the HTML editor can represent flowing natively, and does.

> The symptom from bug 200854 applies here

Ah, right, thanks. I was confused, my comment 12 is wrong and indeed a symptom
of that bug.

Comment 15

[Mike Cowperthwaite]

I've confirmed (and resummarized) bug 178899 which was reported on the symptom 
described in comment 11 for the (c) and (d) cases: extraneous lines being 
included in the quote *beyond* the blank line typed into the reply (or at the 
end of the quote).

Comment 16

[Mike Cowperthwaite]

*** Bug 115498 has been marked as a duplicate of this bug. ***

Comment 17

[jonathan.courtney]

*** Bug 181170 has been marked as a duplicate of this bug. ***

Comment 18

[Mike Cowperthwaite]

Behavior as described in comment 11 has changed somewhat (but is still not 
right) probably due to the patch for bug 144998.

Actual result (source):
-----start-----
Somebody wrote:
> (a) some quoted 
(a) some reply
> text
> (b) some quoted
(b) some reply
>  text
> (c) some quoted text
>   
(c) some reply
> (d) some quoted text
>
>   
(d) some reply
------end------

Actual result (display)
-----start-----
M Cowperthwaite wrote:
| (a) some quoted 
(a) some reply
| text
| (b) some quoted
(b) some reply
|  text
| (c) some quoted text
|   
(c) some reply
| (d) some quoted text
|
|   
(d) some reply
------end------

Differences: zero (not one) blank quoted lines after the reply header and after 
the (a) and (b) breaks;
one (not two) blank, quoted line after the (c) break;
two (not three) blank, quoted lines after the (d) end of text; and
zero (not one) blank unquoted line after any of the reply texts.

Further: the missing space character between quote and text (symptom from 
bug 200854) is no longer missing.  This is also reflected in the quoted lines 
following the (c) break and (d) end of text: one of the quoted blank lines in 
each case now contains three (instead of two) spaces.

Comment 19

[Mike Cowperthwaite]

Note that if the original message is format=flowed, the text generated at breaks 
(a) and (b) is now identical to that for a non-f=f original message.  This is 
also different: it used to be that, replying (as HTML) to an f=f message, the 
sent-as-plain-text maintained blank lines on either side of the entered text.

However, at (c) there are no quoted blank lines (just like (a) and (b)); at the 
end of text (d), there is one quoted blank line, with no extraneous spaces (just 
as in the plain text composition).  In fact, the lack of blank lines surrounding 
the breaks is almost identical to that with plain-text composition (except that 
plain-text drops the quote-status of the end of a line that gets broken -- 
bug 161968).

I started looking into this because there have been some complaints, presumably 
due to the now-overly-aggressive blank-line removal in the HTML-reply-to-f=f 
case.
  <http://forums.mozillazine.org/viewtopic.php?t=297219>
I haven't found a bug opened about this (yet).

Comment 20

[Mike Cowperthwaite]

If it is accepted that this bug is about the (a) and (b) cases in comment 11, 
while bug 178899 is about the (c) and (d) cases, then I think this bug should be 
WFM'd, as a result of the fix for bug 144998.  

It's true there are no longer blank lines auto-inserted between quote and the 
text, when you type <enter> in the middle of a quote -- but (other than the 
margin/padding issue discussed at bug 307112) there is no indication of any such 
blank lines, either; and if you *type* the blank lines in, they properly get 
included in the HTML->plain conversion.

Comment 21

[Mike Cowperthwaite]

No response from reporter => WFM

Comment 22

[Ben Bucksch (:BenB)]

WIth Thunderbird betas, I get *no* empty lines at all between quotes anymore, which is a *major* irritation and worse than before. Dunno which bug report that would cover, but this one seems to have a nice analysis and summary, so REOPENing.

Comment 23

[Mike Cowperthwaite]

RE-WFM'ing.  Immediately after reopening this one, Ben opened bug 314213 for what he was complaining about in comment 22.

Comment 24

[Ben Bucksch (:BenB)]

ok