Closed Bug 99922 Opened 19 years ago Closed 12 years ago

Spaces at end of lines are deleted when mail are sent

Categories

(MailNews Core :: MIME, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED
Future

People

(Reporter: xn--mlform-iua, Unassigned)

Details

Attachments

(1 file, 1 obsolete file)

Mozilla Mail encodes signatures correctly, so that --[space][return] at the
bottom  of a letter will be preserved. Despite this, lines which ends with a
space ([paragraph][space][return]) will be sent out as [paragraph][return]. I.e.
the space is deleted. I have tested this on the Mac version of Mozilla, but I
suppose it is the case for all versions.
Confirming on 2001091403 Win2k:
1. create mail, body: asdhfasdf<space><cr>
2. send mail
3. receive mail
4. Press CTRL+E on mail to edit it, try to find the space
Leif, please read the bug reporting guidelines at
http://www.mozilla.org/quality/bug-writing-guidelines.html and consider using
Bugzilla Helper at http://www.mozilla.org/quality/help/bugzilla-helper.html to
report bugs. Thanks for using Mozilla!
Severity: major → normal
Status: UNCONFIRMED → NEW
Ever confirmed: true
OS: All → Windows 2000
Hardware: All → PC
Platform/OS=All/All as Leif reported the problem on Mac.
OS: Windows 2000 → All
Hardware: PC → All
Status: NEW → ASSIGNED
Target Milestone: --- → Future
A paragraph is defined as ending in a carriage return.  Internally, lines that
DON'T terminate a paragraph DO end in a space and vice versa.  This is required
for the logic of format=flowed (see rfc 2646) to work.

The way I see it, this is by design and not a bug.  However, the filer needs to
have a way of turning format=flowed off.  Turning it off should enable the filer
to use normal formatting.
Product: MailNews → Core
I've been trying to find out about a bug with enigmail (OpenPGP) signed emails that have a signature (not digital signature, normal email signature) appended to the end.

The "-- " is adjusted according to the OpenPGP dash-escaped text part of RFC 2440 to "- -- " and then I believe this bug is causing the delimiter to be changed to "- --", which is preventing non-enigmail user agents recognising the signature block.

Looking at a normal text file with "-- " clearsigned with GPG it seems alright - am I right in assuming this bug is causing this problem?
I thought this was a well-known bug - I'm a Turnpike (TP) user who has never yet seen a TBird incoming PGP (or GPG) signed message pass through a 'proper' sig-sep of <hyphen><hyphen><space><newline>.  Or a proper format=flowed (PGP or GPG) one, come to that. :(

My suspicion is that TBird users never notice it because TBird accepts '--' as a proper sig-sep, whereas TP only recognises '-- ', BICBW.  This is one of the issues which prevents many TP (now no longer under development) users to refuse to entertain TBird, so it might be worth re-visiting.

The PGP escaping of (add '- ' to) lines beginning with '-' is nothing to do with this problem BTW.
Attached patch fixes openpgp problem (obsolete) — Splinter Review
This is a patch I got fromn the enigmail newsgroup, it fixes the problem that openpgp encoded delimiters aren’t recognized.  See thread around news://news.mozdev.org:119/mailman.2517.1146163074.762.enigmail@mozdev.org.
Comment on attachment 220116 [details] [diff] [review]
fixes openpgp problem

Blake, are you the right person to ask for review here?
Attachment #220116 - Flags: review?(mrbkap)
Comment on attachment 220116 [details] [diff] [review]
fixes openpgp problem

I guess I am the closest person to the "right" person to ask here, sadly ;-).

I don't know too much about the OpenPGP protocol here, but this seems like a fine patch. The worst thing that I think can happen is that if someone manually types a line containing "- -- ", then instead of being alone on its line, it will be combined with the following line when viewed on the other end, which isn't too awful.

>--- nsPlainTextSerializer.cpp.v1.5	2006-02-23 21:56:55.000000000 +0100
>+++ nsPlainTextSerializer.cpp	2006-02-24 16:23:13.000000000 +0100
>@@ -1529,14 +1529,14 @@
>+  // In non-preformatted mode, remove any SPACE from the end of the line,
>+  // unless we got "-- " or "- -- " in a format=flowed output. We only 
>+  // check this, when it's a hard line break.

I think I'd prefer if this comment read something more like:
In non-preformatted mode, remove spaces from the end of the line for format=flowed compatibility. Don't do this for these special cases:

>+  // "-- ", the signature separator (see RFC 2646) and shouldn't be touched.
>+  // "- -- ", the OpenPGP dash-escaped signature separator in inline signed 
>+  // messages according to the OpenPGP standard (RFC 2440).
Attachment #220116 - Flags: review?(mrbkap) → review+
Here is an updated patch. Signed-of-by: jens m.noedler <noedler@web.de>

--- nsPlainTextSerializer.cpp.v1.5.0.4  2006-06-13 17:04:10.000000000 +0200
+++ nsPlainTextSerializer.cpp   2006-06-13 17:12:12.000000000 +0200
@@ -1528,16 +1528,17 @@
     // No meaning
     return;
   }
-
-  // In non-preformatted mode, remove SPACE from the end
-  // of the line, unless we got "-- " in a format=flowed
-  // output. "-- " is the sig delimiter by convention and
-  // shouldn't be touched even in format=flowed
-  // (see RFC 2646). We only check for "-- " when it's a hard line
-  // break for obvious reasons.
+
+  /* In non-preformatted mode, remove spaces from the end of the line for
+   * format=flowed compatibility. Don't do this for these special cases:
+   * "-- ", the signature separator (RFC 2646) shouldn't be touched and
+   * "- -- ", the OpenPGP dash-escaped signature separator in inline
+   * signed messages according to the OpenPGP standard (RFC 2440).
+   */
   if(!(mFlags & nsIDocumentEncoder::OutputPreformatted) &&
-     (aSoftlinebreak || !mCurrentLine.EqualsLiteral("-- "))) {
-    // Remove SPACE:s from the end of the line.
+     (aSoftlinebreak ||
+     !(mCurrentLine.EqualsLiteral("-- ") || mCurrentLine.EqualsLiteral("- -- ")))) {
+    // Remove spaces from the end of the line.
     while(currentlinelength > 0 &&
           mCurrentLine[currentlinelength-1] == ' ') {
       --currentlinelength;
Hi,

I've checked the HEAD of MOZILLA_1_8_BRANCH and saw that the
patch is still not in there, but it applies against HEAD.

If there are any reasons _not_ to apply the patch please let
me know them! :-)

Bye, Jens
Attachment #228671 - Flags: review+
Hi,

Could please somebody have and look here and apply the patch?

Thanks a lot, Jens
The fix is ready. But no one listens. And it is a destructive bug, leading to loss of data. That in itself could permit us to set the Severity to critical. But at least it I'll change it to major.
Severity: normal → major
Severity: major → critical
I am not sure if this problem of mine is related to this bug, or if it is a different one. (I am not even sure if it is related to enigmail, but that seems the most likely candidate.) 

A few mailing lists I subscribe to add some text, basically just unsubscribe instructions, at the end of received messages, as follows:

<body of message>

-- 
<sender's signature>


********** *END ENCRYPTED or SIGNED PART* **********

---------------------------------------------------------------------
To unsubscribe, <set of instructions>

In this case, the sender has signed the message with his public key. The row of dashes and all that follows is added by the list server. 

If the sender does not the message with PGP, the following problem does not occur.

When I reply with quotes, everything that has been added by the list server is retained as "new" (unquoted) text in the composer window. The signature itself, as well as the "end signed part" line, are removed, as follows:

<quoted body of original message>

---------------------------------------------------------------------
To unsubscribe, <set of instructions>


-- 
<my signature file>
I forgot to mention my OS and user agent:

Suse 9.3

Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.8.0.7) Gecko/20060911 SUSE/1.0.5-0.1 SeaMonkey/1.0.5 Mnenhy/0.7.4.0
#### Outlook Annoyances ####

Please DONOT limit a fix of this bug to mail signatures.

Another side effect of this bug is...

When you send TEXT mail to an Outlook user
Outlook by default removes all single line breaks 
between any two lines, to do its own formatting.
(unless user have unset "remove extra lines" in the 
settings from outlook tools > options)
this is OK for normal messages.
But if we are sending some message which should 
preserve line brakes, this is a big issue.


A trick to avoid this Outlook annoyances is
For every line which is more than 39 character
add 3 spaces at the end. 
But sadly I found later that my Thunderbird is 
cleaning all extra space I deliberately put on every line.


I'm a little confused by the exact behavior of Thunderbird.  One of the things I tend to do (I guess this is pretty uncommon), is my fingers type a space after every word and period, no matter what.  I end every paragraph with a period and two spaces.  I end most outline-format lines with a space.  

I can't put my finger on exactly what Thunderbird is doing to garble my messages.  Typically, it is leaving my text untouched, so it looks beautiful in incoming source.  However, as leaving it untouched, when the format=flowed formatting kicks in on the recipient side, all of my paragraphs get merged into one crazily long line.  

This is how it looks when composing:
#######################
This is a line that is long enough to be a real paragraph.  I will need to keep typing. 
I ended that last paragraph with spaces.  I wonder what happens if I put spaces and two lines. 

This is two lines down.  The last paragraph ended in spaces.  This one won't.

This is two lines down as well.  How will it display? 



This is 4 lines down after a sentence that ended in spaces.



This is 4 lines down after a sentence that didn't end in spaces. 
########################
by x lines down, I mean (x-1) blank lines.  This is how it looks in TB when it is received:
########################
This is a line that is long enough to be a real paragraph.  I will need to keep typing. I ended that last paragraph with spaces.  I wonder what happens if I put spaces and two lines.
This is two lines down.  The last paragraph ended in spaces.  This one won't.

This is two lines down as well.  How will it display?


This is 4 lines down after a sentence that ended in spaces.



This is 4 lines down after a sentence that didn't end in spaces.
####################

However, if I write really short sentences (like two words), I don't usually see this bug.  I'm not really sure why that is.  Also, the message source viewer seems to hide spaces at ends of lines, even though I suspect they are there (after reading the format=flowed rfc).  

So my idea is that the editor should do a bit of extra work.  When the user types or pastes a line that ends in space, the editor should strip those spaces before converting the message to SMTP format.  This should help to distinguish deliberate newlines from 80-character newlines.  

My issue is described better in bug 125928.  
Keywords: checkin-needed
Comment on attachment 228671 [details] [diff] [review]
don't remove the space after the signature separator for pgp signed mails ("- -- ")

I don't see any reason to believe that this patch is exempt from superreview (all the more so when it's someone other than the original patch author addressing review comments, however trivial they may be).
Attachment #228671 - Flags: superreview?(bienvenu)
I also suspect you would have had an easier time driving it in if you hadn't hijacked a bug: this bug, according to comment 0 through comment 3, is about not liking the way that format=flowed works, and wanting to be able to turn it off. A separate bug, only about handling of signature separators in PGP-signed mails, assigned to the patch author, would have been a far better place to put a patch for that separate issue.
Assignee: ducarroz → nobody
Severity: critical → normal
Status: ASSIGNED → NEW
Keywords: checkin-needed
QA Contact: esther → mime
Comment on attachment 228671 [details] [diff] [review]
don't remove the space after the signature separator for pgp signed mails ("- -- ")

thx for the patch
Attachment #228671 - Flags: superreview?(bienvenu) → superreview+
Attachment #228671 - Flags: approval1.9?
Attachment #220116 - Attachment is obsolete: true
content/base/src/nsPlainTextSerializer.cpp 1.127

I'm going to pretend that this was what this bug was actually about, and close it - anyone who can figure out what it was intended to be pre-hijack is welcome to file a new one.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Attachment #228671 - Flags: approval1.9?
Is this patch guilty and related to bug 398729 ?
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.