Open Bug 168420 Opened 21 years ago Updated 1 month ago

[meta] Format=Flowed (f=f) UI Visibility / Evangelization

Categories

(MailNews Core :: Composition, enhancement)

Product:
MailNews Core
Component:
Composition
Type:
enhancement

Tracking

(Not tracked)

People

(Reporter: holgermetzger, Unassigned)

References

(Depends on 11 open bugs,
URL
)

Details

(Keywords: meta)

Attachments

(1 file, 4 obsolete files)

Format=flowed mini-FAQ
15.14 KB, text/html
Details 
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.0.1) Gecko/20020823 Netscape/7.0 (Compact)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.0.1) Gecko/20020823 Netscape/7.0 (Compact)

One of the things that most newbies do not understand is Format=Flowed. "Mozilla
doesn't wrap at 72 chararcters" is a FAQ. Many new users of Mozilla/Netscape
really are confused why Mozilla shows line breaks in the editor but seems to
forget line breaks when the message has been sent. For example see
<alrr33$59j1@ripley.netscape.com>
I get this question all the time (http://www.hmetzger.de/etips6.html#32) and
many newbies think that Mozilla has a serious line break bug, many of them even
start breaking lines manually.
I filed bug #86607 a while ago, but if this is not an option then we absolutely
need some kind of Format Flowed Evangelization. I checked the Mozilla help and
couldn't find any explanation for format=flowed anywhere.

Reproducible: Always

Steps to Reproduce:
1.
2.
3.
Also see bug 71110.

pi
Status: UNCONFIRMED → NEW
Ever confirmed: true
I agree that it would help greatly if users had some way to find this
information.  I see lots of bugs filed simply because users don't understand
what's going on, and think there's wrapping when there isn't, or vice versa.
I don't think it is the user's fault who don't understand. I believe it is the
poor usability of Mozilla's implementation (very long lines are hard to read) of
f=f combined with several bugs plus breaking compatibility with real plain text
in the design of f=f.

So there are some things to do:

1) Fix those highly visible bugs (like doubling empty lines under quotes, bug
144998, indenting > without showing it beforehand, bug 114954).

2) Display f=f in a resonable fashion (bug 71110) or simply display lines as
they are (actually, the only reasonable use of f=f is rewrapping).

3) Give users a choice (bug 86607).

pi
More bugs:

Bug 26734 format=flowed message generation must follow CJK line-breaking conventions

Bug 104348 Editing draft (or other plain text message as new) loses flowed
quoting information

Bug 121297 Attachment with format=flowed is displayed with <div>

Bug 141095 Paragraph breaks in format=flowed MIME text not displayed properly.

Bug 155015 format=flowed and lines with 2+ spaces at the end

Bug 155622 Wrapping information lost on "Edit as new", "Edit draft" (RFC 2646
format=flowed)

Bug 175953 Edit message as new ignores format-flowed

Shouldn't we mark all those (and those from the previous comment) block this bug?

pi
Attached file Format=Flowed Mini-FAQ (obsolete) — 

    
    

    
  
Suggestions:

Q: Those long lines are poorly legible, what can I do?

Q: What problems are known for Mozilla's f=f implementation (AKA known bugs, see
my comments above)?

pi

    
    

    
  

      
      
      
      

        Attached file
          
        Format=Flowed Mini-FAQ (obsolete)
        — 
      

    


  

        Attachment #115235 -
        Attachment is obsolete: true

    
    

    
  
Very good Mini-FAQ!

One more question (took me quite a while and many bug reports to get it):
Q: What does "f=f" mean?
A: format=flowed

And one more bug, seems a bit different although related, that is why I filed it
as new. See Bug #201280 .


    
    

    
  
Yet the bug in comment 8 is unconfirmed. For the time being I'll add the other
bugs do dependencies, since nobody complained to my suggestion.

Actually, this here does also not look like something that could be fixed, hence
adding the meta keyword.

pi
Depends on: 26734, 71110, 86607, 104348, 114954, 121297, 141095, 144998, 155015, 155622, 175953
Keywords: meta

    
    

    
  
Great FAQ. I agree we need to clear up the confusion about f=f.

    
    

    
  
I just don't see it mentioned (and cannot find the appropriate bug right now),
but it should be stated clearly: If you apply rewrap to quotes those are space
stuffed  and hence not recoginzed as quotes at the receivers end.

This is partially bug 114954, but there is another bug which says that rewrap
loses the quote status.

pi

    
    

    
  
Bug 161968 is the other bug. This is highly visible for people using f=f, please
add it to the FAQ.

Removing dupe bug 175953 from dependency, please also remove from FAQ.

pi
Depends on: 161968
No longer depends on: 175953

    
    

    
  

      
      
      
      

        Attached file
          
        Format=Flowed Mini-FAQ (obsolete)
        — 
      

    


  

        Attachment #115240 -
        Attachment is obsolete: true

    
    

    
  
As this bug is something of a reference point for finding other bugs related to 
"format=flowed", here are two related bugs.  They do not relate to the 
principles of "format=flowed" - but are two message altering bugs which only 
occur with "format=flowed".

The first one is a year old and has stopped me and other people using Mozilla 
for mail.  It was was not thought to have anything to do with "format=flowed" 
until today when I found it only occurs when sending with "format=flowed".  

1 - Bug 141983  "Space added to indented line when sending or saving message" 


The second I just discovered today - an equal and opposite bug when reading or 
printing a "format=flowed" message which removes one space from indented lines!

2 - Bug 204437 "Format=flowed gobbles 1 space in indented lines"


These document my user.js workarounds to turn off both parts of "format=flowed". 
 Now, at last, I think I am going to start using Mozilla for email and therefore 
for browsing - though it will be Netscape 7.02 because I need the spellchecker.

 - Robin      http://www.firstpr.com.au/web-mail/Mozilla-mail/



    
    

    
  
As I read it those are not bugs, since it is a "feature" of format flawed. One
more reason it is BAD. It will not display properly for readers which don't care
for f=f. So we might add a warning to the FAQ that f=f should not be used since
it produces incompatibilities.

pi

    
    

    
  
As far as I am aware these two bugs I mentioned above are not what format=flowed 
is supposed to do.  But even if format=flowed does what I understand it should 
do, then I think it should be off by default (both in terms of sending and in 
terms of interpreting format=flowed messages differently from any other message) 
since in both cases the purpose is to display the message differently from the 
way the sender wrote it.

We should ensure plain text email is, by default, absolutely transparent - so 
that what the sender sees when they write it, character for character, with 
their exact positions and "> " quoting etc. intact is precisely what the 
recipient sees on screen, when copying to the clipboard and in print.  (People 
with screens of less than 72 or so characters will be well aware that they can't 
see most messages as they were written.  Since those people are in the minority 
and are well informed about the limitations of their own gear, I see no 
justification for changing the default sending arrangements for everyone's mail 
clients to supposedly make life easier for these people with mobile devices.)

If *both* sender and recipient understand what "format=flowed" means and choose 
to use it, then that's fine.  Arguably the "format=flowed" read function should 
be on for narrow-screen devices, since ordinary messages can't be displayed 
verbatim anyway - so it can't mess up messages accidentally sent 
"format=flowed" by making the lines excessively long.  But it is reasonable to 
assume that Mozilla always runs with windows which can display 72 to 80 
characters of text.

I think both features should be off by default and there should be two user 
interface options, next to each other, with clear explanations of what they are 
for.   I am surprised that some people think otherwise and am particularly 
surprised at the flak I have copped on another bug for stating this.   I do not 
think it is OK for programmers of a major program like an email client to decide 
that certain alterations to messages are acceptable or desirable as default 
behavior for other people.  As noted in another bug, even supposedly cosmetic 
things like the vertical quote bars stuff up the ability to read technical 
emails containing .diff files.  Its not possible to anticipate all the uses of 
email sufficiently to say that there will be insignificant user impact from any 
system which makes the received message different from the sent message.  This 
is particularly the case when the changes are invisible to the sender, and the 
receiver probably has no idea that the sender wrote something different to what 
they see.

Maybe there needs to be an option for sending format=flowed to particular email 
addresses where the recipient is believed to be reading on a mobile device with 
a small screen.  

If all emails which came in with "format=flowed" were those where the sender 
specifically to set this, and if the default behaviour for the recipient's 
"format=flowed" reading system was to display it at a width they liked (rather 
than too wide in a large window) then it would be OK, or at least not as bad, to 
have "format=flowed" on by default when reading and printing messages. (But 
again, there needs to be a margin set for printing, which might be different 
from the screen margin, because it is generally hard to read if the lines on 
paper are too long, with correspondingly less room on the right for 
annotations.)

Now we have a significant number of clients (at least all the last year or 
more's Mozilla and Netscape) in use, so the majority of messages with 
"format=flowed" which exist today result from this default, and not from the 
sender's well-informed choice.  This situation is a further argument for making 
the read aspect of "format=flowed" off by default.

Because for some or many clients it is on by default when reading, and people 
are sending messages with it on by default, and with virtually all senders and 
recipients firstly being unaware of what is happening and secondly being unable 
to change it or even know where to ask for help, I think the current 
format=flowed arrangement is all wrong. 

I think the answer is not to explain why the current arrangement is good - but 
to change to default off for both send and receive and then give user options 
with a good, potentially evangelical, explanation of the benefits of 
"format=flowed".

  - Robin

    
    

    
  
> We should ensure plain text email is, by default, absolutely transparent
[blablabla]

We have argued about that in the past, and I explained that this is
*impossible*. Not even ordinary plaintext does that. Please keep the general
discussion, if f=f is evil or a world(eh, email)-saver to the newsgroups, as I
*also* explained to you previously.

    
    

    
  
Ben correctly pointed out that my new bug 204437 is not a fault in the 
implementation of "format=flowed" - this is how it is supposed to work.

Burpmaster mentioned the "format=flowed" RFC on 30 May last year in bug 141983 
but "format=flowed" was not mentioned in that bug until today.  If we had 
understood the full meaning of his message, we would have realised that this is 
not a bug in the serializer, DOM to text conversion etc. but a necessary (and to 
some people entirely unacceptable) additional space in all indented lines 
whenever they are sent with "format=flowed".

Please see bug 141983 for how this is still a serious problem, at least as long 
as "format=flowed" is on by default, and for my refutation of Ben's position 
that plaintext email is not transparent.  The problem he refers to is turning 
lines starting with "From " into ">From ".  This is not caused by email or any 
Internet standards - it is a known problem with Mbox mailboxes and is one of the 
reasons why most servers use Maildirs instead.  Lines starting with "From " are 
are much rarer occurrences then indented lines.

 - Robin

    
    

    
  
Quoth the attached Mini-FAQ:

> How does Format=Flowed affect Non-Format=Flowed Readers?
>
> Mail/Newsreaders not capable of displaying format=flowed 
> messages are not affected at all, because the only difference 
> for a simple mailer is the extra single space at the end of 
> each line which should not disturb anyone. 

This is plainly untrue: format=flowed breaks inline patches and any attempts at
manual formatting.

The mini-FAQ could also do with a prominent way to disable format=flowed.

    
    

    
  
Until yesterday I had always assumed that sending with 
"format=flowed" would have no noticeable impact (for human 
readers) on the message, either in the raw text in the 
email, or when it was displayed by any MUA which did not 
implement "format=flowed" display algorithms.  I knew it 
did some stuff with spaces, but I thought that this was 
just at the end of the lines.  I never really looked at
how it worked, because I don't want to use it.  I had no
idea it was the cause of bug 141983.

It seems that some or all of the proponents of
"format=flowed" here on this "bug" also thought that it
had no discernible impact on messages.  Otherwise the
Mini-FAQ would not say:

> Mail/Newsreaders not capable of displaying format=flowed 
> messages are not affected at all, because the only difference 
> for a simple mailer is the extra single space at the end of 
> each line which should not disturb anyone. 


Please refer to the detailed discussions in the last day 
or so on bug 141983.  There, on 30 May last year,
Burpmaster referred to how "format=flowed" reading removes
a space.  I wish I had paid more attention to his comment
because then I would have realised that the sending 
algorithm adds a space.


The "format=flowed" RFC has one author - someone from the 
company which produces Eudora.  It is an Internet 
Standard, and I assumed Internet Standards were always 
wisely designed to be backwards compatible - but
"format=flowed" mangles messages with indented lines for 
all non-compliant MUAs or any other system which uses the 
message, such as a program to verify a digital signature.

It is beyond doubt that this insertion of a space for 
every indented line is a necessary part of the 
"format=flowed" sending algorithm. One proof of this is 
that Eudora does it.  Another is that removing the extra 
spaces causes the "format=flowed" reader to create the
wrong results.

So most of us have been under an illusion for a year or 
more.  Most of us did not know that "Format=flowed" 
mangles emails with indented lines unless they are 
interpreted by a compatible piece of software.  

Whatever its other benefits and costs (such as 
potentially unwanted very long lines when reading on 
screen or printing) of both sending and reading with
"format=flowed", I think that the problems of sending -
the way this changes point-to-point one-way 
communications in ways which are typically invisible 
to the sender, and disruptive or completely unworkable 
to the recipient, without the recipient or the sender 
realising that there is a difference between what 
the sender wrote and what the recipient sees, makes it 
unthinkable that sending "format=flowed" should be on 
by default in Mozilla.

I think we need not evangelization, but realism, technical 
rigour and openness to the problems experienceed by users.

It seems that this evangelistic mode of thinking has caused 
some, most or all of those who have pushed for 
"format=flowed" to be in Mozilla, for it to be on by default, 
and for it not to have a user-interface option, to be unaware 
of a basic fact of the algorithm's operation.

Ben's support for the Mini-FAQ a few days ago indicates that
he too thought that the only changes to the message were in 
spaces on the end of the line - and it seems that he and/or
Daniel Brattell wrote the code to implement it!

   http://www.mozilla.org/editor/serializers.html

> External Contributors:
>
>   Daniel Brattell and Ben Bucksch contributed a lot of code to 
>   the plaintext serializer due to its importance to mail.  Some 
>   of their contributions: format=flowed wrapping, 
>   bold/italic/underline/smiley substitution, html-lookalike 
>   formatting (lists, indentation). 


  - Robin 

    
    

    
  
I agree to the previous comments. Here is my suggestion for the FAQ:

How does Format=Flowed affect Non-Format=Flowed Readers?

Mail/news readers not capable of displaying format=flowed messages (or not
willing to, see e.g. the question "poorly legible" below) will not see exactly
what you wrote. An indented line will be further indented by one space, so any
manual alignment will break. Quote symbols (>) at the beginning of a line will
also be indented by a blank.

Furthermore, if the person answering your mail/posting uses f=f the lines might
be rearranged, so that intended alignments will break. Clearly, this affects
users without f=f enabled, but also people reading with it will see different
alignments than those you intended.

Also note that by the bugs listed below several things might happen, e.g., blank
lines are added in the middle of you message or quotes are not sent as quotes.

pi

    
    

    
  
> How does Format=Flowed affect Non-Format=Flowed Readers?

Maybe the FAQ should also mention the way F=F mangles ascii drawings for non-f=f
readers: lines that starts with a space receive extra indentation, serieusly
disrupting the alignment of the drawings.

This is what made me turn it off. Looking at my usenet postings using google,
the drawings always were mangled. Precompensating for the extra indentation made
my drawings look bad for f=f readers. The only way to fix it permanently was to
turn f=f off, and I never missed any of its functionality, and even more, its
behaviour around quoted text etc. :-)


    
    

    
  
Effects on non f=f readers:
- Lines starting with a space or with > (not quote) get an additional space
- Most lines end with a space (hardly noticable)
f=f readers reproduce the message exactly, even in cases where real plaintext
doesn't.
(That list doesn't consider bugs in Mozilla's implementation.)

> Furthermore, if the person answering your mail/posting uses f=f the lines
> might be rearranged, so that intended alignments will break

Not true, to my knowledge. If you have intended alignments, insert a hard
linebreak (press return) and that will be represented in the mail, so that f=f
readers do *not* make that line "flow".

    
    

    
  
>f=f readers reproduce the message exactly, 

No, they may change the line wrapping. This is likely to happen with quotes
since the line length changed.

>even in cases where real plaintext doesn't.

What do you mean? What does text/plain software change?

>> Furthermore, if the person answering your mail/posting uses f=f the
>> lines might be rearranged, so that intended alignments will break
>
>Not true, to my knowledge. 

If not, it is a bug. That's the whole idea of that concept.

>If you have intended alignments, insert a hard
>linebreak (press return) and that will be represented in the mail,
>so that f=f readers do *not* make that line "flow".

Experience shows that most users are neither aware of the fact that they use f=f
nor of the consequences.

pi
URL: http://www.ietf.org/rfc/rfc2646.txt

    
    

    
  
>> f=f readers reproduce the message exactly, 
> No, they may change the line wrapping.

The reader doesn't change anything. The sender (software) explicitly states that
the paragraph may be rewrapped. The recipient does that. That's exact
reproduction, no alteration at all. You don't complain about HTML pages
rewrapping either, do you?

As for the sender, it's quite obvious in the composers (at least the HTML
composer) that lines reflow. If they do while composing, it's not to be expected
that they will be static once they are sent (unless you have prior knowledge
about old-style plaintext email and inappropriately apply that to Mozilla). This
can be observed with the copy in the Sent folder as well.

> What do you mean? What does text/plain software change?

See bug 141983, comment 31, 34, 36 and probably others and an old newsgroup
thread on .mailnews about f=f (opened by Robin).

    
    

    
  
>As for the sender, it's quite obvious in the composers (at least the HTML
>composer) that lines reflow.

If you have been using E-mail/Usenet since before WWW (like I have), using HTML 
in E-mail/Usenet is a sure way to alienate your (oldtime) readers real quick.
HTML in Usenet postings is flamebait, simply, nothing else.
So, please don't use the HTML composer as a measure!

> If they do while composing, it's not to be expected that they will be static
> once they are sent (unless you have prior knowledge about old-style plaintext
> email and inappropriately apply that to Mozilla).

WYSIWYG wordprocessers have continuous linewrapping, yet most people expect 
them to print the text as it looks on their screens, not smaller or wider. When 
you have E-Mail, Usenet and wordprocessing programs predating Mozilla by a big 
margin, the expected behaviour will be dictated by prior knowledge.
And don't expect people to read the (new object's) documentation...

> This can be observed with the copy in the Sent folder as well.

(Won't the message display have the same width as the composer?)
It's kind of useless to be able to see how your message will come out *after* 
you sent it, and I doubt many people will postview their messages for the 
layout.

The fact of the matter is, f=f is neither widespread nor backward compatible, 
so it breaks the formatting for a whole lot of readers. Even if mozilla can 
force senders to create f=f messages, it still can't force recipients to use an 
f=f enabled viewer. People who require backward compatibly (or fixed 
formatting) need an easy way to disable f=f.


    
    

    
  
> HTML in Usenet postings is flamebait

eh, I didn't suggest to *send* HTML. I talked about using the HTML *composer* to
send f=f plaintext, which is the default for email. The lines break at the
window border. Resize the composer and the lines will reflow. That's exactly how
the reader sees it.

    
    

    
  
ben.bucksch@beonex.com wrote:

>>> f=f readers reproduce the message exactly, 
>>
>> No, they may change the line wrapping.
>
>The reader doesn't change anything. 

But his software might.

>The sender (software) explicitly states that
>the paragraph may be rewrapped. 

And the sender usually does not know that. That is what the FAQ addresses. f=f
is as far away from WYSIWYG as you can get. That is not understood by most users.

>The recipient does that. That's exact
>reproduction, no alteration at all. 

It is an alteration to what the sender has seen on his screen and *expected* to
go out.

>You don't complain about HTML pages rewrapping either, do you?

That is completely different. For web pages you do expect that. In MailNews you
don't. You expect quotes to look exactly as the quoted message plus the quote
symbol.

>As for the sender, it's quite obvious in the composers (at least the HTML
>composer) that lines reflow. 

But only for that (do the quotes also flow BTW?). Many people use plain text
editor, because there is no need whatsoever for that editor when sending only
text/plain as most people do for very good reasons. Let me say it plain:
HTML-sending is usually conceived as luser-style.

>If they do while composing, it's not to be expected
>that they will be static once they are sent 

If you think you chose text/plain in the preferences, you do expect text/plain.

>(unless you have prior knowledge
>about old-style plaintext email 

That has nothing to do with old-style.

>and inappropriately apply that to Mozilla).

How will you know? If people knew they would not be surprised all the time by
those effects.

>> What do you mean? What does text/plain software change?
>
>See bug 141983, comment 31, 34, 36 and probably others and an old newsgroup
>thread on .mailnews about f=f (opened by Robin).

I see that you claim period, comma and the quote symbol would be altered by
plain text. Simply not true. So again: What is changed with text/plain?

pi

    
    

    
  
> Simply not true

If you argue on this level, I'm out here. I am so sick of this. The FAQ was
objective, please keep it at that. Add the limitations, e.g. as I stated in
comment 23, but please keep the "text/plain forever" out.

> For web pages you do expect that. In MailNews you don't.

*I* expec them both to be exactly the same. Guys, you grew up with plaintext on
Usenet, but the vast majority of users didn't, so your expectations are not
universially shared. I believe that f=f meets today's users and more importantly
today's requirements better than plaintext. We don't live in a world with 80
chars terminals anymore.

    
    

    
  
(That said, I respect anymore who prefers plaintext. Just as I respect anymore
who prefers HTML. As long as we can still communicate and you don't force your
formattign on me. Peace ;-P )

    
    

    
  
> ... you don't force your formattign on me.

Isn't this exactly what mozilla is doing with f=f?

It's hard enough to explain to new users why Usenet postings in HTML are a bad 
idea, how can you explain to them that mozilla's plain text option is not plain 
text either, but requires another f=f enabled viewer to see it as he/she 
intended.

It's very convenient mozilla provides linewrapping, but neither my standalone E-
mail client nor Google Groups do, and paragraphs on a single line are just 
plain unreadable.


    
    

    
  
[I see that you claim period, comma and the quote symbol would be altered by
plain text.]
>> Simply not true
> 
> If you argue on this level, I'm out here.

I see, you claim something and don't show how it would happen. text/plain allows
sending those characters, so if you think otherwise I am waiting for a proof.

> The FAQ was objective, please keep it at that.

What in comment 21 is not objective?

> Add the limitations, e.g. as I stated in
> comment 23, but please keep the "text/plain forever" out.

That was never suggested, please stay with the facts.
 
>> For web pages you do expect that. In MailNews you don't.
> 
> *I* expec them both to be exactly the same. Guys, you
> grew up with plaintext on Usenet, but the vast majority
> of users didn't, so your expectations are not 
> universially shared.

I see it every day with many users.

> I believe that f=f meets today's users and more importantly
> today's requirements better than plaintext. We don't live in a world with 80
> chars terminals anymore.

This could be easily solved with endless lines where the reading software wraps
appropriately, but this is not the question. This bug here is about coping with
f=f. It needs to explain the problems which are not at all obvious to the user.
Typical things I see every day are quote which are marked by " >" instead of
">". That is what the userer typed and what he expects to be sent. Or unwanted
indentations. Or underlines with ~~~~ which moved. Basically all failures of
WYSIWYG.

pi

    
    

    
  
(Of course my last paragraph does not apply to f=f when used properly, but 
comment 22, 23 and 24 list enough noticable effects.)

    
    

    
  
> > ... you don't force your formattign on me.
> Isn't this exactly what mozilla is doing with f=f?

Au contraire, it gives me (the reader) the choice of the window width and
formatting of quotes, without ambiguosity / false hits.

> paragraphs on a single line are just plain unreadable.

Agreed. (But f=f doesn't do that, as you said.)

> you claim something and don't show how it would happen

I explained it back then on the newsgroup extensively. I am not going to reopen
the debate in this bug *again*, that's why I won't explain it here. Read the
archive, if you don't believe me.

> Typical things I see every day are quote which are marked by " >" instead of
> ">".

Implmentation problem. Doesn't appear at all in the HTML composer (it has a
Preformat mode when you need it, BTW). (Unless you do Message Edit as new, which
is again an implementation bug.)

> What in comment 21 is not objective?

"will not see exactly what you wrote" is a vast overstatement IMHO, if the only
problem with f=f (not our implementation) is to add a single *space* when there
is already one, only for non-supporting mailers, and even for that is a known
workaround.

Rewrapping is not a problem, as stated. Nothing gets rewrapped, if you don't
want to. Nobody with the slightest clue creates an ascii table by hitting space
until the line autowraps. In any case, f=f allows the sender to explicitly state
what can be rewrapped and what not, so it's an implementation problem in any
case (if any).

Implementation problems belong to the bug list and workaround tips in the FAQ,
not f=f advocacy.

(Why the heck am I still talking?)

> Or underlines with ~~~~ which moved.

Indent the underlined line by one space and it will work. Maybe the fix for bug
141986 will do that automatically. Until then, the FAQ could mention is as
non-obvious workaround.

    
    

    
  
> > > ... you don't force your formattign on me.
> > Isn't this exactly what mozilla is doing with f=f?
>
> Au contraire, it gives me (the reader) the choice of the window width and
> formatting of quotes, without ambiguosity / false hits.

So, it seems to me that f=f is all about formatting... :-)

> "will not see exactly what you wrote" is a vast overstatement IMHO, if the
> only problem with f=f (not our implementation) is to add a single *space* when
> there is already one, only for non-supporting mailers, and even for that is a
> known workaround.

I'm still very uncomfortable with the way you dismiss the spacing problem (in
ascii tables, ascii drawings and the like) as being a problem of "non-supporting
mailers", I do not have a choice as to which viewer my recipients use (and I
really shouldn't have to care).

However, I'm interested in the known workaround you mention, it doesn't seem to
be in the FAQ...


    
    

    
  
The workaround is quite important, so I'll spell it out here again:

When sending multiple text lines that should be in some way aligned: if any of
the lines begin with a space, then all the aligned lines must begin with a space.

    
    

    
  
> When sending multiple text lines that should be in some way aligned:
> if any of the lines begin with a space, then all the aligned lines
> must begin with a space.
  ~~~~

Which does not work if aligned to something quoted.


>> What in comment 21 is not objective?
>
> "will not see exactly what you wrote" is a vast overstatement IMHO,

Hm, it is mathematically the most precise way to state it.

> if the only problem with f=f (not our implementation) is to add a single
> *space* when there is already one, only for non-supporting mailers,

Once more: This is also happening to > and the word From.

And again: If you write a paragraph flowed, the person who answers also uses
format=flowed but assumes everything is fixed and aligns something, that might
badly fail due to a new rewrapping later. You call this a user error. But the
user is always right.

pi

    
    

    
  
Ben, you wrote:

> "will not see exactly what you wrote" is a vast overstatement IMHO,
> if the only problem with f=f (not our implementation) is to add a
> single *space* when there is already one, only for non-supporting
> mailers, and even for that is a known workaround.

I am exasperated by your repeated failure to acknowledge the 
seriousness and pervasiveness of the message corruption which results
from your preferred position of default on sending with "format=flowed".

The debate now is different from that of a few days ago because we now
know (bug 141983) that when a message is sent with "format=flowed" and
viewed or processed on any system which does not have a "format=flowed"
decoding algorithm, then:

1 - All lines starting with a space will have a space added.

2 - All lines starting with a ">" will have a space added.

3 - (As far as I know) lines with multiple trailing spaces
    will have them deleted.

On the other bug we have debated some of the impact of this, such as on
archived email and Usenet postings, digital signature checkers, ASCII
diagrams etc.

In a technical, byte-for-byte sense, these are gross changes to most
messages.

In a user-perception sense they are very often noticeable, gross or
functionality degrading changes.

All this is news to you, me and everyone else - except perhaps
Burpmaster - since I discovered this a few days ago.

Your old arguments for "format=flowed" being on by default never
convinced me, but they are largely irrelevant now to anyone who still
believes that it should be on by default, because the changes listed
above are serious, pervasive, cryptic (in the sense that the sender may
not realise their message is being garbled and the recipient may not
realise that the sender really wanted to send something different from
what they received) and go far beyond the scope of what was previously
discussed: the behaviour of "format=flowed" messages in MUAs with
"format=flowed" decode algorithms.

By any measure these are significant costs you are imposing on millions
of users and those who read their messages now and in the future.


"will not see exactly what you wrote" uses the word "exactly"  There are
no gradations or degrees of "exact".

Any difference at all means the message is not exactly the same.  Yet
you continue to avoid the reality of this, and the costs your position
imposes on millions of people, by characterising lack of exactitude
as an "overstatement".

I can't recall anyone in any technical debate I have been involved in
since about 1996 being as resistant to changing their position. Then it 
was a minor technical principle regarding software sound synthesis.  
But the issue here is whether or not Mozilla, by its default 
configuration, should disrupt unknown millions of emails of unknown 
millions of Mozilla users and the unknown millions of people who read 
or technically apply their messages - now and into the indefinite 
future.

As of a few days ago, we now know that your idea of sending with
"format=flowed" on be default stuffs up the messages of many Mozilla users.


Please answer the question:

  How can the mangling of plain-text messages for many ordinary
  users, who have no interest in, knowledge of, or benefits from
  "format=flowed" be justified by whatever arguments you have for
  keeping it on by default?


As I have argued elsewhere, I think its a big mistake to use the HTML
composer when sending plaintext messages.  I think its bug that this is
the default.

Knowledgeable people use the plaintext editor - which I am finding to be
a really good thing (now that I am using Mozilla / N7.02 all the time
thanks to a user.js to turn of sending with "format=flowed").  So it is
not to the point to argue about the experience of people who use the
HTML editor.

Those who use the plaintext editor have a right to expect that the
message the send is exactly what they see on the screen - irrespective
of whether the newlines are automatic or manual.

Having "format=flowed" sending on by default violates that expectation
in many instances, in a cryptic manner.

It is not good enough to argue, as you have, that plain-text mail
already has degradations because this is no justification for
introducing more, because the ">From " one is relatively minor and rare
(and nothing to do with email, just lousy mailbox implementations and
because the problems you mention with lines starting with "." and "," 
do not seem to exist.

   - Robin

    
    

    
  
> Once more: This is also happening to [...] the word From

Just as in plaintext, where the software usually adds a >, which is arguably
much worse, because it looks like a quote or at beast garbage. A space is - in
comparison - neglectable.
(And, yes, software *does* add it. Mozilla adds it in every case, IIRC. UW-IMAP
does. sendmail does, IIRC. Extremely popular software each.)

> - All lines starting with a ">" will have a space added.

If they are not a quote, then that *must* be the case, because > is the quote
character - by convention in real plaintext and by spec in f=f. Anything else
would be ambiguous or wrong, depending on how you see it. With f=f, a reader
supporting f=f will be able to differentiate between a quote and a line starting
with >, and the latter will appear exactly as the user typed it.

I hope you can see the value of being able to exactly represent what is a quote
and which line just happens to start with a quote. If you don't, then all
discussion is meaningless, because our values are too different. For me, it's
the meaning that matters. For me, text/plain loses information here.

> Any difference at all means the message is not exactly the same.

Right. And that's why this whole discussion is so pointless, because text/plain
doesn't *exactly*, byte-for-byte, transfer the message either, for exactly the
same reasons as f=f.

> As I have argued elsewhere

Yes, and please keep the discussion *there* and don't reopen it here. What a
waste of time.

> Knowledgeable people use the plaintext editor

haha. *shaking head* I guess the developers are all not "knowledgeable", then.

> the problems you mention with lines starting with "." and ","
> do not seem to exist.

I have never said anything about "," (to my knowledge). As for ".", I was wrong.
I was thinking about SMTP (RFC 821: "SMTP indicates the end of the mail data by
sending a line containing only a period."), but it seems that SMTP has a
provision to keep that ".", namely by adding a "." to every line starting with a
".", and later removign the first "." on a line. Exactly the same that f=f
specifies with space, BTW, just that less software supports f=f.

    
    

    
  
>> Once more: This is also happening to [...] the word From
>
>Just as in plaintext, 

Which standard requires it? It might happen in broken implementations,
though. For f=f this is 4.1. of RfC 2646. But I cannot find anything
like this in mail or news RfC, correct readers do allow putting
From at the beginning of a line.

>> - All lines starting with a ">" will have a space added.
>
>If they are not a quote, then that *must* be the case, because > is the quote
>character - by convention in real plaintext and by spec in f=f. Anything else
>would be ambiguous or wrong, depending on how you see it. With f=f, a reader
>supporting f=f will be able to differentiate between a quote and a line
>starting with >, and the latter will appear exactly as the user typed it.

That is the point. It is not what the user typed for users which don't have or
want f=f, and we are discussing this question from the FAQ right now.

>I hope you can see the value of being able to exactly represent what is a quote
>and which line just happens to start with a quote. 

This is not the question.

>> Any difference at all means the message is not exactly the same.
>
>Right. And that's why this whole discussion is so pointless, because text/plain
>doesn't *exactly*, byte-for-byte, 

It does modulo broken implementations.

>transfer the message either, for exactly the same reasons as f=f.

You just named the reasons for f=f, and they are not the same. The discussion is
pointless if you ignore the facts. text/plain has no problem with > or From or
anything else at the beginning of a line.

>> Knowledgeable people use the plaintext editor
>
>haha. *shaking head* I guess the developers are all not "knowledgeable", then.

Do we need to teach a logic course here? If knowledgeable people people use
something it does not imply that *all* of them do.

>I have never said anything about "," (to my knowledge). As for ".", I was
>wrong.

Good.

>I was thinking about SMTP (RFC 821: "SMTP indicates the end of the mail data by
>sending a line containing only a period."), but it seems that SMTP has a
>provision to keep that ".", namely by adding a "." to every line starting with
>a ".", and later removign the first "." on a line. 

Whatever happens during transfer is a different issue, like any
content-transfer-encoding.

>Exactly the same that f=f specifies with space

No, it is not a transfer-encoding. That is bad design of format-flawed. It
claims to be readable as text/plain, but it is different. You may argue this is
not important, other people have different opinions. But facts are things are
different if you don't have f=f at the receiver's end.

pi

    
    

    
  
I think this whole discussion is getting stuck on the wrong issue.
Yes, f=f is better than "old" plain text if only every viewer would support it.
But, f=f is not backward compatible with "old" plain text, f=f messages just 
are not 100% the same as "old" plain text messages.

However, this bug is about F=F evangelization. Right now, mozilla seems to be 
making the same mistake that many have made before them: mozilla forces f=f on 
its users instead of providing them with a choice and letting the users choose 
what is best for him/her. When you force people, people will resist. When you 
give them a choice, they will eventually choose the solution that is the 
easiest to them. Most times, they will choose the solution that has the 
largest "users base". For mozilla, if they choose f=f you have succeeded, if 
they choose plain text, somehow the majority of the people felt that plain text 
was good enough for them, so why change that?

So, maybe instead of "forcing" everybody to use f=f, maybe mozilla could give 
its users a choice?


    
    

    
  
Ben, thanks for confirming that there is no problem with lines starting
with "." and ",".   You mentioned ", " in bug 141983#c31.

Pi, there is no problem with emails having lines starting with "From "
unless they are stored in an Mbox mailbox.  This is a linear file
multi-message storage format where the message delimiter is a line
starting with "From ".  So whenever such a line is stored in an Mbox it
is prepended with a ">".  There is no way of reversing this.  I
understand that Mozilla uses Mbox format for its local mailboxes on the
PC.  But these are not at all involved in sending a message unless you
save it to Drafts and re-edit it, and if the Drafts is a local mailbox.
 All my mailboxes are on an IMAP server.  Some old low-performance IMAP
servers use Mbox.  The average mailspool file on a Unix machine is an
Mbox too.  High-performance mail IMAP servers use Maildir mailboxes for
the user mailboxes and for the Inbox.  Maildir stores one message per
file, so there is no problem.

The ">From " problem was an occasional problem for me before I used 
Maildir mailboxes - in terms of sending and reading received messages.
I guess that some messages would have arrived with this problem due to
problems in Mbox mailboxes outside my server, but in 2 years or so
since I have been using Maildirs here, I definitely have not seen 
a single ">From ".  So I think it is relatively rare and getting 
rarer.  In my experience it was never a serious problem.  It is not 
an email problem as such, but Mbox mailboxes are still widely used 
so it is a practical real-world problem - and sending and storing 
with "format=flowed" overcomes it.

Apart from the occasional ">From " problem, the only other fundamental
limitations on plain-text email are imposed by SMTP: a maximum line 
length of just over 1000 (if I remember correctly) and the 7 bit 
character coding restriction.


Robbert, I entirely agree with you.  "Format=flowed" should not be
forced on anyone, but this is what Mozilla currently does to all its
users, and to the recipients of their messages, unless the user knows
how to turn it off with a user.js pref.


The purpose of Mozilla and Bugzilla is, ultimately, to serve the
interests of Mozilla users - usually by optimising the software.

It arises, as others do, from users being unhappy (or us thinking 
they are or will be unhappy) with Mozilla.  But this particular 
bug, as originally conceived, locates the problem not in Mozilla, 
but in the minds of the unhappy users.  The original solution, 
emodied in the name of the bug, is also different from the usual 
bug: rather than change the software, a programme of 
evangelization is proposed to change the contents of the minds of 
users.  Furthermore, their unhappiness with Mozilla is not, 
or until this wee, was not generally attributed to any deficiency 
in Mozilla - but to their own failure to understand the benefits of 
"format=flowed".

I suggest another approach:  Make sending - and perhaps receiving -
with "format=flowed" Off by default.  Have well explained user 
interface options for turning it on, perhaps with per-recipient 
options for sending with it.

No amount of evangelization will make users happy with their 
messages being corrupted when read in a non "format=flowed" 
receiving environment.   When this bug began, it seems that 
no-one involved in it realised that this was happening, but 
as noted in Bug 141983 the problem is a big one - dataloss.


Ben keeps avoiding my suggestion that sending with "format=flowed"
should be off by default.  Maybe it it is hard or impossible for 
him to concieve of this or the benefits of not forcing it on 
most users, because, as far as I can tell, his estimation of the 
benefits of wider and ubiquitous use of "format=flowed" usage are 
so great as to render all objections relatively insignificant.

*Maybe* this would have been a valid judgement when this was debated in
the past, based on the notion that messages sent with "format=flowed"
were not visibly changed.  But as of the start of this week, when we all
realised that sending with "format=flowed" seriously (though Ben seems
not to think so) alters plain-text messages when not interpreted by a
"format=flowed" algorithm at the receiving end, having sending with
"format=flowed" on by default seems completely at odds with the 
interests of most plain-text Mozilla users.

 - Robin



    
  
Depends on: 141983

    
    

    
  
I have done a little reading on the 'net, and it seems format=flowed is a
bigger issue than some people here make it seem. A search with google for
format=flowed generated more than 65000 hits (pages&posts). For a feature
which should be backward compatible and not influence normal operation,
this seems to be quite a lot. Particularly, it seems a lot of users require
help to understand the new linewrapping ("I have set wrap to xx, but it does
not wrap!") and quote bar behaviour.

Also, it seems mozilla is not the only one with unhappy users. A very
significant 13000 hits (1/5 of the traffic) discuss disabling format=flowed.
Interestingly, a large number of these hits are for eudora (the inventors of
f=f!) who (like mozilla) failed to provide their users with an UI-option to
turn f=f off.

There really seems to be a need for better user education as well as an option
to disable f=f...

    
    

    
  
Googling for format=flowed turns up lots of mailing list 
archives which reproduce the headers.  Eliminating pages 
with "charset=us-ascii" is a reasonable way to get rid of 
those, though it would also get rid of some  pages which 
are genuine discussions of "format=flowed"

Google finds 1,200 pages for:

 disable "format=flowed" -"charset=us-ascii"

of these the pages which mention browsers are:

  Mozilla      178
  Netscape     137
  Eudora        92
  
But whatever the level of awareness, as long as sending 
with "format=flowed" is on by default, many messages of 
many Mozilla users will be corrupted in many situations.

  - Robin


    
    

    
  
Alright then, this is the most accurate I can get:
format=flowed -"charset=" -"content-transfer-encoding"
17200 pages + 6100 postings = 23300 hits

Of these hits, 2330+1260=3590(=1/6) discuss disable|disabled|toggle|off.

I'ld still consider these numbers significant.


    
    

    
  
> the only other fundamental limitations on plain-text email 

You keep diminishing the problems of plaintext and ignoring the values of f=f.

[values of f=f deleted, to not ignite further discussion]

> The purpose of Mozilla and Bugzilla is, ultimately, to serve the
> interests of Mozilla users - usually by optimising the software.

FYI: The purpose of BugZilla is *reporting* bugs and *technical* discussion
about them (and only them), not neverending debates. Political discussions
belong to the newsgroups.

> it seems a lot of users require help to understand the new linewrapping ("I
> have set wrap to xx, but it does not wrap!") and quote bar behaviour.

Exactly, *understanding*. Many people think that vertical bars mean that the msg
was sent in HTML. Clearly wrong. That's what this bug is for, to educate them.

    
    

    
  
Here's a message I posted to the newsgroups:
http://groups.google.com/groups?selm=woYoa.1%24ER6.126%40news.oracle.com

See also bug 202195.  Many/most of us use 2 spaces to separate sentences.  The
f=f option doesn't play well with this.  If the sentence is getting terminated
on eol, the second space sometimes gets carried forward to the next line.  I'm
not proposing a solution; this should be open for discussion.  I do wonder if
rfc 2646 is fundamentally flawed.

For now, I've turned off f=f until this bug is fixed.

    
    

    
  
To get forward, please

1) add bug 141983 to the FAQ,

2) correct the question "How does Format=Flowed affect Non-Format=Flowed Readers?",

3) consider explaining the consequences of the bugs in the FAQ,

4) ben.bucksch@beonex.com wrote
>You keep diminishing the problems of plaintext 

Those remarks are not helpful. All things you mentioned are not problems of
plaintext, all are just bad or false implementations.

>and ignoring the values of f=f.

This is not the question. This discussion comes from the question, why
format=flowed is incompatible with plaintext. This is a fact which needs to be
explained in the FAQ. It is not important if anybody considers those differences
important. For this question it is also not important, if f=f solves some other
problems or not.

>That's what this bug is for, to educate them.

Bugzilla is not used by average users.

pi

    
    

    
  
I wrote that Ben had indicated in bug 141983 comment 31 that lines
starting with a comma caused some problems.  But this was my mistake -
incorrectly parsing in my mind what he wrote:

> (".", ">", "From ")

Sorry about this.  I apologised to Ben as part of our correspondence.

Ben has suggested a workaround for the extra space in lines starting
with a space or a ">".  This is in bug 141983 comment 29.  This is
a modification to the "format=flowed" encoder to which is intended to
reduce, from the recipient's point of view, the changes to the message
when the encoded message is read without "format=flowed" decoding.

His intention, as I understand it, would be to add a space at the
front of all lines of any block of text which were a contiguous set
of non-empty lines which were all shorter than 80 characters.  I think
the intention is to preserve the indenting relative to the block but
indent the entire block.  As I understand it, this does not seem to
lessen the corruption of the message at all, though this is a matter
on which different recipients would have different views depending on
the nature of the message.  I haven't worked through it properly but I
think it would also alter the message as seen via a "format=flowed" 
decoder.

My previous comment about Mbox addition ">" to lines starting with
"From " should have been addressed to pi (no upper case).  I was
responding to the end of his comment 28: "What is changed with 
text/plain?"

  - Robin



    
    

    
  
> All things you mentioned are not problems of plaintext, all are just bad or
> false implementations.

Like Mozilla's message store.
They are common part of life with plaintext. Not with f=f. Solving these
problems is the direct cause for some of the problems of f=f. As such they are
very much relevant.

    
    

    
  
>> All things you mentioned are not problems of plaintext, all are just bad or
>> false implementations.
>
>They are common part of life with plaintext. Not with f=f. Solving these
>problems is the direct cause for some of the problems of f=f. As such they are
>very much relevant.

Then maybe the choices of f=f should be questioned.
But you can't blame (problems in) an old standard for the compatibility issues
of a new standard!


    
    

    
  
ben.bucksch@beonex.com wrote:
>> All things you mentioned are not problems of plaintext, all are just bad or
>> false implementations.
>
>Like Mozilla's message store.

Which could be fixed.

>They are common part of life with plaintext. Not with f=f. 

You are mixing things. It is not a problem with plaintext. It is a problem of
content-transfer-encoding 8bit, quoted-printable solved that long before f=f
without incompatibilites.

>Solving these
>problems is the direct cause for some of the problems of f=f. As such they are
>very much relevant.

They? You mean "From " at the beginning of a line, right?

pi

    
    

    
  
> Which could be fixed.

But it won't. Neither will Unix' mailboxes.

> quoted-printable solved that

oh! QP is OK? QP looks *far* messier in non-supporting readers than f=f.

Let=3D54s use=
QP=3D54s style of=
flowing paragraphs,=
then.

    
    

    
  
This message is for everyone but Ben - I am honouring his
request (after many hours of private emails) that I not try
to change his mind.

Ben correctly stated that quoted printable is a mess when
received without a suitable decoder.

HTML, quoted printable and format=flowed all solve
particular problems in sending emails, and they generate
other problems - in particular they all result in message
corruption in the absence of a suitable decoder.

Mozilla only sends HTML after warning the user about
problems with people not being able to read the message.

No-one is suggesting that Mozilla send with quoted
printable by default.

Somehow - perhaps in ignorance of how it works by adding
spaces at the start of many lines - Mozilla is currently
sending with "format=flowed" on by default, without any
warning or user option to turn it off.

This is about as wrong in principle as having quoted
printable on by default.

Who (apart from Ben - who will never change his mind)
thinks that the benefits which arise uniquely from
default on sending with "format=flowed" justify millions
of Mozilla's users having their messages corrupted?

Ben tells me the decision has been made, and will never
be changed, so I might as well give up.  He says the
same thing about fixing the occasional ">From ..."
problem caused by Mbox mailboxes - that its useless
trying to convince the powers-that-be to fix this dumb 
bug.  Yet he cites this as one of his reasons for 
imposing "format=flowed" on everyone by default.

Ben does not seem to regard extra spaces in other people's
messages as a significant issue.  I found it to be a 
complete waste of time trying to get him to respect other
people's views about the need for their messages to arrive
exactly as they saw them when they sent them.


Who decides whether or not Mozilla sends with format=flowed
by default?  Do they read this bug, or bug 141983?


  - Robin


    
    

    
  
> This message is for everyone but Ben

heh, yet you're talking about me and try to state *my* position, partly what I
said in *private* email, and your statements are wrong, and I don't correct them
all here.

> No-one is suggesting that Mozilla send with quoted printable by default

(with the disadvantage of being incompatible with some (strictly
standard-conform) 7bit mailservers.)

> ">From ..." ... that its useless trying to convince the powers-that-be
> this dumb bug

heh, it's indeed useless to try to convince people to get rid of standard Unix
mail delivery.

> Yet he cites this as one of his reasons for 
> imposing "format=flowed" on everyone by default.

No. I cite that as reason for exactly the space "alteration" you complain about
in f=f, and I cite it as one reason how pointless your "unaltered message
transfer" argument is.

> Ben does not seem to regard extra spaces in other people's
> messages as a significant issue.

Right, when weightened against the alternatives of e.g. comb wrapping.

And the latter is the important part, and that's what you always omit despite
better knowledge, and that's part of what enrages me about your statements.

> trying to get him to respect other
> people's views about the need for their messages to arrive
> exactly as they saw them when they sent them.

huh? I respect your view, see comment 30. It doesn't do what you want? Fine, you
have the choice to turn it off. Use it.

(arguments deleted to not ignite further discussion)

> Do they read this bug, or bug 141983

They should be very well aware of this discussion. :-( If you are asking where
this discussion belongs, then I already answered that many times: the
netscape.public.mozilla.mail-news newsgroup. It happens to have all the
discussion already, including (since recently) the note about bug 141983. If you
want to continue discussion (although I see absolutely no need to, because the
arguments of both sides are known), please continue on the newsgroup, after
reading the old discussion(s) there between us.

    
    

    
  
>oh! QP is OK? QP looks *far* messier in non-supporting readers than f=f.

The difference is that QP is correctly declared as a content-transfer-encoding
(which would be correct for format=flowed, but it was falsely not declared a
proper MIME encoding). MIME is 10+ years old, so there is absolutely no reason
not to support it, the main reason being that most languages (not including
English) cannot be written without MIME, so it is needed anyway. And you also
gave one reason yourself:

>(with the disadvantage of being incompatible with some (strictly
>standard-conform) 7bit mailservers.)

pi

    
    

    
  
Ad comment 47: Bug 202195 is not related to f=f, I agree to the conclusion, though.

pi

    
    

    
  
Ben wrote:
> huh? I respect your view, see comment 30. It doesn't do what you want? Fine, you
> have the choice to turn it off. Use it.

Manually adding settings to your preferences file? What kind of a choice is that
for "normal"? How are they even going to know they can turn it off?
If you want people to have a choice, you need an option on the "Send Format" tab
(that's where the QP option belongs too, IMHO).

    
    

    
  

      
      
      
      

        Attached file
          
        Format=Flowed Mini-FAQ (obsolete)
        — 
      

    


  

        Attachment #122020 -
        Attachment is obsolete: true

    
    

    
  
I have just seen one more example of all quoted lines being space-stuffed. It
was from Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.3) Gecko/20030312
which is long after bug 161968 was fixed. I am not able to reproduce, but we
certainly need to figure out what causes this serious bug. It makes Mozilla
users look like lusers.

BTW: On the usefor mailing list some people use f=f, the regularly have "quoted
lines" indented. If not even those experts can use it the normal user will never
be able to.

pi

    
    

    
  
You may add bug 161091 and bug 153807 to the FAQ.

    
    

    
  
I think the first is a dupe. The second for sure is, I only have to find it ...

pi

    
    

    
  
Updates to FAQ's part 8:
 - bug 161968 is mentioned as "fixed" but the bug is still new and has never 
been marked fixed; I'm not sure what, if anything, was fixed regarding this bug. 
The symptom described in the initial report is still present.
 - bug 104348 was WFM'd by its reporter, and in fact the situation described by 
bug 104348 comment 21 no longer occurs (1.4 Final).

    
    

    
  
Content-Type: text/plain; charset=US-ASCII; format=flowed
X-Mailer: Apple Mail (2.552)

Yay :-)

    
    

    
  
More updates for this bug:
Adding dependencies on:
bug 125928: PlainText message loses carriage return after 2 spaces (composed
            with format=flowed) (bug 155015 was duped to this one)
bug 173918: wrapping of quoted flowed (f=f) lines disobeys limit for chars per
            line
bug 220575: Edit|Rewrap forces hard line breaks, interferes with format=flowed
            wrapping
bug 223279: Inconsistent handling of multiple spaces at wrap point (flowed or    
            not)

I'm not sure that bug 161968 is related to f=f, altho it sure would be nice to 
get it fixed.  I suggest removing it as a dependency here.


Holger, are you updating the FAQ anymore?  The reference to 155015 in item 8 
should be updated to 125928; maybe those other three bugs mentioned above should 
be included as well.  (And, the items mentioned in comment 63 could be 
integrated.)

Also, in the FAQ, item 2: "What happened to the old-style ">" that used to 
appear when I replied to a message?" -- this makes it sound like when you hit 
Reply, the compose window no longer shows the '>' prefixes.  In fact, the prefix 
shows in the compose window; it's the displayed message with quoted text that 
shows the excerpt bars.
Depends on: 125928, 173918, 220575, 223279

    
    

    
  
I'm rather busy at the moment and can't promise to work on the FAQ any time soon
(where 'soon' means at least unti next weekend), so if anyone wants to pick it
up, please feel free to do so. Mike? ;-)

- Holger

    
  
Depends on: 144230

    
    

    
  

      
      
      
      

        Attached file
          
        Format=flowed mini-FAQ
        — 
      

    


  
I've rewritten this FAQ somewhat, added some additional items that I hope
explain f=f and the ramifications a little more, updated the section about
bugs, and also improved the HTML/CSS of the document.  Holger's seen this and
said he liked it.

        Attachment #123686 -
        Attachment is obsolete: true

    
  
Depends on: 98397

    
    

    
  
Remarks/corrections:

5, first paragraph, last sentence: "a viewer that encounters such a line" must
be "a format=flowed-capable viewer ..."

next paragraph: "(For mail composed without f=f, a line beginning with "From "
is prefixed with ">"; this is visible, and typically treated as a quote, in the
recipient's mail viewer.)"
This is wrong. Some readers do it that way, some remove the > for display, some
encode the F in quoted printable, some do nothing like that. Best is to drop the
entire sentence. It is not useful here.

6 "removal of spaces cause a problem, these are by far the minority." I would
not follow that judgement. It does depend on context. Don't play politics here,
but list facts, please.

"Messages where these changes are critical often have a specific technical
application, and may not be processed by a standard mail client" -- sure they
can use standard mail clients.

"Lesser problems are caused for recipients who do not have f=f-capable mail
clients, and are exposed to the extra indentation imposed by space-stuffing."
What is "lesser", it is just ugly and looks like a mistake of the sender. We
should be honest and let people know it.

"The advantages of format=flowed apply to a much wider range of messages."
Again, this is your jugement.

7: "In addition, quoted text is properly maintained at the correct "quote level"
(this due, again, to avoiding the combing effect)." Not correct. Keeping correct
quoting levels is totally independent of format=flowed.

8: "For quoted text, this can be especially problematic.  Suppose the above text
is originally read on a sufficiently wide window, quoted and passed on with a
comment; then it is read on a narrow window, and quoted again. A likely result
is:" Sorry, this is bullshit. If this happens, it is a bug which basically
Outlook Express has. For ages other readers have done it correctly without
format=flowed.

13: For easy reading annd finding you should not drop the bug numbers. For
example, in 11 you refer to such a bug, it is pretty hard to find it in 13.

In the end "The second contains much overlong pontification by a vocal opponent
of f=f.". Sorry, the FAQ is again not the place to put your judgement here.

Also you dropped some bugs from the list. For example bug 144998 lets Mozilla
users really look like lusers. Those messages are modified so that they look
totally different with additional empty lines.

I feel that your changes are -- in your words -- a "much overlong pontification
by a vocal advocate of f=f". Be honest, list the facts, not your opinion or
judgement.

pi





    
    

    
  
> 6 "removal of spaces cause a problem, these are by far the minority." I would
> not follow that judgement. It does depend on context. Don't play politics
> here, but list facts, please.

The "context" is the sum of Mozilla users, and I think it could be statistically
proven that the majority of them don't care *at all* about these spaces.

Apart from that, this is an "Evangelisation" document. Judgement in favor of f=f
is the idea of it and totally appropriate. Judgement also was the base for
inclusion and enabling of f=f in Mozilla.

> 7: "In addition, quoted text is properly maintained at the correct "quote
> level" (this due, again, to avoiding the combing effect)." Not correct.
> Keeping correct quoting levels is totally independent of format=flowed.

Not correct.
1. Quote levels are not kept *at all* in plaintext, from a machine's POV
2. One of the *goals* of f=f is to reflow quotes when possible and thus make
quotes work (most of the time) *despite* later involvement of broken mailers
like OE.

    
    

    
  
>> 6 "removal of spaces cause a problem, these are by far
>> the minority." I would not follow that judgement. It
>> does depend on context. Don't play politics here, but
>> list facts, please.
> 
> The "context" is the sum of Mozilla users, and I think it
> could be statistically proven that the majority of them
> don't care *at all* about these spaces.

Really? You think so? What makes you believe it? I see a lot of people which
wonder why their quotes are broken (the split quoted lines and manually add a
quote symbol).

> Apart from that, this is an "Evangelisation" document.
> Judgement in favor of f=f is the idea of it and totally
> appropriate.

Even if I follow that, presenting false facts in favor of
the idea is not the right thing to do.

And anyhow the FAQ is not the place to insult people with different judgement.

>> 7: "In addition, quoted text is properly maintained at
>> the correct "quote level" (this due, again, to avoiding
>> the combing effect)." Not correct. Keeping correct
>> quoting levels is totally independent of format=flowed.
> 
> Not correct. 1. Quote levels are not kept *at all* in
> plaintext,

What? Of course, Quoting is done by putting > (or possibly "> " or other
appropriate symbols) in front of the quoted line. Period.

> from a machine's POV

What is POV?

> 2. One of the *goals* of
> f=f is to reflow quotes when possible and thus make 
> quotes work (most of the time) *despite* later
> involvement of broken mailers like OE.

That is not what the statement says. It false says that quoting depends on how
the text to be quoted is presented in a possibly narrow windows. This is not
correct. Even broken OE does not do it tat way.

pi

    
    

    
  
Ben wrote:

> 1. Quote levels are not kept *at all* in plaintext, from a machine's point
>    of view.
   

Plaintext email clients which send exactly what the sender sees on screen and
which display and print it exactly as sent achieve one important, vital, goal:
within clearly understood limitations (character set and right margins), they
allow one person to reliably transfer a message to another person, without the
software in any way **** around with it.  

There are good arguments for systems like Format=Flowed, representation of
leading ">" characters as vertical bars etc.  They should be well documented
options, off by default.  If people want to use a special feature - such as
sending with F=F to help recipients with narrow screens at the cost of screwing
up ordinary messages - then its best they turn it on manually, per message, per
recipient or globally. 
  

If its zealous to ask - and indeed insist on behalf of ordinary users(*) who
shouldn't have to cope with overcomplex software changing their message in any
way at all between what is visibly sent and visibly received - that Mozilla
default to straightforward, simple, reliable behaviour, then I am a zealot.

But in my opinion, people who inist on foisting these complications on tens or
hundreds of millions of ordinary users are the zealots.

  - Robin

*  Ben has written that the exact locations of spaces, line endings etc. 
   is not part of the message - or that the meaning and function to the
   sender can be reliably assumed and used in some way to reformat the
   message.  But the exact representation of the message is important
   to many ordinary people, from a human and technical point of view.  
   They should be able to put spaces, newlines and ">" characters 
   anywhere they like in a message, for whatever purposes they like, 
   and have it reproduced verbatim at the other end.  None of us, 
   including Ben, can reliably enough discern the intention behind the 
   use of any such character, space, or newline to justifiably change 
   it by default in the sent message - especially invisibly to the sender.
   

    
    

    
  
> What makes you believe it?

E.g. that most people don't know what HTML or plaintext is nor really care. Note
that the Usenet population is not the average Mozilla user.

> quotes are broken (the split quoted lines and manually add a quote symbol).

This is a bug/problem in the plaintext composer. The HTML composer doesn't have
this problem at all.

BTW: The FAQ also wrongly assumes the plaintext composer everywhere. Most users
use only email, and there, the HTML composer and sending as (f=f) plaintext is
the default, so a good chunk of what you write there is wrong or irrelevant for
them.

> presenting false facts

I never intentionally suggested to do that.

> not the place to insult people

Which insult? "minority"?

> Quoting is done by putting > (or possibly "> " or other
> appropriate symbols) in front of the quoted line.

In general, yes. But it's ambiguous, see above, which is a serious problem for
machine processing, not to the least because of people like you who complain
when the machine processing goes wrong in edge cases. Machine processing allows
us e.g. to avoid the most confusing parts of the comb effect, which was the
original argument.

And many people disagree with the >. See e.g. the GNU Emacs mailer.

> That is not what the statement says.

I didn't claim that - the statement in the FAQ was indeed confused and wrong.
But yours was wrong, too.


Robin, please don't keep implying that real plaintext mails appear exactly as
sent on the recipient's end, because that's simply *not true* for many or even
the most mailers used, as discussed at length above. If you want that, use PDF
or PNG.

    
    

    
  
Thanks, Robin, for explaining exactly what I think. I am French, and not very
fluent in English, thus I could not say all that the same way than you did, but
I completely agree with all your comment (#71).

I would like to add something. The fact that the spaces are considered
non-important (so that they can be freely exchanged with newlines) has a very
bad impact on at least two sorts of messages :
- those where there is an ASCII schema, or a math formula;
- those written in a language, such as French, which often use non-breaking spaces.

As I said, I am French, and I hate to have this :

   Ceci est un « exemple », eh oui !

... transformed into that :

   Ceci est un «
   exemple », eh oui !

... or even that :

   Ceci est un « exemple », eh oui
   !

The case of math formula is even worse.


For these reasons, I strongly agree that the f=f should be an option, and *not*
the default option.


Olivier Miakinen

    
    

    
  
>> What makes you believe it?
> 
> E.g. that most people don't know what HTML or plaintext
> is nor really care. Note that the Usenet population is
> not the average Mozilla user.

I don't personally know the average Mozilla user. I know
many of them, though, admittedly, most of them from Usenet.
The key to that question (what people see as problems, IOW:
what they do expect) is if WYSIWYG is important to people.
Lots of the bugs listed in the dependencies deal with the
surprise of people that things look differently after they
sent it than from what they did compose.

We will not be able to sort this out here, so the fair
approach is to stay with the facts and let the people judge
themselves. Actually, Holger did a great job in that. His
FAQ was well written and unbiased.

>> quotes are broken (the split quoted lines and manually
>> add a quote symbol).
> 
> This is a bug/problem in the plaintext composer.

According to the new FAQ, this is complaining about (correct) space-stuffing.
Anyhow, no matter if it is breaking quoted lines, pasting quoted text or just
typing a quoted line from scratch, it is about expectation. A machine will
probably in the near future not be able to understand the intention of the author.

> BTW: The FAQ also wrongly assumes the plaintext composer
> everywhere. Most users use only email, and there, the
> HTML composer and sending as (f=f) plaintext is the
> default, so a good chunk of what you write there is wrong
> or irrelevant for them.

Which part of what I wrote is irrelevant? Some bugs do not exist there, but what
else? Most of my remarks are not related to particular bugs.

>> presenting false facts
> 
> I never intentionally suggested to do that.

I did not assume so, so let's get those out.

>> not the place to insult people
> 
> Which insult? "minority"?

No: "The second contains much overlong pontification by a vocal opponent of f=f."

I missed the final comment, that those won't probably be fixed because it would
break f=f. Certainly not true for pasted quotes.

>> Quoting is done by putting > (or possibly "> " or other
>> appropriate symbols) in front of the quoted line.
> 
> In general, yes. But it's ambiguous, see above,

Not really. It has been done for many years. But you are right, a machine cannot
understand the intention of the sender -- either way. But for quoting text in a
reply, it is clear, you know what is quoted and can treat it that way.

> And many people disagree with the >. See e.g. the GNU
> Emacs mailer.

??? I see lots of perfect messages from that client.

> Robin, please don't keep implying that real plaintext
> mails appear exactly as sent on the recipient's end,
> because that's simply *not true* 

It has always. I cannot find any evidence contradicting it.

pi

    
    

    
  
> Actually, Holger did a great job in that. His
> FAQ was well written and unbiased.

FWIW, I also liked Holger's original FAQ better.

> Anyhow, no matter if it is breaking quoted lines, pasting quoted text
> or just typing a quoted line from scratch, it is about expectation.

All of that will work as expected in the HTML composer.

> Which part of what I wrote is irrelevant? [because of HTML composer]

I meant the FAQ, not what you wrote.

[plaintext doesn't always appear exactly as sent]

> It has always. I cannot find any evidence contradicting it.

Just send your message to an Outlook user and see how he sees it.

    
    

    
  
Ben, you wrote:

> Robin, please don't keep implying that real plaintext mails appear exactly 
> as sent on the recipient's end, because that's simply *not true* for many or 
> even the most mailers used, as discussed at length above. If you want that, 
> use PDF or PNG.

Plaintext clients have forever reproduced the message precisely as seen 
by the sender to the recipient, within the limits I mentioned - the right 
margin and the compatibility of fonts at each end.  (A well written client will
compose, read and print with a fixed width font, and will not truncate words
which are longer than the margins, either when sending or displaying.)  The
right margin thing is clearly understood by the sender - and anyone who chooses
to, or who is forced to, read the message on a screen less than 80 characters or
so wide is well aware that they are not able to see the message as (most) people
prefer to write them.   There are abuses of plaintext email, such as clients
which send all paragraphs as one long line - this ignores the line breaks the
sender sees on screen, and which they rightfully expect to be reproduced to the
recipient.  

Plaintext clients typically involve difficulties with quoting text when
replying, because the quoted text lines don't fit the editor's margins.  That is
a pain, and F=F can help.  But that is a second order problem, which the sender
can solve manually or hopefully with the help of software.  It is a first order
problem for the client at either end to alter a message in any way by default,
especially in a way which the sender can't anticipate, because we cannot
reasonably assume that such alterations are not a problem for the sender and
recipient.

Ben, you have consistently claimed that the traditional plaintext approach to
mail and Usenet is broken and does not in fact convey a message without changes.
   I and many others do not share your view.  Whether or not your view is
thought to be valid, the principle still remains that Mozilla should have
complicated features which tend to screw up messages off by default.

  - Robin



    
    

    
  
Regarding some of the concrete issues raised:

I see how my description of how combing can mess up the quote levels is unclear. 
I suggest changing:

> Suppose the above text is originally read on a sufficiently wide window,
> quoted and passed on with a comment; then it is read on a narrow window,
> and quoted again.

to

> Suppose the above text is originally quoted in one mail client without
> additional wrapping and passed on with a comment; then it is quoted again by
> a different client which wraps to a narrower text width. 

Does that work?


> I see a lot of people which wonder why their quotes are broken
> ([they] split quoted lines and manually add a quote symbol).

OK, I see where this comes from:  If they type the symbol at the beginning of a 
line in the unquoted-text portion, then type Delete to bring the quote line up 
to follow, then that line will be space-stuffed and not appear as a quote.  If, 
however (like I have always done in this situation), they manually add a quote 
symbol to a split quote by typing it *at the beginning of the broken quote*, it 
gets sent as a quote.

I'll write this up into the FAQ, too, unless I simply shouldn't bother because 
everyone thinks my edits are hopeless.


> bug 144998 lets Mozilla users really look like lusers.

I tested this before dropping that bug from the FAQ; the line doubling occurs 
with f=f turned off, and so I don't think it's a f=f bug.  There may be a 
situation where f=f makes the symptom look worse, I'll test some more.


Regarding the rest:

Anyone who cares to replace my edit of the FAQ is welcome to write their own.  
I did think that it would be useful to give an example of "combing effects" 
which are the primary thing that f=f is intended to fix, and to define 
"space-stuffing" which is a term that gets much bandied about; so perhaps you 
could at least keep those parts.  

It is true that I like f=f, and did not feel any need to keep this document from 
being biased towards it. However, dismissing as "judgement" my characterization 
of the ratio of messages for which f=f is a problem to those for which it is not 
-- well, that's simply obstination.  You don't need to have a statistically 
sound sample of 1,000,000 messages from 10,000 email users to realize that, *by 
far*, most people writing email are focused on the *words*. They don't give a 
hoot about the wrapping except where it appears broken and less legible because 
it was *not* reflowed into a narrow window.

I daresay that reflowing is the expected behavior for most email users because 
practically all of them are used to web pages that reflow.

    
    

    
  
Mike Cowperthwaite, what I said wasn't meant as personal critique, I just think
that the FAQ is way bloated with particularies by now, burrying the core points.
And it was hard to understand the "problems" described.
If I think that you/we shouldn't bother, then mainly because the FAQ is not used
anywhere where it would matter, to my knowledge.

    
  
Depends on: 223983

    
    

    
  
I think people here should take a break, get some fresh air. :-)

In my personal opinion the function of a FAQ is not to express a certain opinion
or push people in a desired direction. A FAQ should clarify and clear up, but
nothing more. Therefore I suggest that the FAQ should be toned down a bit.
Perhaps we need another FAQ dealing with f=f problems, and the FAQ dealing with
evangelization sticks to simple and calm explanation.

- Holger
Product: MailNews → Core

    
    

    
  
*** Bug 273276 has been marked as a duplicate of this bug. ***

    
  
Depends on: 341282

    
  
Assignee: ducarroz → nobody
QA Contact: esther → composition
Product: Core → MailNews Core

    
    

    
  
Please, please, please turn off format=flowed as the default format for plain text email!!

Thousands of newbies are dismayed and embarrassed to discover that their job and college applications, and other important correspondence, "randomly" lack line breaks!! But they can only see that these line breaks have been removed AFTER the message has been sent.  

This behavior completely violates WYSIWYG, and it is utterly counter-intuitive!  No doubt there is some perceived benefit to this "feature" but it needs to be much, much, much better understood by the user community before it is implemented as the default behavior.

Why am I entering this request as "bug" (not a feature request)?  Because anytime software behaves in an unexpected and inexplicable way, it is IMHO a bug.

Flowed format DOES NOT need evangelism!  The current implementation (any anything remotely like it) needs to become an opt-in feature with plenty of instructions, guidance and warnings!

    
    

    
  
> But they can only see that these line breaks have been removed
> AFTER the message has been sent.

Not true, there are no line breaks during composition. The normal mail composer wraps automatically at variable widths.

> before it is implemented as the default behavior.

It's the default since almost 10 years.

    
    

    
  
Hi Ben,

I was very pleased to receive your note.  I'm glad someone is still willing to look at this issue.

Since I don't see how to attach files to comments in this forum, I will send (to your personal email address) an image of an email from my Sent Messages folder. You will see that the 3 sentences forming the body of the message no longer have a blank line (or line break) between them.  While I composed this message, those line breaks were in place and visible, indicating separation of the body into 3 paragraphs.

I was told that the line breaks were removed because, during the composition and editing process, I had inadvertently placed one or more invisible space marks after the periods ending my sentences, and for this reason (this is a reason?), Thunderbird had decided that line breaks between my paragraphs were superfluous and "extra".  But they are not at all superfluous, they are essential to the structure of the message.

I was advised to turn off flowed format, but first I wondered if the diagnosis was correct.  Unfortunately, it is very difficult to determine whether there are hidden spaces after periods simply by examining messsages in the Sent folder. Even if you highlight the sentences in question, it will always suggest that there is exactly one space after the period (this is true for emails that are missing and not missing line breaks).

Soon I discovered that the only way to reliably determine whether there were indeed invisible spaces after the periods in my message was to prepare to forward the message.  Once the message appeared in the forwarding window, I highlighted the sentences and -- yes!! -- wherever a line break had disappeared, there were at least two hidden spaces after the period!  But even more bizarrely, in the forwarding window, the missing line breaks re-appear again!!  It's unbelievable!!

I do hope you have a moment to take a look at the attached image, and the original note, which I will forward to your personal email in a moment,

Many thanks,

Jordan

    
    

    
  
Please, please, please turn off format=flowed as the default format for plain text email!!

Thousands of newbies are dismayed and embarrassed to discover that their job and college applications, and other important correspondence, "randomly" lack line breaks!! But they can only see that these line breaks have been removed AFTER the message has been sent.  

This behavior completely violates WYSIWYG, and it is utterly counter-intuitive!  No doubt there is some perceived benefit to this "feature" but it needs to be much, much, much better understood by the user community before it is implemented as the default behavior.

Why am I entering this request as "bug" (not a feature request)?  Because anytime software behaves in an unexpected and inexplicable way, it is IMHO a bug.

Flowed format DOES NOT need evangelism!  The current implementation (any anything remotely like it) needs to become an opt-in feature with plenty of instructions, guidance and warnings!

    
    

    
  
I'm sorry everyone for inadvertently duplicating my first post -- very sorry.

Anyway, I now have images of the destructive behavior that I am trying to describe -- where blank lines (or line breaks) between paragraphs are stripped out if the preceeding period is followed by one or more spaces.

I just don't know how to share these images...

Jay

    
    

    
  
> line breaks ... stripped out
> if ... I had inadvertently placed one or more invisible space
> marks after the [lines]

I think that is simply a bug, not a problem with format=flowed. I think that's already filed as bug, too.

Line breaks that you manually enter (by hitting RETURN/ENTER) are normally preserved.

    
    

    
  
Ben and others,

Above all, thanks for listening -- especially to someone who barged into your forum.

The problem that I have already described may be a bug that is nominally unrelated to format flow.

But, interestingly, the problem is *completely fixed* by turning off format flow.  

That's what makes me (and others out there on the net) associate this problem with format flow).

But again, thanks for giving my views a hearing,

J.

    
    

    
  
(In reply to Jay, comment #85)
> Anyway, I now have images of the destructive behavior that I am trying to
> describe -- where blank lines (or line breaks) between paragraphs are stripped
> out if the preceeding period is followed by one or more spaces.

Why would one end a line with a space? 

Prog.

    
    

    
  
> Why would one end a line with a space?

By accident. (And sometimes, Thunderbird does that, IIRC when copy&paste - that's another bug.)
It shouldn't mess up formatting, though. It's a bug.

> the problem is *completely fixed* by turning off format flow.  

Because it's a bug in our implementation of format=flow (but no conceptual problem).

    
    

    
  
A final comment to give a precise illustration of the problem in action.

If you send the following plain-text message:

----
There is no space after this period.

There is 1 space after this period. 

There are 2 spaces after this period.  

There are 3 spaces after this period.   
 
End
----

It will be received like this:

----
There is no space after this period.

There is 1 space after this period.
There are 2 spaces after this period. 
There are 3 spaces after this period.  
End
----

Since I've finally learned a little etiquette, I won't repeat the reasons why this is such a destructive behavior/bug.

I'll just say thanks from myself and all the users who depend on you!

J.

    
    

    
  
Yup, as said, that's a know bug. Found it, it seems it's bug 125928. This has been fixed over 2 years ago, shortly after TB2 release, so please try the Thunderbird 3 RC2. I just tried it and I don't see the bug, so it seems fixed.

    
  
Depends on: 571502

    
  
Severity: normal → S3

    
  
Summary: We need Format=Flowed Evangelization → [meta] Format=Flowed (f=f) UI Visibility / Evangelization

          You need to log in
          before you can comment on or make changes to this bug.