242896 - HTML email uses incompatible <pre wrap=""> tag

Status: RESOLVED WORKSFORME

(MailNews Core :: Composition, defect)

Description

[Dan Higgins]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040207 Firefox/0.8
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040207 Firefox/0.8

A friend of mine just reported to me that my mail program sucks. Oops. He's
using a MS product and he's forced to scroll to the right to see all the text of
my messages. No wrapping occurs.

I noticed in a view source that the HTML mime part uses <pre wrap=""> tags
around message text. This works fine in Gecko based HTML viewers, but it's
non-standard and unsupported on other HTML viewers.

This *appears* to be caused by replying to an HTML message from a MS email program.


Reproducible: Always
Steps to Reproduce:
1. Make sure you set prefs to send both plain-text and HTML parts, and include
original message text inline on replies.
2. Have a friend email you a message from a MS mail prog.
3. Reply to your friend's message.
4. Have your friend reply again back to you.
5. Reply once more, but this time send it to yourself.
6. View source on this last reply. I see a <pre wrap=""> tag in there starting
where the last message body was included.


Expected Results:  
Don't use the <pre> tag, or any non-standard HTML tags. It would be better to
use <div style="margin-left: 10px; border-left: 1px solid #000000"> or similar
to wrap previous message bodies.

Comment 1

[Dan Higgins]

I just reproduced this on Mozilla 1.7rc1 (Linux). It's even easier to get than
the steps above.

To reproduce:
1. Make sure you set prefs to send both plain-text and HTML.
2. Have a friend send a PLAIN TEXT message to you from a MS emailer.
3. Reply to the message, but send it to yourself.
4. View source on the reply above. I see the <pre wrap=""> there in the HTML part.

This <pre> tag simply gets repeated for however many replies there are in the
thread.

Comment 2

[Scott MacGregor]

I've seen lots of recent complaints about this. I don't know that we changed
anything recently about this. Would be good if a volunteer could test out
various milestone releases to find out when this first appeared.

My first suspicion is the format=flowed stuff.

can you attach a test message sent from outlook? (File / Save As Files, save it
as a .eml file then post to this bug).

If you look at the view source for the REPLIED message (which adds the pre) do
you see any format=flowed text in the view source?

Comment 3

[Scott MacGregor]

ignore my request, I was easily able to trigger a message that caused this.

Comment 4

[Scott MacGregor]

cc'ing Ben B. He might understand the format=flowed stuff better than I.

We are wrapping the plain text from outlook in a pre tag here:

http://lxr.mozilla.org/mozilla/source/mailnews/mime/src/mimetpla.cpp#231

when I try the same test in mozilla, the plain text message I am replying to has
a format=flowed attribute on it so we don't even run through this code so no pre
tag is applied.

Comment 5

[Ben Bucksch (:BenB)]

> Don't use the <pre> tag, or any non-standard HTML tags.

<pre> is perfectly standard HTML. It's not incompatible at all, so the summary
in INVALID.

We always used <pre> (as <pre wrap="">), in fact, IIRC even 4.x did so. I can't
explain a sudden rise in complains, unless something made HTML mails more
likely. mscott, did something change in *that* area?

We are using <pre> to preserve the whitespace in the original mail. Using <div>
as suggested would completely collapse consecutive spaces etc. (e.g. when used
for marker, tables, ascii-art etc.), which would be an alteration of the quote
which many would find unacceptable.

We should not rely on CSS (<div style="...">) - we are talking to MUAs, not
full-blown browsers, and many of them may not have CSS implemented, even if they
support HTML.


It is true that only Mozilla and Communicator 4.x support the "wrap" attribute
of <pre>. However, we regard it as optional. If that is ignored, the quote still
displays as sent by the original sender, with what should be no more than 80
chars per line. If lines are longer, it's a problem of the original sender.
Other people have to cope with 80-char displays as well (many console users,
e.g. mutt users), so Outlook users here suffer from the same (actually even
lesser) problem they often cause to others. No pity ;-).


We could convert consecutive spaces to non-breaking spaces, and IIRC we actually
do in the flowed converter. I don't remember why we didn't in normal plaintext
mode, maybe to keep the code simpler, maybe because there were other reasons.

Comment 6

[Ben Bucksch (:BenB)]

> It's not incompatible at all, so the summary in INVALID.

strike that.


Note that quoting plaintext in HTML has always had problems, even in
Mozilla/4.x, for example not honoring the variable width font pref.


Anyways, I argue that using <pre> is correct here. <pre> is specifically for
plaintext stuff, and we have plaintext here. <pre> is the straightforward and
obvious tag to use. How that is rendered is up to the *displaying* user-agent.
For example, the HTML spec says for <pre> (among others) "May disable automatic
word wrap" - note "may".

Comment 7

[Scott MacGregor]

Ben I think the problem here is that when someone in outlook views the replied
message from us they see one huge line of text without any breaking because pre
is preserving all white space.

I'll try to post a screen shot unless Dan does it first.

I'm pretty sure that behavior is somewhat new because I just started getting
complains about it in 2004 and I've been starting to get a lot of them. So I
don't think it's something we've been doing sinced 4.x days although lxr does
tell me I'm wrong.

Comment 8

[Ben Bucksch (:BenB)]

Oh, I thought you spoke about soft-wrapping when normal lines are wider than the
window.
Linesbreaks in the original mail definitely are supposed to be preserved in
quotes, no matter which recipient MUA (as long as it's not terribly broken).
What you describe would indeed be a major bug.

Can you as well attach the source to a sample msg (sent by Mozilla, with quote)
which shows the problem? Ideally also the original, quoted msg.

Comment 9

[Dan Higgins]

Attachment: Sample text message from MS mailer and mixed format reply.

I have changed the hosts, domains, and names in the headers to protect the
innocent.

Comment 10

[Dan Higgins]

The issue again is converting plain text to HTML in a reply. If a user's mail
app is displaying the plain text part, there is no problem. But if the mail app
displays the HTML part, the previous message bodies all appear in this <pre
wrap=""> tag and any non-Netscapeish viewer will ignore the wrap attribute.

We *want* the mail app to do its own wrapping for the HTML. So keeping each
paragraph on one line is the right thing to do. It's the markup around the
paragraphs that really should be looked at.

I still strongly suggest that only standard HTML-4/CSS-2 be used, except of
course for custom built-in attributes and classes *that don't affect display*.

Comment 11

[Dan Higgins]

Attachment: Actual email showing <pre> tag & long line problem.

If you view this in a text viewer, you can see in the HTML part that the long
lines don't get broken up but they do get put into a <pre wrap=""> block.

Comment 12

[Ben Bucksch (:BenB)]

> If a user's mail app is displaying the plain text part, there is no problem.

Actually, the plaintext part of the msg you attached exhibits the same problem.
Some clients may deal with it in that case, but not others. For example, some
console clients with 80 char consoles may not break the lines, resulting in the
same scrolling problem you see with the HTML part. And I guess the same problem
was in the original (Jon's) mail.

Comment 13

[Dan Higgins]

Attachment: The original msg received that caused the problem.

Comment 14

[Dan Higgins]

(In reply to comment #12)
> > If a user's mail app is displaying the plain text part, there is no problem.
>
> Actually, the plaintext part of the msg you attached exhibits the same 
> problem.

I see what you mean. I'll try to break it down once more, with this new bit of 
info:

1) In the HTML part of a multipart reply to certain messages that were plain-
text, long paragraphs are NOT broken up but ARE wrapped in a <pre wrap> tag. 
This causes standard HTML viewers to put the whole paragraph on one line.

2) In the TEXT part of a multipart reply to certain messages that were plain-
text, long paragraphs are NOT broken up but SHOULD be.

For #1, the answer should NOT be to break up the long lines; instead let the 
HTML viewer do the wrapping, using standard HTML tags.

For #2, the answer should be to break up the long lines.

Perhaps these 2 things should be tracked separately. ?

Comment 15

[Ben Bucksch (:BenB)]

The long lines are in the original mail already, so the other mailer (Microsoft
Outlook Web Access) *sends out* these long lines itself, and we just quote them
verbatim: we don't break them in the plaintext quote and just wrap them in a
<pre> in HTML.

Let me get this straight: OWA sends out msgs with extremely long msgs, which is
known to cause break some clients (those which do not break long lines in
plaintext while displaying). But when the client gets these long lines back, in
HTML <pre wrap>, it doesn't break them up, and its users are complaining about a
problem they caused *themselves*, for others, and blame Mozilla. Really great.

So, what do we do? Both plaintext and HTML part show the problem, because the
problem was in the original mail already.
 - We could say that it's not our problem and that we need to quote verbatim.
 - Or we fix it up, but how?
   - We can't wrap the plaintext part normally (at 76 chars or so),
     because that will e.g. cause the comb effects that MS clients are
     infamous for.
   - If we wrap only very long lines (e.g. > 120 chars), we'll still see
     the problem with paragraphs with e.g. 100 chars.
   - We could scan the msg and if there's any such very long line,
     break them all, but libmime's design doesn't allow that
     (processes line by line).
Given these options, I'd say leave the lines as-is, at least in plaintext.

I could very well argue that <pre> is the right tag to use in HTML - it's made
for the purpose of displaying plaintext in HTML, and we do just that: Take the
plaintext, stuff it in <pre> and that's it. What gets stuffed in there and how
that is later displayed in other clients is not our problem. The same code is
used in for msg display in Mailnews and quoting, so we add the "wrap" attribute
for ourselves.

OTOH, the receiving client has very little chance to break the lines, short of
wrapping all <pre>s at window width (you could argue that this is a good idea)
or supporting the wrap attribute.

> let the HTML viewer do the wrapping, using standard HTML tags.

Which ones? As I said, consecutive whitespace must be preserved. We can't rely
on CSS either, the msg must display OK (with various definitions of "OK") with
very basic HTML renderers.

We could (and probably should) add the width attribute (we already do add the a
non-standard "cols" attribute, it seems). However, that is not widely supported,
I read. But at least we do the right thing.

The only other option I see is to use nsbr, as mentioned above, but I am
somewhat reluctant of such a large change, because it then will happen for *all*
uses of plaintext - all normal internal msg display, all quotes.

Of course, there's also the option of "screw 'em", i.e. let MS fix it on their
side by stopping to send these long lines in the first place, which would
arguably be the right fix.

Actually, I *think* we had this bug filed long ago (bug# 3nnnn something), and
it was WONTFIXed, but I can't find it anymore even after long searching.

Comment 16

[Dan Higgins]

I give on reworking the composition code for text/plain messages. Not using
<pre> altogether does make it quite a bit harder to make the text/html part look
like the original plain text. You'd have to treat consecutive spaces and
newlines (not hard in theory, but not worth the work for treating plain-text
messages, IMO).

Still, IMO, the wrapping for the entire text/plain part (incl. indented
original) should occur when the email is sent out, based on the *user's* own
"Wrap plain text at ..." setting. This is apparently what Outlook does, since
that's what I used before I got this complaint from my friend using the web mail
client. The comb effect you mention is from their wrapping it in a lazy way
(including the indentation itself in the width of each line). But this is aside
from this bug so I let it be.

FYI, CSS 2.1 adds a "white-space: pre-wrap" attribute. The current Gecko does
not support it (wish it did), but I've tested a block like this:

<pre wrap="" style="white-space: pre-wrap">some long line</pre>

and it does work. This would at least be supported on other HTML viewers in the
future as they support more CSS.

That's all for me. Cheers.

Comment 17

[Ben Bucksch (:BenB)]

> <pre wrap="" style="white-space: pre-wrap">some long line</pre>

We could (and probably should) certainly do *that*, but I guess no renderer
supports it /yet/.

Comment 18

[Ben Slade]

This seems to be similar to bug 196033.  The discussion below headed off into
the <pr wrap=""> direction because of what the reporter noticed, but it could be
part of a more general problem covered in bug 196033 and the reporter here just
didn't notice that.

Basically, when Moz forwards plaintext non-flowed email content, the content
gets converted to HTML <pre> content in forwarded emails.   It makes the
forwarded emails unreadable (no line wrapping) even in Moz.

PS.  Hi Scott, this is Ben way back from AOL Email Ops (no longer working for AOL)

Comment 19

[plop]

This may be linked with bug 56921 bug 267832 and bug 116225

I think my problem (and solution) may have plce in here, instead of opening a new bug:

HTML emails from AOL or Hotmail may contain some <PRE things. This cause problem with line wrapping. I end up with very long lines, thats are severall screens large.

There should be two new features against this:
- a "force wrap" option in the "view" menu specifically for HTML emssages ("message body as" choices are not enough) (as for source, see bugs mentioned at top of message)
- an optionnal re-wrap filter with a tick option in the profile; at the moment, I filled .thunderbird/default/XXXXX.slt/chrome/userContent.css with

pre { white-space:-moz-pre-wrap }

it wraps long lines as I want, but, it insert unnecessary empty lines in some cases when viewing the email, and when replying. It's obviously not a perfect solution, but at least, for now, I dont have the horizontal bar anymore :)

Example of what is a problem for me:

</SPAN></TT></PRE><PRE style=3D"BACKGROUND: white"><TT><SPAN style=3D

When AOL/Hotmail put this in the middle of a paragraph, TB get's mad.

Comment 20

[Jorg K (CEST = GMT+2)]

I'm not aware of <pre wrap=""> being used in current TB.
If it's still a problem, please open a new bug.