Open Bug 144230 Opened 22 years ago Updated 8 months ago

TAB characters displayed as just 4 spaces for flowed display (expect 8-space stops) (inconsistent with compose window)

Categories

(MailNews Core :: Backend, defect)

Product:
MailNews Core
Component:
Backend
Type:
defect

Tracking

(Not tracked)

People

(Reporter: stephan, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: helpwanted) 

TAB stops are inconsistent in the compose window and the read window.

Compose the following message:

START MSG
12345678901234567890
no tab
        one tab
                two tabs
END MSG

and send it to yourself.  The first line is just the digits from 1-0 repeated,
and the second line is the phrase "no tab" on a line by itself, the third line
is a single tab stop followed by "one tab", and the fourth line is two tab stops
followed by "two tabs".  From looking at the message we deduce that the tab is
set to the equivalent of eight spaces, which is conventional for email (although
probably not specified anywhere as a standard -- seems more de facto).

Make sure you are using a fixed width font for message display.

When you receive the message and look at it it will look like this in the mail
receive window:

START MSG
12345678901234567890
no tab
    one tab
        two tabs
END MSG

From looking at the same message in the mail receive window, it appears that the
tab stops are computed at four spaces rather than eight.

The "Message Source" window shows the tab stops back at eight spaces.

I would prefer eight spaces generally, and maybe they can be configured
otherwise in preferences (although I see no reason anyone would have to not want
them be eight spaces).  However, it seems really critical that at least they be
consistent throughout!
Yuck - this is even worse than I thought.

It seems that this is not about the display of tab stops.  It would seem that
Mozilla Mail, when packaging up a message to send, actually converts tab
characters to spaces rather than sending tab characters.  Here is the test:

1.  Create a mail message with tab stops in Mozilla, send it to an address.
2.  Create the identical mail message in Netscape 4.7, send it to the same address.

The mail created with tab stops in Mozilla shows up incorrectly when received
using either of the two mail readers.  The mail created with tabstops in
Netscape 4.7 shows up properly in both of the two mail readers.
Wrong both times.  I have investigated further.  It does seem about the display
of tab stops.

Even using the test case below.  If you create a message with tab stops and send
it, it has tab stops in it and they are set every eight spaces.

On receipt, the message does NOT have tab stops in it, and the system shows them
by inserting four spaces for a tab stop.  However, in the message source window,
it is displayed correctly.  So, if you receive a message in which someone has
placed tab stops, and copy the message from the mail read window and paste it
into another program (say microsoft word) there are NO tab stops - whereever
there was a tab originally is pasted as four spaces.  However, if you open the
message source window, and similarly copy the message from that window and paste
into microsoft word, there are bonafide tabs set up properly in the resulting
microsoft word document.

This is now shown in RC3 on Win98.  This seems really bad, and not a good thing
for 1.0 at all -- displaying mail wrong seems like it should not be allowed into
the project.  I am interested that there is no discussion on this bug -- is it a
dupe?  I have searched and searched...
Severity: normal → major
Confirmed on Solaris/SPARC, 20020626.

Message is sent correctly, containing tabs, not spaces.

Composition window uses 8 spaces for a tab.

Received message display uses 4 spaces for a tab.

View source window uses 8 spaces for a tab.
Status: UNCONFIRMED → NEW
Ever confirmed: true
OS: Windows 98 → All
Hardware: PC → All
close, but not perfect.
compose uses tab stops (8 space intervals)

read uses 4 spaces.

test:

START MSG
12345678901234567890
no tab
        one tab
                two tabs
Anotab
A	one tab
AA	one tab
B		two tabs
BB		two tabs
END MSG

while I do agree that this is really annoying I'm going to mark it as normal
because I don't believe it satisfies the criteria for major.
Severity: major → normal
Keywords: helpwanted
Summary: TAB stops inconsistent in compose window and read window → TAB stops inconsistent in compose window (8 space stops) and read window (just 4 spaces)
wow, this bug is fun. that message looked fine when i got it from bugzilla. so i
selected edit as new and saved it as a draft, then i looked at the draft, this
is what i got (And what I expected to get since this is how I first encountered
the bug and why I went looking for a report):

START MSG
12345678901234567890
no tab
        one tab
                two tabs
Anotab
A    one tab
AA    one tab
B        two tabs
BB        two tabs
END MSG 

(copied directly from the pane)... I have no idea what makes draft mode special.
Whiteboard: [EDITORBASE]
editorbase-. Doesn't sound like its within editor:core. Seems to be application
specific.
Whiteboard: [EDITORBASE] → [EDITORBASE-]
Reconfirmed in 1.2.1, same mystifying behavior, and tab-formatted incoming email
continues to be useless.
timeless's testcase is odd.  It looks to me like the serializer is converting
tabs to 4 spaces, but only if the tabs are not leading the line?
EDITORBASE+ to make the serializer consistent (may want a different bug for that)
Whiteboard: [EDITORBASE-] → [EDITORBASE+]
*** Bug 88826 has been marked as a duplicate of this bug. ***
I'd argue that either our send routines should convert tabs to spaces or the
current behaviour is good. You cannot rely on tabs having a certain width. If we
do send tabs, the viewer having a different tab width makes users aware of the
problem that the reader might not see the message the same way they composed it.
If we used 8 spaces in both composer and viewer, the user might assuem that all
is fine and everyone will see a message that way, while that is not the case.
w.r.t. comment in #11, I couldn't disagree more.  Yes it is theoretically true
that tab stops can be set to different widths on different email readers -- but
practically and de facto that is rarely the case.  Mozilla's reader itself
doesn't allow tab stops to be set as per user preference - and if that were
implemented this bug wouldn't even be a bug - but I would still argue that it
would make sense, in that world, for Mozilla to behave the way every email
reader back to the old Unix System V "mail" worked - namely to DEFAULT to 8
spaces on a tab.

There is an additional problem, which is that Mozilla doesn't actually place a
tab character into the compose window and render it as four spaces.  It actually
converts the tabs to spaces -- TERRIBLE idea, so that if you copy and paste the
email into some other editor so you can see it correctly, you are still screwed.

Why do something intentionally to make our users lives miserable?
Comment #11:
> You cannot rely on tabs having a certain width. ... the reader might not 
> see the message the same way they composed it.

Spaces also look different to sender and reciever, and have another problem, so
I see no gain to substituting spaces:

  * Same problem, 8 spaces look different to sender and receiver:  The size of 8
spaces, relative to other characters, depends on the font.  Sender and receiver
often use different fonts.  

  * Other problem, spaces fail at alignment:  Using proportional fonts (as most
do), spaces do not align things unless they're used at the very begnning of a
line.  Because sender and receiver use different fonts, the "$" below may appear
aligned to the sender, but not the receiver.  Tabs address this exact problem:
-----------
Excel                        $310.00
Windows XP Pro               $200.00
Free-as-in-speech software   $priceless
-----------


> If we used 8 spaces in both composer and viewer, the user might assuem 
> that all is fine and everyone will see a message that way, while that is not 
> the case.

Could you clarify? Do you mean, 'User Joe Schmoe will see that tabs look
different when they compose e-mail and when they read it.  Therefore, they will
'discover' how tabs function'.  If I understand correctly, it seems like a
stretch for discoverability.  They'll only get confused.
The reporter of bug 221744 discovered that the tab->space conversion in the 
message display window is a result of format=flowed display.  If the message is 
not f=f, or if f=f display is turned off, the tabs are piped to the window as 
tab characters and are displayed at 8-column stops.  When an f=f message is 
displayed with f=f, each tab is given raw four-space substitution, no column 
alignment, as seen in comment 15.

I also noticed that, even with f=f turned off, you lose column alignment if 
you're displaying with variable-width text.  I'm not sure how those tabstops are 
handled, and I would say that's another bug altogether.
Blocks: 168420
Requesting blocking for 1.6b -- since flowed display is the default, most people 
are seeing tabs misdisplayed in their incoming mail.
Flags: blocking1.6b?
Summary: TAB stops inconsistent in compose window (8 space stops) and read window (just 4 spaces) → TAB stops mishandled in message displayed with flowed display (just 4 spaces)
Flags: blocking1.6b? → blocking1.6b-
Summary: TAB stops mishandled in message displayed with flowed display (just 4 spaces) → TAB stops inconsistent in compose window (8 space stops) and read window (just 4 spaces for flowed display)
I don't know if I should request a new block, but it would be good
if something could be done about this. Tab expansion should be
consistent between message composition and display. And I'd like
to be able to configure the width! (Not via UI necessarily.) The
default should probably be 8 characters.
*** Bug 94473 has been marked as a duplicate of this bug. ***
*** Bug 256105 has been marked as a duplicate of this bug. ***
Product: Browser → Seamonkey
*** Bug 279023 has been marked as a duplicate of this bug. ***
=> Backend (this issue is Core)

The problem is with the display, not with the editor.

There really ought to be a   Core :: MailNews : Message Display  component.
Component: MailNews: Main Mail Window → MailNews: Backend
Product: Mozilla Application Suite → Core
Summary: TAB stops inconsistent in compose window (8 space stops) and read window (just 4 spaces for flowed display) → TAB characters displayed as just 4 spaces for flowed display (expect 8-space stops) (inconsistent with compose window)
Whiteboard: [EDITORBASE+]
*** Bug 281255 has been marked as a duplicate of this bug. ***
*** Bug 222628 has been marked as a duplicate of this bug. ***
*** Bug 336177 has been marked as a duplicate of this bug. ***
Tabs get displayed as a certain number of spaces that brings the number of characters in the line up to the next multiple of 8. KHTML does the same thing as us so I don't think we want to change that.

The other problems in this bug are apparently in the serializer.
(In reply to comment #24)
> Tabs get displayed as a certain number of spaces that brings the number of
> characters in the line up to the next multiple of 8. KHTML does the same
> thing as us so I don't think we want to change that.

roc: Are you saying this bug is invalid, or are you affirming the expected behavior and saying it should be fixed?  I still see the symptom (see 
comment 14) with a trunk Thunderbird build.
All I'm saying is that the problem is apparently in the serializer.
sorry for the spam.  making bugzilla reflect reality as I'm not working on these bugs.  filter on FOOBARCHEESE to remove these in bulk.
Assignee: sspitzer → nobody
Filter on "Nobody_NScomTLD_20080620"
QA Contact: olgam → backend
Flags: wanted-thunderbird3?
Product: Core → MailNews Core
This problem exists in Thunderbird 3.0 b4. Funny I don't remember noticing it in 2.0.

Here is the setup: I compose fixed-font plaintext email (say, a an outline with indentation a combination of tabs and four spaces, to align the outline elements on 4-character column boundaries) and mail it. I displayed as fixed-font plaintext email in the viewer. The indentation columns are mangled because the display window interprets the tabs as 4-column... so there goes all my pretty formatting. Microsoft Outlook displays the email with 8-column tabs, is there any settings property I can change in Thunderbird to display plaintext emails using tabs interpreted as 8 columns?
I just checked a 2.0.0.23 installation and see that the problem was already happening.

By the way, I skimmed over RFC 3676 and RFC 2646, which describe the format=flowed MIME parameters, and I don't see anything about tabs at all, except where it warns to avoid using OpenPGP with format=flowed.  That is, I do not see any justification there for why tbird would be displaying tabs as 4 spaces.  I really can't imagine why the current behavior exists.
Hi.
This bug is over 10 years old. Congratulations!
It's still not solved :(
Can you look at it? Seems like easily solvable.
Can anyone tell me what the steps are if I would like to fix this in the code?
I'd start by looking around http://mxr.mozilla.org/comm-central/ident?i=OutputFormatFlowed - if you find the bug, create a patch and attach it to this bug.
The tab size for 4 is defined here:
http://mxr.mozilla.org/comm-central/source/mozilla/content/base/src/nsPlainTextSerializer.cpp#31

Is this the bug? Maybe it's enough to change this to 8.
(In reply to Peter Thomassen from comment #34)
> Maybe it's enough to change this to 8.

No, that's not enough. A tab should advance to the next tabstop, which could be any number of spaces up to 8 depending on which column the text immediately before it stopped at.

And if the display font is proportional, even that won't work to properly align a table that was created with a fixed-width font in the sender's editor -- altho I don't have a suggestion to address that case.
Just got minor problems with sending patches to Linux kernel maintainers because of tabs being converted to spaces.
Is there anyone around who can finally fix this?
This is currently being looked at in bug 1377836, which is really just another duplicate.

There a various aspects to the problem:
1) Sending a plain text message which includes tabs works.
   Tabs are shown 8 spaces wide in the compose window.
   They are contained in the sent message and in the received message.
2) When viewing a plain text format=flowed message that contains tabs,
   tabs are replaced with four spaces for display destroying any alignment.
   https://dxr.mozilla.org/comm-central/rev/408fd2480e7e9e33d7a0f5f154553f2a297a2f16/mailnews/mime/src/mimetpfl.cpp#565
   If preference mailnews.display.disable_format_flowed_support is set, tabs are displayed
   as tabs in flowed messages. Non-flowed messages already show tabs.
   In the source view, which is essentially a giant <pre> block, tabs are shown
   4 spaces wide.

Those findings aren't new, they are already contained in comment #3 and comment #14.

Re. comment #37: You can send messages/patches containing tabs with no problem. You might consider sending those messages non-flowed (why would you flow a patch?), so when viewing them, the tabs are displayed.

So far, no Core::Serializer is involved, but read on for problem 3)
3) I have a patch that displays tabs as tabs even in format flowed messages in
   bug 1377836. Replying to such a message, there are two issues:
   In the quoted content in a HTML reply, the tab is shown as one space wide, since HTML
   doesn't support tabs at all, they are just white-space.
   In the quoted content in a plain text reply, tabs are converted to spaces in the
   plain text serialiser. Refer to bug 1377836 comment #9. That's the hard part to fix.
Severity: normal → S3

Still an issue in 78.14.0

You need to log in before you can comment on or make changes to this bug.