Open Bug 54570 Opened 23 years ago Updated 8 months ago

No end-of-signature detection in message display

Categories

(MailNews Core :: Backend, defect)

Product:
MailNews Core
Component:
Backend
Type:
defect

Tracking

(Not tracked)

People

(Reporter: len, Unassigned)

References

Details

(Whiteboard: See comment 71. Please no more comments, but you can vote.)

Attachments

(4 files)

using 2001123109 Linux
141.51 KB, image/jpeg
Details
Patch to end sig at a newline
1.10 KB, patch
Details | Diff | Splinter Review
This uses a [-a-zA-Z0-9]*: scheme to stop the sig
1.99 KB, patch
Details | Diff | Splinter Review
Now ignores some common things in real signatures...
2.51 KB, patch
Details | Diff | Splinter Review 
The current message display detects signatures and shows them in light gray
rather than standard black text. However, if a signature is encountered in a
mailing list digest message, the end of signature is not detected and the
remainder of the digest message is shown in gray.

Two obvious solutions:

a) The signature detection chould be disabled for digest messages

or

b) There should be some "end of signature" detection code rather than the
current assumption that the signature runs to the end of the message. This could
be as simple as ending the signature when encountering a line beginning with a
header token "[a-zA-Z]*:"
Reporter is this still a problem in the latest nightlies?
Yes.


Marking NEW as per reporter comments.
Status: UNCONFIRMED → NEW
Ever confirmed: true
*** Bug 102165 has been marked as a duplicate of this bug. ***
reassigning to ducarroz.
Assignee: putterman → ducarroz
Keywords: nsbeta1
Status: NEW → ASSIGNED
Target Milestone: --- → mozilla1.1
I know the reporter stated it still happens on more recent builds (11/7/2001)
but I don't see the problem when using the dup's (102361)scenario and I don't
have steps for this report.    The build I used was 20011218 on windows plain
text compose with reply and forward.  Reply doesn't show previous signatures,
but does show all the text (no greyed out) from all message bodies in the
replies.  Forwarding does show all the signatures from all the forwarded
messages in grey but the all the message bodies are not grey.  Reporter can you
try this again with a build after 12-18 and possibly give me some steps to
reproduce it.  Thanks.
The reporter in bug 102361 pointed out this problem is with mailing list digests
only. I have now subscribed to one and will investigate more.
I subscribed to a digest and have viewed several over the past few days.  I do
not see the continual grey text after a signature within a digest.  Reporter  if
this is still happening with current nightly builds (12-20-2001 or later) could
you please tell me the digest you subscribe to so I can see this problem.  Thanks.
Attached image using 2001123109 Linux — 
This clearly shows this problems still exists and is annoying ;-)
I looks like a comments is one which starts with "-- \n" and ends with a quote
:-(
Keywords: nsbeta1 → nsbeta1+
I believe this problem is not limited to Linux, but across all platforms. I'm
seeing it on the Mac OS X 2002010808 trunk build and I've seen it on Windows
builds. The list I see it most often in is the PowerList
(http://www.listmoms.net/lists/powerlist/). I don't see that the gray style
begins at a signature and goes to the end of the digest, as was originally
reported. But the signature style does overflow the actual signature and often
large portions of the next message in the digest are styled in gray. Here's some
text from the PowerList. Paste this into a mail message and see the gray style
overflow. 

Begin quoted text:

told me, you have to send back the unit for a motherboard swap...
It seems a lot of Tibook owners (myself included) have encountered 
this problem, I even saw one person who got it twice on the same 
machine....

--
 Yves Calvez http://www.linafromparis.com/
---------------------------------------------------------------------- 
Subject: Re: VNC From: "robert_brooke gravitt" <brooke@gravitt.org> Date: Fri,
11 Jan 2002 10:04:29 -0500 X-Message-Number: 14 
Al, 
VNC is an AT&T (yep, the pla

End quoted text: 

Can someone look at this before Mozilla 1.1? This is most annoying to those of
us who subscribe to list digests. Or please consider simply removing the gray
style until the bug can be fixed. Thanks. 
*** Bug 118467 has been marked as a duplicate of this bug. ***
Seeing this in 0.9.7 on Win98 too. It occurs when a -- is detected, but the
message doesn't end. In addition, any indent formatting will switch it off again
once on. This can be annoying, so could somebody please either turn it off, or
improve the end-of-message algorithm?

thanks -mike
OS: Linux → All
This is a brute, but trivial fix.
Simply unsets signature if a blank line is found.
Keywords: patch, review
Instead of stopping at a newline, I remade it so it will now stop when a
apparent mailheader is encountered.
This has much a better result for common mails than the newline hack.

PS. Why cant I obsolete my own attachment?
Adding nsbeta1- but if the patch is good, let's get it in.
Blocks: 122274
Keywords: nsbeta1+ → nsbeta1-
It now does *not* stop the signature on something that looks like a link, this
is simply done testing for a '/' after the ':'. Also some people like to write
mailto:xxx, which is now also ignored.
Would it be possible to make this pref so that user could decide how s/he wants
end of signature to be decided? I'd prefer ending signature at the first empty
line as provided by comment 13 because if the preferred length of signature is
up to 4 lines there shouldn't be many empty lines in that. I'd suggest at least
the following choices: first empty line, first new header line, to the end of
message, disable automatic signature detection.

Current signature location algorithm seems to produce incorrect results with
digest messages as others have commented. Also I'm seeing problem with
signatures that start immediatly after quote ("\n> some text\n\n-- \nSignature
here").

In addition, when replying only signature(s) should be removed, not everything
after them. This is especially relevant to digest messages where you'd want to
reply to (say) last message in digest and when you select "Reply" only the first
message in digest is quoted because all of the rest is taken as signature and
ripped of.

*** Bug 173235 has been marked as a duplicate of this bug. ***
*** Bug 179066 has been marked as a duplicate of this bug. ***
*** Bug 152820 has been marked as a duplicate of this bug. ***
Blocks: 178011
My report [Bug 179066] is verified as duplicate of this [Bug 54570],
so I comment here.

I'm not sure if this is the same problem, or just similar related problem,
digest of one of maillists I subscribe show often turns the text to italic
past the message border. (-- equally annoying as digest reader)

Here I put a fragment of the digest that turns italic, so that you can
check if the problem can be duplicated by cut&pasting to your mail message.

Someone posted text in plain-text and HTML encoding, and at the HTML part,
at the end of X-SIGSEL tag, it changes to italic, till the end of the digest.

muchan <muchan@promikra.si>

=========================================================================
- ---------------------------------
Y! Messenger en tu celular: prob?el nuevo Yahoo! Messenger para SMS aqu?
- --0-11521883-1036678483=:68149
Content-Type: text/html; charset=iso-8859-1
Content-Transfer-Encoding: 8bit

<P>Don:
<P>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; You can see the
Rolleiflex TLR FW 4.00 and another cameras&nbsp;in this link:
<P><A
href="http://www.photographical.net/photokina2002_2.html">http://www.photographical.net/photokina2002_2.html</A>
<P>It's a beatiful camera. Best regards. Carlos.-
<P><A
href="http://fotoalbum.ubbi.com/fotoalbum/bienvenido_album.asp?Id_album=13806&amp;Id_categoria=2&amp;Id_subcategoria=12&amp;Pagina_back=4">http://fotoalbum.ubbi.com/fotoalbum/bienvenido_album.asp?Id_album=13806&amp;Id_categoria=2&amp;Id_subcategoria=12&amp;Pagina_back=4</A>
<P>&nbsp;<B><I>Don Williams &lt;dwilli10@san.rr.com&gt;</I></B> wrote:
<BLOCKQUOTE style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #1010ff 2px
solid">At 01:10 PM 11/6/2002 -0300, Carlos Manuel Freaza wrote:<BR><BR>
<BLOCKQUOTE class=cite cite="" type="cite">Hi
RUGers:<BR><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Rollei formally announced the Wide-Angle Rolleiflex for all the world. You can
look it at:<BR><BR><A
href="http://www.rollei.de/en/news/index.html">http://www.rollei.de/en/news/index.html</A><BR><BR>Best
regards. Carlos.-</BLOCKQUOTE><BR>First Question.&nbsp; I went there but
couldn't find a wide angle version.&nbsp; Is it a TLR?<BR><BR>Second
question.&nbsp; When I used to go to Rollei-USA they featured a 2.8GX in the
upper left hand corner of the first page.&nbsp; Doesn't seem to be there
now.<BR><BR>Third question.&nbsp; Which of the current TLR's use a battery to
power the light meter, if any?<BR><BR><BR><X-SIGSEP>
<P></X-SIGSEP>Don Williams<BR>La Jolla,
CA<BR>&nbsp;<BR></P></BLOCKQUOTE><p><br><hr size=1><b><a
href="http://ar.rd.yahoo.com/mail/tag"http://ar.rd.yahoo.com/mail/tag/?http://ar.mobile.yahoo.com/sms.html">Y!
Messenger en tu celular</a>: prob?el nuevo Yahoo! Messenger para SMS <a
href="http://ar.rd.yahoo.com/mail/tag"http://ar.rd.yahoo.com/mail/tag"http://ar.rd.yahoo.com/mail/tag/?http://ar.mobile.yahoo.com/sms.html">aqu?/a></b>
- --0-11521883-1036678483=:68149--

------------------------------

Date: Thu, 07 Nov 2002 06:44:21 -0800
From: Don Williams <dwilli10@san.rr.com>
Subject: Re: [Rollei] TLR Lens "bloom" - definition
Message-ID: <5.1.1.6.2.20021107063919.00b34750@pop-server>
References: <E3E262EB19EC864599FE735C15E4F4A52EA79F@smsmel_nt2a.mel.sms>
<3DC9D52A.F64C74AF@pacbell.net> <00c101c2860c$42e61660$d6eff1c3@qnu350>
<3DC9DE0E.AC131D1E@pacbell.net> <001c01c28612$13c24fb0$6601a8c0@sphillips>
<00d701c28637$8ec05d90$539a9c40@VALUED20606295>

At 03:15 AM 11/7/2002 -0800, Richard Knoppow wrote:

>>I meant to proof read my post and fix the bad typing. I also meant to post
>>the title of this book. The bio comes
>>from:
>>A History of the Photographic Lens  Rudolf Kingslake, (San Diego) 1989,
>>The Academic Press ISBN 0-12-408640-3


It's still available from Alibris and Amazon, at something over $60, used,
typically $65-$75 if you are interested.  Alibris reports a "fine" copy at
$76.50 and I have found their descriptions to be accurate, more accurate
than Amazon because at Alibris you are buying from individual sellers.


Don Williams
La Jolla, CA

=========================================================================
The mailman mailing list software uses the following separator between messages
in a digest...

--__--__--

i'm not sure how 'standard' that is, but it would be excellent if that would end
the signature formatting.
*** Bug 199660 has been marked as a duplicate of this bug. ***
*** Bug 199879 has been marked as a duplicate of this bug. ***
*** Bug 201581 has been marked as a duplicate of this bug. ***
*** Bug 203420 has been marked as a duplicate of this bug. ***
Comment on attachment 68880 [details] [diff] [review]
Now ignores some common things in real signatures...

Seth, what do you think of this patch?
Attachment #68880 - Flags: review?(sspitzer)
boris, seems like a reasonable approach.

let's see what kaie thinks.
Assignee: ducarroz → bzbarsky
Status: ASSIGNED → NEW
Attachment #68880 - Flags: superreview?(sspitzer)
Attachment #68880 - Flags: review?(sspitzer)
Attachment #68880 - Flags: review?(kaie)
Seth, it't not my patch.  I'm just rustling up reviews for it. ;)

Over to patch author....
Assignee: bzbarsky → davh
We have a patch and it (just) needs reviewing. requestiong blocking 1.4 status
Flags: blocking1.4?
I've been running with this patch for over a year and there are some negative
effects, namely that some mail signatures are ended prematurely and thus some of
the lines are in normal type. Second when replying to one of these only the
grayed text is removed. That being said, I must say that I seldom notice these
negative effects, possible because it occur fairly seldom in the mails I get.
Thus compared to the current behaviour I believe that both are acceptable
compared to the completly unreadble digest-messages.

I'd personally prefer to disable the special display of signatures altogether,
since we never can make the detection work perfectly with all messages.

The sig detection code in mimetpfl.cpp says, it tries to handle signature
detection for "usenet" postings. What about restricting this detection for those
messages we are displaying from a news server?

Digest messages are sent by mail, not over usenet servers, I guess, so this
would fix the bug.

I personally would be more than happy with a pref to disable all special
treatment of sigs.  Sigs are used and abused in too many ways for anyone to
possible keep track of these days, and being able to turn it all off would suit
me just fine.  

Cheers,
Karl P
Flags: blocking1.4? → blocking1.4-
Pretty much any of various alternatives discussed over the last two years is
better than what we have now.  I'd be happy to encourage movement by  paypaling
$20 to whoever can get one of the following included in the release as the
default configuration:

1 - SIG ends with newline [ better to have a 1/2 black SIG than a 1/2 gray digest!]
2 - SIG ends with user-configurable reg-exp, defaulting to new line or message
header or dashed line.....
3 - A way to turn off special SIG processing in general
4 - Digest detection.  If the message looks like a digest, don't try to figure
out where SIGS starts and ends, since invariably the code will get it wrong.

If the code is going to make the wrong decision here, let it get the SIG looking
strange instead of the actual message contents looking invisible!!
*** Bug 178011 has been marked as a duplicate of this bug. ***
Cross-References: (pardon the cross-post)

To save others the time I spent researching this issue ... these
bugs solve essentially the same problem:  

The signature delimiter, "--[space]/n", does not reliably indicate
that the rest of the e-mail is a signature.  Assuming it is reliable, and
processing mail on that basis, causes problems.

Bug 54570  - No end-of-signature detection in message display
Bug 58406  - [RFE] let user choose signature separator
Bug 216728 - Let delimiter behaviour with position of signature be overridden

Personally, I see a broken, buggy feature; whatever the intent is, it doesn't
work.  Do it right or at least give people the option to turn it off.  I found
the bug while helping a user in the Mozillazine forums:

http://forums.mozillazine.org/viewtopic.php?p=379925&sid=5d5307d9baecffa10d28d46c649914fd#379925

I'd like to know if there is any chance this "feature" will be removed for 1.7
final, or at least have a pref to turn it off. It's lame that this has dragged
on for as long as it has, and it will be even lamer if it sticks around for many
more months as part of 1.7 / Netscape 7.2. IMHO.
You can turn the signature styling off by adding these lines to userContent.css:

.moz-txt-sig {
  color: black !important;
}

.moz-txt-sig > a {
  color: blue !important;
}

There is no such thing as "after the signature" or "end of signature". Please
file bugs against the software generating these digests.
> There is no such thing as "after the signature" or "end of signature". Please
> file bugs against the software generating these digests.

Ben, I'm not sure what you're saying here.  The software making the digest often
creates the digest by stringing together a bunch of messages that have been sent
in.  In the same way that there is no 100% effective way for Mozilla to detect
sigs, there is also no 100% effective way for the digest-creating software to
detect sigs.  

The problem is that there is no way to detect sigs reliably.  This is why we
need to rip out this feature ( my vote ), or add an option to turn the feature
off, or add an option to specify which delimiters to use for sigs. 
> there is no 100% effective way ... to detect sigs

There is: Everything starting from "-- " on its own line to the end of the msg
is a sig. It's trivial for the digest generator to replace "-- " with "--", but
it's very hard for Mozilla to detect when the sender messed up and there's
non-sig content in the (formal) "sig".
Ok, I see what you're saying.  You're saying that a sig is no longer a sig when
it is followed by something else.  So the digest program should make all the
sigs into message content.  This doesn't sound right either though, because we
would be throwing out the information that some parts of the digest are 'sig' as
opposed to normal content.  Not to mention that by the time the gazillions of
digest making program out there were changed we'd be at Mozilla 29839283.3.

Assuming we are keeping the formatting difference for sig/non-sig, it makes
sense to go by a simple rule:  Air on the side of caution, and end sigs earlier
rather than later.  

The existing suggestions ( ending at MailMan delimiters, and at mail headers )
seems like it would satisfy this rule in most case.  For those who are still not
satisfied, we should provide an option to turn it off completely or add custom
sig-ending regexps.

Alternatively, we could just always turn off sig handling for messages that have
certain headers, like the 'Mailing-List: ' header.
> we would be throwing out the information that some parts of the digest are
> 'sig' as opposed to normal content.

That's what you get for using plaintext digests. There's no way to keep this
information. Note that this problem/bug doesn't exist at all (to my knowledge)
in real MIME digests.
Product: Browser → Seamonkey
*** Bug 277245 has been marked as a duplicate of this bug. ***
*** Bug 274237 has been marked as a duplicate of this bug. ***
*** Bug 279545 has been marked as a duplicate of this bug. ***
I find this bug annoying. It is a mozilla bug, it can't be considered solely an
advocacy issue, since it is mozilla's choice to display signature text in an
eye-straining light grey! Many other colours, easier on the eyes, could be
substituted! It is also Mozilla's choice to decide that a signature can be any
length -- paragraphs upon paragraphs upon paragraphs of grey text if need be.
There is no logic in deciding that a signature can be any length; in the real
world, they are not, and it is a bug that the software doesn't recognize this
admittedly 'fuzzy' fact. If you have to, have a vote on what should be the
maximum number of lines a signature can have and live with it. How about 10 lines?

I read somewhere on the internet ((this proposal:
http://www.karlsruhe.org/rfc/son1036.txt) in section 4.32) that signatures
should be only a few lines long. I don't know if this rule is real, if it on
point to this discussion (not quite, I'm afraid), or if it is broken often
enough to be pointless, or if and how email readers should consider it (it
refers to posting agents). 

In messages where signatures are not otherwise indicated, Mozilla could only
check for sigs in the last few lines of the message body (start looking from the
bottom of the message), and if it finds none, assume there is none! That is
something to think about anyway. 

======= 
In the meanwhile ...

I guess it is not easy for mail software to accurately decide what is a
signature and what is not. I have read that some people generally dislike the
way signatures are demarcated from the rest of a message. I could easily live
with this situation if mozilla had easily accessible options for how it colours
text it supposes to be a signature.

It would be good if such an option were easy to change from one specific state
to another, as one might want to process signatures in one way for most messages
(eg demark them in grey), and only sometimes disable this greying temporarily
when you encounter an email (probably a digest email) which has this problem.

This is completely neutral then, on the question of interpreting what is a
signature and what isn't. It simply allows one to select a colour to display it,
whether or not a user agrees with what mozilla is programmed to consider a
signature.

 
===






There's a 3.5 year old patch attached to this bug which partially solves the
problem, seriously mitigating its seriousness at the expense of a new problem
which is substantially less serious than the current one.

Why has this fix not been applied?
I don't consider myself a good reviewer candidate for this patch. I would have
to dig into the code and learn before I am able to judge about it. I wish there
was a person to review, who has more insight into mime code. Maybe you could
contact ducarroz?
Attachment #68880 - Flags: review?(kaie.bugs)
(In reply to comment #43)
> > there is no 100% effective way ... to detect sigs
> 
> There is: Everything starting from "-- " on its own line to the end of the msg
> is a sig. It's trivial for the digest generator to replace "-- " with "--", but
> it's very hard for Mozilla to detect when the sender messed up and there's
> non-sig content in the (formal) "sig".

Okay, could you give a pointer to the RFC that specifies this behaviour for mail. If such RFC doesn't exist (or specifies such behaviour for news only) then I'm not sure that it's fair to require mail digest message generators to have such behaviour.

According to comment #33 from Dennis Haney the patch that has been attached to this bug does have some problems but it sounds like those problems are minor compared to current behaviour.

*** Bug 335451 has been marked as a duplicate of this bug. ***
Priority: P3 → --
Hardware: PC → All
Target Milestone: mozilla1.1alpha → ---
This bug has been open for over 6 years now.  A patch was provided nearly five years ago.  The alternative suggestion of a pref to completely disable the feature should be very easy to implement also.  Why has the "blocking-mozilla1.1" flag been removed and no replacement added?  This bug should be easy to fix.

I also dispute that the bug is "minor".  It prevents replying to messages that contain the string "--" at start of line, which is a frequent occurrence not only in the case of mail digests but also people (increasingly more common) who are unaware of the convention of using this text to set off a signature.  Replying to messages is a major feature that is broken because of this bug in a way that will be experienced by a significant number of people.  I'd say this qualifies as a major bug.
Re comment 53 and 55:
> It prevents replying to messages that contain the string "--" at start of line

No, it does not. It needs to be "-- " on its own line. Note the space! Try it. Send yourself just "--" on its own line, with text before and after. You can reply just fine.
Re: Comment #56:

Ben Bucksch disagrees with the statement that 'It prevents replying to messages that contain the string "-- " at start of line',  But in fact the statement is exactly true for Thunderbird 2.0.0.6 running on Max OS X.  So at least on that combination, the bug qualifies as a major one.
Please read again. I merely pointed out the difference between "--" and "-- ".
This is a major bug, but in the *sending* software, not Thunderbird. Thunderbird behaves correctly as specced.
What specification requires sending software not to include "-- " at the start of a line?  On what basis do you assert that doing so is a bug?
It is very common Usenet and email convention.

http://en.wikipedia.org/wiki/Signature_block
"A signature block ... is a block of text automatically appended at the bottom of an e-mail message, Usenet article, or forum post."

"The formatting of the sig block is prescribed somewhat more firmly: it ... must be delimited from the body of the message by a single line consisting of exactly two hyphens, followed by a space, followed by the end of line (i.e., "-- \n"). This ... allows software to automatically mark or remove the sig block as the receiver desires. A correct delimiter is required for a news posting program to receive the Good Netkeeping Seal of Approval."

GNKSA
http://www.gnksa.org/gnksa.txt , Section 10 and 15
10) "Included text SHOULD NOT contain the original article's signature,
unless by explicit request of the user"
(i.e. this explicitly asks to remove signatures automatically.
Thunderbird tries to follow GNKSA.)
15) "Posting software SHOULD separate any signature appended to outgoing
articles from the main text with a line containing only `-- '"

F=F includes it for backwards compatibility
http://www.apps.ietf.org/rfc/rfc3676.html#sec-4.3
"There is a long-standing convention in Usenet news which also commonly appears in Internet mail of using "-- " as the separator line between the body and the signature of a message."

Son-of-RFC-1036
http://www.chemie.fu-berlin.de/outerspace/netnews/son-of-1036_en.html
"NOTE: A more subtle posting-agent rule, suggested for experimental use, is to reject articles that appear to contain quoted signatures
...
the signature SHOULD be preceded with a delimiter line containing (only) two hyphens (ASCII 45) followed by one blank (ASCII 32)."
That doesn't answer my question.  Those quotations merely suggest that this is the conventional way of separating a signature.  And only the first one appears to be talking about e-mail as opposed to USENET. 

My question was, on what basis do you determine that sending those characters outside of a signature is a bug?  What standard says something along the lines of 'a mail user agent should not send messages containing the string "-- ", except at the start of a signature'?
It does answer your question.
Several sources say that this applies to email, too, not just usenet.
Thunderbird simply adheres to the references which explicitly say that signatures should be remove before quoting.
Given that there is a *very* widespread convention / quasi-standard (as the links show) that "-- \n" signifies a signature, putting it in without it being a signature is a bug and causes effects like what we see here.
> Given that there is a *very* widespread convention

This simply isn't true. Of the population of mail users, how many use this style of sig? Of the population of mail clients, how many support it? Almost none.

Outlook, OE, Hotmail, Yahoo, gMail, AOL, and Lotus all do not implement it -- or at least, I've never received an e-mail from one one of those apps that includes it. Other mail applications, like lists, don't support it. That's probably 95% of mail users.

On my computer and all those my company supports, I never see it unless we configured TB to use it -- and then we're usually asked to remove it by puzzled users.

The convention, to the degree there was one, is dead and buried.
We could list 100 clients that do use it explicitly. That's pointless.
(In reply to comment #64)
> We could list 100 clients that do use it explicitly. That's pointless.
> 

Ben,

Pointless is that anyone replying to the bottom of a message I send them gives me content I can't quote in a reply back.  What's the use of a mail client that can't generate replies?  Thunderbird is the only mail client that "breaks" replies like this that I've ever used and I've been using email since 1986.  

What's the big resistance to just adding a "turn this broken signature detection off" button?  Eight years, and this is still an issue affecting real people with real email.  An off switch would make pretty-much everyone with a problem with this happy, and allow us to actually work with Thunderbird in the real world where mail clients just don't seem to care that the user is replying below a signature that they're not stripping out.


> anyone replying to the bottom of a message I send them gives
> me content I can't quote in a reply back.  What's the use of a
> mail client that can't generate replies?

Thunderbird will not recognize a sig delimiter with a proper "> " quote marker (or any other quote marker). If the sender's mail client does not insert any such quote markers, then the sender is faulty, and "What's the use of a mail client that can't generate replies?"

> What's the big resistance to just adding a "turn this broken
> signature detection off" button?

The signature detection is not broken, the sender is.

But I have no problem with a pref (for about:config, no "button" in UI) to disable this feature.
Since the sender is using Outlook, I doubt that I'll have much success in (a) getting them to switch clients or (b) getting the vendor to fix the issue. ;)

There's a patch in bug 201581- but my OSX install isn't happy getting libIDL compiled from macports, so I can't see if it applies to the current sources- what's the process for trying to get the patch included/tested?  

Thanks,

Paul
(c): have them change the silly outlook default setting;)
In outlook:
Tools -> options > preferences > e-mail options > on replies and forwards | when replying to a message [prefix each line of the original message]
> The signature detection is not broken, the sender is.

In the grand scheme of things, it doesn't matter which client is broken.  There is an interoperability failure between Mozilla and one of the world's most popular e-mail clients (among other pieces of software) because of a disagreement about whether or not to follow GNKSA (a set of rules regarding how USENET clients should behave) when sending e-mail.

Asking almost all business users to reconfigure their mail client is not the solution.  Changing Mozilla to accept the de facto standard conventions is.

(In reply to comment #66)
> The signature detection is not broken, the sender is.
> 
> But I have no problem with a pref (for about:config, no "button" in UI) to
> disable this feature.

(In reply to comment #67)
> Since the sender is using Outlook, I doubt that I'll have much success in (a)
> getting them to switch clients or (b) getting the vendor to fix the issue. ;)

If a pref for signature detection is implemented, would it be possible to make it have three values: GNKSA compatible (current behavior), disabled (never detect signature) and automatic (current behavior unless the sending mail agent is outlook or other known broken agent in which case the detection is disabled)?

I believe that de facto standard is not the same as the standard. Interoperability is still important so perhaps a workaround is required for commonly used broken senders.

> I doubt that I'll have much success in b) getting the vendor to fix the issue

That's essentially what this bug is for me: Microsoft ignores standards and conventions, and is unresponsive to bug reports, and people bug us instead, to work around Microsoft, because *we* have an open Bugzilla. That's not very nice to us.

Please do bug Microsoft about it, because a) I think they need to fix it and b) I think they may be more responsive these days than they used to. Please post references (URLs) to bug reports you filed here.

> perhaps a workaround is required for commonly used broken senders.

I think that's a good idea, far better than the heuristics proposed earlier.

I can live with disabling this signature recognition only for Outlook. If anyone wants to attach a patch, I'll review it.
FWIW, the relevant code is in mailnews/mime/src/mimetpla.cpp, and it's ugly, and I'm the culprit :-(.
Attachment #68880 - Flags: superreview?(sspitzer)
So from (In reply to comment #70)
> I believe that de facto standard is not the same as the standard.
> Interoperability is still important so perhaps a workaround is required for
> commonly used broken senders.

Isn’t this one of those Tech Evangelism bugs and should be tagged as such?

(In reply to comment #71)
> Please do bug Microsoft about it, because a) I think they need to fix it and
> b) I think they may be more responsive these days than they used to. Please 
> post references (URLs) to bug reports you filed here.
>
> > perhaps a workaround is required for commonly used broken senders.
>
> I think that's a good idea, far better than the heuristics proposed earlier.

Sounds reasonable as long as the list stays compact, otherwise heuristics would be more manageable.

Not sure about comment #63, GMail happily generates and highlights signatures with -- in text/plain. It *is* commonly used and would be unproductive to remove this feature from Tbird.
I think it might make sense to check if the "presumed signature" is more than say 10 lines, and if it is, just assume the detection failed and treat the whole mail as normal text. Also if another "-- " line is found, that should also make signature detection give up.
Assignee: davh → nobody
Component: MailNews: Main Mail Window → MailNews: Backend
Product: Mozilla Application Suite → Core
QA Contact: esther → backend
I wonder why noone suggested the simplest and most obvious thing to implement: change formatting only of the *very last signature* and not try to detect signatures in the middle of the body of text, which is doomed to fail.
Because that doesn't work if the last signature is in the middle of a long mail
None of the length detection suggestions help where the reply is a line or two long, but needs quoting.  

A non-UI off preference for signature detection is still a quick, easy fix- I don't understand why, coming on seven and a half years into this bug there's still such resistance to it.  Can anyone explain that?  We're going to see more and more mobile devices doing email, and sooner or later we're going to see more bad clients, one off switch makes everything interoperate just fine.   An alternative that would work for me is an "always quote signatures" option.  

The patch to disable detection in bug 201581 is almost three years old at this point.  Please can we just get an off switch?
We have a suggestion and compromise to ignore the signature delimiter when the sending software is one of the broken email programs (User-Agent/Mailer). That's better than heuristics, as pointed out.

We don't need more comments. Please do not make any further comments.

We need somebody to implement it. I may do it this summer, if I'm in a good mood, or somebody beats me to it.
Product: Core → MailNews Core
Why not change the code that decides what colour the sig appears in. If (what mozilla considers) the sig is under, say 20 lines, print it as it does now. If the sig is over 20 lines, print it in the same colour as the rest of the email.

Treat the long ones as signatures in every way except as to the colour the UI prints them. Signatures under 20 lines get the current treatment, Signatures over 20 lines get, by default, printed in the same colour as the rest of the email's text. 

You'd need only one extra setting: colour for long sigs. No other code need change.

By default, longer sigs appear in the colour of the rest of the body.
 
If people want to change it, have a setting modifiable in user-content.css
moz-txt-sig-long

----
Also, this bug seems to happen most often (or always?) in list digests. List digests shouldn't have signatures, right?

Do most list digest contain a precedence header called "list"?

Why not, if the precedence header, among the email's headers, says "list" apply different display rules, so that a signature is not displayed as a sig, but just as normal text (since no sig should appear in a "list" in the first place)?
> Why not change the code that decides what colour the sig appears in. 

Because the problem isn't only one of the colour: the sig is no quoted when you press reply with quote, which is a much worse problem than a display issue.

> Also, this bug seems to happen most often (or always?) in list digests. List
> digests shouldn't have signatures, right?
>
> Do most list digest contain a precedence header called "list"?

Yes, or "Precedence: bulk" which is also quite common.

> Why not, if the precedence header, among the email's headers, says "list" apply
> different display rules, so that a signature is not displayed as a sig, but
> just as normal text (since no sig should appear in a "list" in the first
> place)?

Because there are also non-digest messages with these settings.  I agree that it's better than the current situation, but the best solution is still simply a pref to enable or disable the feature, which should be easier to implement.
I respectfully request that you make *all* signature "handling" optional, because your users have to deal with what happens in real life, and not what we'd like to have happen.  I agree that MS and others should fix their problems, but the reality is that isn't going to happen.  You, however, can make a positive difference.

Comment 79 says "We don't need more comments. Please do not make any further comments." yet this bug is not fixed and your users are still complaining about the problems it causes.  The life-span of this bug, the quantity of comments and the number of duplicate bugs should indicate the problems that this is causing TB users.  Like me.  Personally, I agree with comment 27, it's absurd that this bug has remained for this long!  And I do consider it a bug, not a feature.

Disabling the "feature" for certain senders per comment 71 is not going to work because in the real world there will always be broken senders not on the list.

Please at least make it optional.  We could probably live with the about:config option, though I fail to see the issue with a GUI button (comment 66).  Other hack-arounds include turning the broken behavior  off if more than one "-- " is detected in the message (i.e. a digest, as per comment 75, and my main pet-peeve), but that may not solve every problem reliably, like a simple "off" button would.

Note that the grayness problem can be fixed via a Chrome hack as per comment 40 or http://kb.mozillazine.org/Signature_display_color, but truncation of the message when replying to a digest (noted in comment 55, comment 82 and probably elsewhere) is still a major flaw in Thunderbird's handling of real life.

Ben, you are technically correct and I get that you are frustrated by other vendors unwillingness to fix non-conventional behavior.  But your refusal to provide an alternative is only hurting and annoying *your* users.  That is a very slippery slope for a developer (remember "Fun-Pigin"?).

Please stop the debate about the ideal world and listen to what your users want and need!
Whiteboard: See comment 71. Please no more comments, but you can vote.
This bug should be separated into two.

There are two problems:
1) The signature's effect on the display of email
2) The signature's effect on quotation

The first is simply a question of the colour to draw a sig meeting certain criteria. It is easy to implement any number of fixes. There is already one workaround (which affects all sigs), to change userContent.css (comment #40). None of these fixes need affect anything other than colour. Some people have suggested a GUI setting to choose the colour, since editing usercontent.css is a task 99% of users are likely incapable of even discovering. Another suggestions have been heuristics in which the program just guesses about what is or isn't a signature: these heuristics could be used ONLY for the purposes of display.

The second question goes into actually determining what is and isn't a signature, for the purposes of quotation, among other issues perhaps. This is a more difficult problem (in reality, there isn't a solution), since we have to consider aspects of a signature other than its utterly arbitrary colour. The problem exists no matter what colour the sig is drawn.

So could the owner of this bug split it? Both issues are important, but the insoluability of the second is being used as an excuse to prevent any attempt to deal with the first. They can obviously be dealt with separately.

Many bugs have been duped here, which have ONLY discussed signature colour, not quotation issues. It's not fair to dupe a bug, and because of it being related, BUT NOT EQUAL TO, a more difficult issue, put off any solution.

Don't let the perfect be the enemy of the good. It's been the enemy for 7 years. Divide and conquer.
Blocks: 713296
Severity: minor → S4
You need to log in before you can comment on or make changes to this bug.