Closed Bug 285715 Opened 20 years ago Closed 18 years ago

Thunderbird alters inbound PGP signed email so that it cannot be verified

Categories

(Thunderbird :: General, defect)

Product:
Thunderbird
Component:
General
Platform:
x86
Windows XP
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: nbc, Assigned: mscott)

Details

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.6) Gecko/20050223 Firefox/1.0.1
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.6) Gecko/20050223 Firefox/1.0.1

Inbound mail is altered by Thunderbird in some way so that messages signed in
the body of the message using PGP cannot be verified. PGP reports that the
message has been altered. It is necessary to use View | Message Source to verify
the message. Occasionally this fails too, but it is not possible to show / prove
that this relates to Thunderbird. The problem has been known for some time, and
is regularly commented on in PGP newsgroups and PGP help forums, but does not
appear to have been reported as a bug in Bugzilla (search for "PGP" in the
Thunderbird section produces no relevant data.

Reproducible: Always

Steps to Reproduce:
1.Read messages signed in the message body with PGP
2.PGP reports that the message has been altered
3.Check to make sure other email programs can correctly verify the message -
they can.
Actual Results:  
Using View | message source enables the message to be verified.

Expected Results:  
The message should have been verifiable in the main mail window. The software
should not have altered the message to prevent verification.

I am using PGP 8.1, but the problem is known to exist with all versions of PGP
and with GnuPG.
imap or pop3? Does this happen on messages smaller than 25K or so? Or just
messages > 25K? If this only happens on large imap messages, it probably has to
do with IMAP mime parts on demand, where we don't load the whole message, but
just the parts we display inline.
(In reply to comment #1)
> imap or pop3? Does this happen on messages smaller than 25K or so? Or just
> messages > 25K? If this only happens on large imap messages, it probably has to
> do with IMAP mime parts on demand, where we don't load the whole message, but
> just the parts we display inline.

POP3, and any size messages at all. 


Dupe of 144998?
No, I don't think so. It might have to do with the x-mozilla-status lines we
add. Are you using enigmail?
Reporter said he's using PGP 8.1, and enigmail doesn't work with PGP. (Though
based on my own experiences and anecdotal evidence, I suspect enigmail+gpg would
fix 90 percent of his problems.) And x-mozilla-status should be well outside of
the signed material, right? Whether using inline or PGP/MIME, the headers are
never part of the signed material.
yes, x-mozilla-status headers are headers. Other than that, we don't message
with the message - we do use the native line endings, but if you're on windows,
that should be the same
I have installed and been using the new PGP 9.0 beta over the weekend. The same
problem exists with messages PGP signed in the body of the message. Emails
signed in-line verify OK, as the new PGP 9.0 runs as a service and reads them
"before Thunderbird gets its hands on them", so to speak. Discussion elsewhere
in the newsgroups and PGP forums suggests that (and it seems likely and obvious
to me) that the problem lies not in the headers, but from Thunderbird's known
propensity for altering the message body as it is received - perhaps by
inserting breaks or in some other way altering the formatting of the message.
The fact that most messages verify if View | Message Source is used and the
messages are tested therein would indicate that it is a problem with altered
formatting within the main read window.

The comment that Enigmail/GnuPG would solve the problem is possibly true, but I
have no wish, not should I need, to change to a program that is out of date by
current standards, and lacks features of the "official" program. In any case,
the problem is within Thunderbird (not PGP) and should be rectified regardless,
as it may affect other programs too.
what is the "known propensity for altering the message body as it is received"?
As I said before, other than using the native line endings for storing the
message, we don't alter the message body at all. How is PGP actually reading the
contents of the message, or do you know?
(In reply to comment #8)
> what is the "known propensity for altering the message body as it is received"?
> As I said before, other than using the native line endings for storing the
> message, we don't alter the message body at all. How is PGP actually reading the
> contents of the message, or do you know?

Thunderbird changes the layout of the message between reading it and displaying
it in the normal window. I don't have the technical ability to say how - it may
be the carriage returns, the spacing or whatever - but the message in the main
read window has been altered from the message in the View | Message source
window. This is well known and discussed in the PGP newsgroups and elsewhere. I
don't need to be technically well informed to know that the two are different,
the original message can be read and the signature verified by PGP, in the
normal window it can't  - PGP says the message has been altered. That much is
inescapable. I have no idea how PGP reads the message - I'm not Phil Zimmermann
- but suffice to say that it can detect if the message is not EXACTLY the same
as the original - that's the whole point of the program. It's up to the TB
programmers to find out what exactly is changed - but changed it certainly is. A
further, if  purely empirical, piece of evidence is that it does not seem to
happen with short test messages (much less than one line of text) - so it
probably has to be something to do with line ends, carriage returns, and/or
spacing - characters are inserted or lost. 

I'll try to get the exact details, but a recent message in alt.security.pgp
states that a bug was originally filed in 2002 (can't confirm that, but that's
why I was expecting to find one when I originally searched) and that nothing has
been done since then. 

Presumably, since Enigmail/GnuPG does not show this problem, that combination
must read the original message source not the message as viewed, so the
probability is that the Enigmail developers could point you in the direction of
what is being changed?
I've spoken to the author of the message I referred to, and it would appear that
we may well be talking about the same bug as #144998, which is still to be fixed
*two years later*. I'm told that the problem is in two parts - both concerning
quoting. TB converts ">" to ">>" in between downloading the message and
displaying it, but hides the extra symbol in the display window. The second
problem is that, again somewhere between downloading and displaying, it inserts
an extra carriage return before and after the quoted section and, once again,
hides it in the display window. 

Does this help - can it be fixed at last?


An additional problem, also part of this bug, is that when you "reply" to a
message PGP inserts a space before the quote carat which also breaks PGP
signatures. Removing the space by manual editing enables correct authentication.
Here's an example of the bug. I found a message in a Netscape usenet group with
quoted material in it. I saved it as a TXT file and an EML file.

The TXT file looks like this:
> >I can't seem to install extensions anymore on my FF 1.0.4. I click the
> >install button and nothing happens. I have admin rights on my WinXP

The EML file looks like this:
>I can't seem to install extensions anymore on my FF 1.0.4. I click the
>install button and nothing happens. I have admin rights on my WinXP

And there are instances where you get " >" instead of the two examples above.
This is an automated message, with ID "auto-resolve01".

This bug has had no comments for a long time. Statistically, we have found that
bug reports that have not been confirmed by a second user after three months are
highly unlikely to be the source of a fix to the code.

While your input is very important to us, our resources are limited and so we
are asking for your help in focussing our efforts. If you can still reproduce
this problem in the latest version of the product (see below for how to obtain a
copy) or, for feature requests, if it's not present in the latest version and
you still believe we should implement it, please visit the URL of this bug
(given at the top of this mail) and add a comment to that effect, giving more
reproduction information if you have it.

If it is not a problem any longer, you need take no action. If this bug is not
changed in any way in the next two weeks, it will be automatically resolved.
Thank you for your help in this matter.

The latest beta releases can be obtained from:
Firefox:     http://www.mozilla.org/projects/firefox/
Thunderbird: http://www.mozilla.org/products/thunderbird/releases/1.5beta1.html
Seamonkey:   http://www.mozilla.org/projects/seamonkey/

To the best of my knowledge this bug has existed for some years and is still not
fixed. I have had to switch to Enigmail/GnuPG to avoid it. Regular reports of
problems arising from it appear in the PGP newsgroups.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Thunderbird version 1.5 (20051201)

This is still a problem.  For anyone using PGP to encrypt or sign E-mail messages containing quoted text or for anyone using PGP to sign newsgroup messages, this makes Thunderbird unusable (unless that person uses the workaround in the original Description).  For that reason, this bug should have Severity = Major.
Severity: normal → major
My research on verifying pgp signed mail

Thunderbird 1.0.6.

>>
>> 1. Thunderbird while displaying the message alters ">" at line begin
>> into ">>"
>> 2. Thunderbird displaying the message alters single "CR LF"
>> characters ending paragraph into "CR LF LF". [before and after quoted text]

on Thunderbird 1.5 (Windows/20051201) it was semifixed:

Problem of adding "LF" before and after citated text paragraph was fixed BUT they made it WORSE adding additional space into cited line.

character ">" at line begin is now replaced with "> >" :)

Of course ^U operation or view->message source is proper solution (as
was in 1.0.6) before. Source mail is properly displayed there and its
signature is verified correctly! 

Wiktor

Thunderbird version 1.5.0.2 (20060308) was supposed to be a "security fix".  This bug is really a security bug, affecting the use of PGP.  However, this bug was not fixed in that version.  

I have [View > Message Body As > Plain Text].  But when I view newsgroup messages, I see blue vertical bars in place of greater-than symbols (>) where prior messages are quoted.  When I view the source of such a message, I see the greater-than symbols, which is why the message source can be verified but not the displayed message.  Aside from this indication of message quoting, I noticed no other visible difference between the displayed message and the message source.  

Note that, for messages I have tested, the message source shows plain ASCII formatting, without any HTML.  Thunderbird should not be altering ASCII-formatted messages.  
(In reply to comment #17)
> Thunderbird version 1.5.0.2 (20060308) was supposed to be a "security fix". 
> This bug is really a security bug, affecting the use of PGP.  However, this
> bug was not fixed in that version.  

I have the same Problem with Thunderbird 1.5.0.2 and PGP 8.1. You can see in message source that Thunderbird adds an extra space character on every line which already begins with space characters.

    message window           message source window
    ==============           =====================

    "text"            -->    "text"
    " text"           -->    "  text"

You can verify this by signing a new message which has space characters in the beginning of several lines and verifying the signature of the message source in outbox.





(In reply to comment #18)
The bug is still in version 1.5.0.4. Is nobody else interested in fixing this bug?
I think there is some misunderstanding here. The reporter writes that the message _can_ be verified using the message source. Thus, for certain Thunderbird does not modify the message (otherwise that verification would fail as well).

However, the user interface does indeed modify the text body. There are features like beautifying text, re-wrapping and re-flowing which alter the text displayed on purpose (these are features of Thunderbird, and most of them can be disabled). I think it's just a false assumption to try to verify a message from the displayed form, this is basically bound to fail. Several of the standards around email are just not clearly defined with inline PGP (or even orthogonal to it), and displaying the message for sure pro:
-- content-encoding (should it be applied before or after the signature creation, a hot candidate for problems is quoted-printable)
-- "flowed" messages
-- HTML
Status: NEW → RESOLVED
Closed: 18 years ago
Resolution: --- → INVALID
Whilst it is quite correct to point out that the alterations only occur in the user interface, that is where all the users I have come across are attempting to use PGP. Most do not realise that they can hit CTRL-U and get the original message. Why should they have to? Most do not wish to be technicians, they just want to use the program for it's express purpose - sending mail. It doesn't happen with any other program I have used to date. I switched to using GnuPG/Enigmail some time back to get round this problem, (which works just fine) as I don't have time to play around with every message. However, if I were a dedicated PGP user it would stop me from using Thunderbird. It just depends how keen Mozilla are to promote the program as a viable alternative to Microsoft and others - if they are keen, it *does* need fixing.

Actually, from memory, I believe that when replying to quoted material, TB *does* corrupt the outbound message by adding space before the quote carat sign, so it will not verify at the other end anyway - regardless of the UI.




See bug #363302 for the case of an outbound message.
Status: RESOLVED → REOPENED
Resolution: INVALID → ---
This bug was marked INVALID, and correctly so. Reporter: Please keep it like that, thanks.
Status: REOPENED → RESOLVED
Closed: 18 years ago → 18 years ago
Resolution: --- → INVALID
The bug is NOT resolved, there is merely a workaround for it - which we all know about. The bug in the outbound mail still exists, and there is no workaround for it. It STILL needs fixing. I will certainly leave it as it is, I am tired of arguing with people who can't see the wood for trees. The bug has been the same for some years and will, no doubt, stay that way. Since GnuPG and Enigmail together solve the problem for me, the PGP users who still cannot verify messages will have to take up the cudgel on their own behalf, or switch back to another mail client. 
FYI, it is invalid, because this is by design, not a bug. The client may not display to the user exactly 1:1 what was in the raw email (RFC822), in the same form. There are many reasons: MIME, quote recognition, smiley icon replacement etc.pp.. If you want the raw mail, that's what Message Source is for.
Windows 7 Ultimate SP 1 (x64)
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1
PGP 10.1.2

I do not know when it happened because it has been a long time since I tested this.  However, it seems that this is no longer a problem, at least for messages that are originally composed by Thunderbird and encrypted by PGP in the Thunderbird composition window.  The resolution might be a result of my disabling Format=Flowed for both outgoing and incoming messages.  

Note:  I have not tested this relative to a message composed by an application other than Thunderbird.
You need to log in before you can comment on or make changes to this bug.