363302 - Thunderbird Alters Outgoing Messages, Invalidating PGP Signatures

Status: RESOLVED INVALID

(Thunderbird :: Message Compose Window, defect)

Description

[David E. Ross]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.8) Gecko/20061030 SeaMonkey/1.0.6 Mnenhy/0.7.4.0
Build Identifier: Thunderbird version 1.5.0.8 (20061025)

When sending a message via Thunderbird, the message gets altered during the sund operation.  This invalidates any PGP digital signature on the message.  PGP signatures are supposed to assure both authenticity and integrity.  The former means there is assurance that the asserted sender really sent the message.  The latter means there is assurance that the message was not altered after it was signed.  Thunderbird violates this latter, which prevents the former.  

An alteration can be easily seen if the message contained a quoted prior message.  

Reproducible: Always

Steps to Reproduce:
1.  Forward a message in a Compose window.  
2.  Use PGP to sign the contents of the current window.  
3.  Send the message to yourself.  
4.  Receive the sent message.  
5.  Open the message source.   
6.  Use PGP to verify the signature of the contents of the view-source window.  


Actual Results:  
The PGP verification fails with an indication that the signed content was altered.  

The > indicators for quoted content have a space added at the beginning of each line.  That space did not appear in the Compose window when the PGP signature was applied.  

Expected Results:  
The PGP verification should succeed.  

No spaces should be added in front of quote indicators.  

I will be testing for addtional alterations.  

Note that bug #285715 (relating to incoming mail) was closed because users should verify PGP signatures in the view-source window and not in the normal display (user interface window).  In this case, however, I was able to verify the signature in the normal display but not in the view-source window.  This serious lack of consistency needs to be addressed somehow.

Comment 1

[Scott MacGregor]

how recent is this problem? did you see it in 1.5.0.7 as well?

Comment 2

[David E. Ross]

I just tried it for the first time today, in response to the closure of bug #285715.  I don't have a prior version of Thunderbird installed on which to see if this is an old bug.  When I finish my next set of tests, I will add attachments to this bug, including the unsigned text of the message to allow others to test this.  

This bug is similar to (but not the same as) bug #144998, which was marked RESOLVED FIXED on the trunk on 1 June 2005.  That bug reported extra lines being inserted after a quoted prior message.  My test indicates that #144998 is now fixed.  

Comment 3

[David E. Ross]

Attachment: Raw message content before PGP signature.

Comment 4

[David E. Ross]

Attachment: Message content as wrapped by PGP while signing.

Comment 5

[David E. Ross]

Attachment: Sent TBird message source (headers removed)

Comment 6

[David E. Ross]

In the attachments: 
1.  The raw message content has the quoted portions wrapped but the new (unquoted) portions unwrapped.  
2.  The message content as wrapped by PGP -- between BEGIN PGP SIGNED MESSAGE and BEGIN PGP SIGNATURE -- shows how the message appeared in the TBird Compose window just before I selected the Send button.  
3.  The TBird message source (sent, not received) shows not only the indenting of quoted portions; it also shows that TBird doubled the blank spaces at the beginning of any line that began with a blank space.  

This testing involved using only ASCII messages.  

In order to use a PGP digital signature to verify authenticity and integrity of a message, not one character in the message can be altered, deleted, or added after the signature is generated.  This prohibition includes leading or trailing blank spaces and control characters (e.g., tab, CR, LF).  That is why I set the wrapping point for PGP at 68 while I set the wrapping point for TBird at 72 (to prevent post-signature wrapping by TBird, which would insert CR-LF).  

Note that the signature for the message involved here did indeed verify in the TBird user interface message window.  However, when sent to someone using a different E-mail application, it cannot be verified.  This is because the message is sent as shown in the TBird view-source window, not as shown in the message or Compose window.  I have verified this by receiving the same message using Eudora, which displays the message as contained in the source for verification.  

All this raises the question whether bug #285715 was really invalid and should have been closed.  Users should be relying on the Compose and message viewing windows, not on the view-source windows.  After all, if the recipient of a signed message cannot verify the signature, the usual assumption is that something was altered either in transmission or in the recipient's E-mail application.  The recipient then informs the sender of the problem.  The sender checks the message as sent, and it's okay.  Perhaps the sender resends the message, but that doesn't help.  For a message from a non-TBird user to a TBird user, the user interface is the problem; the recipient can't use it but must instead use the view-source window.  That work-around is clumsy and not the way users expect to use TBird.  For a message from a TBird user to a non-TBird user, no such work-around exists.  

Comment 7

[Patrick Brunschwig]

I can reproduce this partially. The difference that I obtained between the message as displayed (and verified by GnuPG if called from Thunderbird) vs. the message source is line 37 of the message source.

The line is displayed (and verified OK) as follows:
. Just be aware that there are many rogue bots, crawlers, and spiders

The message source does not contain the leading ".", and is stored like this (i.e. the leading "." is missing):
 Just be aware that there are many rogue bots, crawlers, and spiders

However, the message is sent correctly (I verified it using KMail), i.e. the leading "." is present and KMail can verify the message correctly.

Finally, I cannot reproduce the bug with the leading spaces (using Thunderbird 2.0b1 RC2). 

Comment 8

[Patrick Brunschwig]

I found some further interesting detail. If I look at the message source from the window, the leading "." on line 37 is present; but if I save the message as .eml file, then the leading "." is missing.

Comment 9

[Mike Cowperthwaite]

(In reply to comment #8)
> I found some further interesting detail. If I look at the message source from
> the window, the leading "." on line 37 is present; but if I save the message
> as .eml file, then the leading "." is missing.

That's bug 339595.

David Ross and Robert Henson seem to be mostly complaining about the reflowing mentioned in bug 285715 comment 20, which is a result of using format=flowed, a feature that is turned on by default but can be disabled.  See bug 168420 and the FAQ attached to that bug.

Comment 10

[Ben Bucksch (:BenB)]

Of course the client may alter messages during receive and send operation, for more reasons than I can list offhand. This is a not a bug, but by design. Marking INVALID.

If you don't want that, there may be preferences to avoid particular alternations, e.g. using the plaintext composer (instead of HTML composer), disabling format=flowed on send etc..

Comment 11

[Ben Bucksch (:BenB)]

> No spaces should be added in front of quote indicators.

FYI, there is a semantic difference between a > as quote indicator and a > that the user types, e.g. in "where cost > potential income", with a linebreak after "cost". If the meaning is to be preserved, the mail client *must* alter the > the user typed, to avoid confusion with the quote indicator.

The format=flowed spec mandates that, but it's necessary for the mentioned reasons anyways. I think you can disable this behaviour by disabling f=f during sending.

Comment 12

[Robert H Henson]

Once again, you write nonsense. I don't use format-flowed, and never have, except in experimenting to try to find answers to the bug. There IS a bug in the way the mail is handled, it DOES break PGP messages, and it will not go away unless fixed. As with the bug 285715, the inescapable fact is that Thunderbird prevents PGP users from verifying messages and other programs do not - the exact reason, from the end users point of view, is immaterial. Those who chose to go down the Enigmail/GnuPG route are OK, the rest will have to change clients. If the usual reactionary attitude to bugs that no-one can be bothered to fix (for whatever reason) continues, I would recommend that course of action to them. That's my last word on the subject, I really can't be bothered to talk to the proverbial "brick-wall" any longer.

Comment 13

[Patrick Brunschwig]

Excuse me, but I disagree. It is totally irrelevant how the mail is stored, as long as it is presented correctly (to the user viewing a message as well as to internal functions that do something with the data).

There are many reasons why a mail is not stored in the way it is presented, it could also be stored in a special, not readable format. I think it's wrong to assume that the internal data structures could and should be used to e.g. verify a message with GnuPG. Furthermore -- as I wrote -- the mail that is sent to the recipient is not modified, thus I don't see why you would care how the message stored at all.

Comment 14

[David E. Ross]

Windows 7 Ultimate SP 1 (x64)
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1
PGP 10.1.2

I do not know when it happened because it has been a long time since I tested this.  However, it seems that this is no longer a problem, at least for messages that are received by Thunderbird and decrypted by PGP in the Thunderbird window.  The resolution might be a result of my disabling Format=Flowed for both outgoing and incoming messages.  

Note:  I have not tested this relative to a message received by an application other than Thunderbird.