Open Bug 1843860 Opened 2 years ago Updated 5 months ago

"Edit as New Message" and inline "Forward" not possible with some PGP-signed messages (no text, "Attached Message Part" attachment)

Categories

(MailNews Core :: MIME, defect)

Product:
MailNews Core
Component:
MIME
Version:
Thunderbird 115
Type:
defect

Tracking

(Not tracked)

People

(Reporter: info, Assigned: mkmelin)

References

Details

Attachments

(1 file)

test-decrypted.eml
2.04 KB, text/plain
Details

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0

Steps to reproduce:

Environment: TB 115 64-bit, Windows, IMAP (no local folders)

Please note that I am not sure how exactly we can reproduce that issue, because the problem appears with some messages and doesn't appear with others. However, I'll try to do my best to describe how we found the issue:

  • Compose a message that contains some text and some attachments (three TIF files in our case, but I don't believe that the kine of attachment matters).
  • Send that message in PGP-encrypted form with signature to the recipient.
  • Permanently decrypt that message in the "Sent" folder (context menu -> "Create Decrypted Copy In").
  • On that decrypted version of the message, try context menu -> "Edit As New Message", or try "Forward".

Actual results:

  • In both cases a new windows opens that should contain the new / forwarded message so that we can edit it before we send it.
  • However, in both cases, the new message is empty, and the original attachments are missing.
  • Instead the message contains a new attachment with the name "Attached Message Part".

Expected results:

  • The new message (windows) that opens when we try "Edit As New Message" or "Forward" should contain the original message text (either plain or HTML) and the original attachment, and should not contain a new attachment that hasn't been part of the original message.

Right now, I am out of time, so I can't investigate this issue as thoroughly as I would like to. However, here are a few hints; I'll focus only on "Edit As New Message" because it to our best knowledge it behaves identical to "Forward":

  • When the new message opens, it does not contain the original text (as explained above). However, it does contain the signature if a signature is configured.

  • It seems to happen only with messages that originally were not only PGP-encrypted, but also PGP-signed, and where the signature has been intact during the permanent decryption.

  • If we try "Edit As New Message" on the *original, encrypted form" of the sent message, everything works as expected. It's only the permanently decrypted form of the sent message where the issue appears.

  • However, when we composed a new and simple message with a short plain text and one short PDF attachment with signature, but without encryption and sent this message to somebody, "Edit As New Message" and "Forward" worked as expected on the sent message in the "Sent" folder.

  • The issue appears in the same form in TB 102.

It seems that TB's "Forward" and "Edit As New Message" fails with messages of content type "multipart/mixed" if they contain a PGP signature and if their structure is not simple. Weirdly, it's TB itself that produces the messages that it can't forward or edit as new afterwards. That is, it's not some exotic software at the other end that sends us malformed messages that TB can't process afterwards, but TB itself (when sending) drops such messages into the "Sent" folder.

Forwarding and editing as new are equally important. Edit as new saves us lots of time every day and prevents us from making bad mistakes that could happen when we would forward instead.

If necessary, I could provide the structure of a message that causes the issue. In this case, somebody would need to teach me how to extract that structure. I cannot provide any of those messages completely because each of them contains sensitive data that I'm not allowed to publish, not even in part.

As a final remark, a good starting point for further analysis might be to try to understand why the issue does not appear with the encrypted and signed versions of messages, but does appear with the decrypted version of those messages.

Best regards, and thank you very much in advance,

Binarus

Component: Untriaged → MIME
Product: Thunderbird → MailNews Core
Attached file test-decrypted.eml —

There are a few bugs about not being able to forward (same MIME code path as "edit as new"), for example bug 935933, bug 1065326, bug 1512720, bug 1834439. We only looked at the example in bug 935933. That has a broken MIME structure, displaying works, forwarding doesn't. That's already a hint that forwarding is picky about MIME structures.

It is know that TB fails to forward multipart/signed wrapped into multipart/mixed (bug 1710044, example attached there). This was moved to bug 1731109 and never fixed.

Observed behavior of TB 115 is that permanent decryption sometimes creates multipart/signed wrapped into multipart/mixed. We enclose an example where personal and key information have been removed. This message was created as a permanently decrypted copy of an encrypted message.

We have done further research and have found that TB doesn't process certain MIME structures correctly. The following is not a solution, but a hint for the developers. It is reproducible with all messages that can't be edited as new or forwarded we have seen so far.

Since the following code is a bit lengthy, I'll present the method and the findings first and the code afterwards.

We have saved one of the problematic messages (File -> Save as), have loaded it into a text editor and have examined its MIME structure. Then we removed the outer container which was multipart/mixed, but actually contained only one part (which doesn't make a lot of sense, does it?). When doing so, the outer container became multipart/signed.

Then we loaded the altered message back into TB, and -tadaa- "Edit as New" and "Forward" suddenly worked as expected. The signature was shown as present and valid when opening the altered message for reading.

At a first glance, RFC 1551 does not forbid the original MIME structure of those messages (but please correct me if I am wrong). That means that TB can't forward messages or edit them as new that have a certain MIME structure although that structure is syntactically correct. Please note that it's TB itself that produces messages with that MIME structure, e.g., when permanently decrypting messages.

Thunderbird not being able to parse those messages correctly also leads to another glitch (which eventually is worth a new bug report): When permanently decrypting a message that had been encrypted and signed, and then opening / reading the decrypted version, there is no hint in the UI that the message had been signed. I am quite sure that TB doesn't recognize the signature because of its inability to parse the message correctly. Please don't ask me why we then can read it at all ...

Anyway, I'd like to stress again that everything works after the manipulation shown below, including the recognition of the signature (and its correct treatment in the reading UI), and including Edit As New and Forward.

As a solution to the problem, I'd suggest a deep look into the respective libraries. Correctly parsing valid MIME structures is a sine qua non for every MUA; it absolutely belongs to the core functionality that we can't go without. It would be very good if this bug could be fixed with high priority.

As I've already stated, that bug exists in 102 as well. I can't say anything about previous versions.

Original message structure (formally valid, but Edit As New and Forward does not work, and the fact that there's a signature is not shown in the UI when reading the message, a lot of headers and other unimportant stuff left out):

To: ...
From: ...
Subject: ...
Content-Type: multipart/mixed; boundary="------------NtXTJBYIKVtkkp3g95rJ0DOO"

--------------NtXTJBYIKVtkkp3g95rJ0DOO
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------t53n1y4JQ3dRe5ZBG0P62fgR"

--------------t53n1y4JQ3dRe5ZBG0P62fgR
Content-Type: multipart/mixed; boundary="------------lIGUpMjDvpR2c2ZjLWPqRW00";
 protected-headers="v1"
Subject: ...
From: ...
To: ...

--------------lIGUpMjDvpR2c2ZjLWPqRW00
Content-Type: multipart/mixed; boundary="------------tN1nFkS26YPNtQNOwlnOGiBx"

--------------tN1nFkS26YPNtQNOwlnOGiBx
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

<lengthy base64 stuff> ...
--------------tN1nFkS26YPNtQNOwlnOGiBx
Content-Type: application/pgp-keys; name="OpenPGP_0xEDB36EEF79A4A39B.asc"
Content-Disposition: attachment; filename="OpenPGP_0xEDB36EEF79A4A39B.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

<sender's public PGP key> ...
-----END PGP PUBLIC KEY BLOCK-----

--------------tN1nFkS26YPNtQNOwlnOGiBx--

--------------lIGUpMjDvpR2c2ZjLWPqRW00--

--------------t53n1y4JQ3dRe5ZBG0P62fgR
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

<message's PGP signature> ...
-----END PGP SIGNATURE-----

--------------t53n1y4JQ3dRe5ZBG0P62fgR--

--------------NtXTJBYIKVtkkp3g95rJ0DOO--

Manipulated message structure (Edit As New and Forward work as expected, and when reading, the signature is shown correctly):

To: ...
From: ...
Subject: ...
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------t53n1y4JQ3dRe5ZBG0P62fgR"

--------------t53n1y4JQ3dRe5ZBG0P62fgR
Content-Type: multipart/mixed; boundary="------------lIGUpMjDvpR2c2ZjLWPqRW00";
 protected-headers="v1"
Subject: ...
From: ...
To: ...

--------------lIGUpMjDvpR2c2ZjLWPqRW00
Content-Type: multipart/mixed; boundary="------------tN1nFkS26YPNtQNOwlnOGiBx"

--------------tN1nFkS26YPNtQNOwlnOGiBx
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

<lengthy base64 stuff> ...
--------------tN1nFkS26YPNtQNOwlnOGiBx
Content-Type: application/pgp-keys; name="OpenPGP_0xEDB36EEF79A4A39B.asc"
Content-Disposition: attachment; filename="OpenPGP_0xEDB36EEF79A4A39B.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

<sender's public PGP key> ...
-----END PGP PUBLIC KEY BLOCK-----

--------------tN1nFkS26YPNtQNOwlnOGiBx--

--------------lIGUpMjDvpR2c2ZjLWPqRW00--

--------------t53n1y4JQ3dRe5ZBG0P62fgR
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

<message PGP signature> ...
-----END PGP SIGNATURE-----

--------------t53n1y4JQ3dRe5ZBG0P62fgR--

Dear developers, may I ask why this bug report is still in status "UNCONFIRMED"? I believe that it is easy to reproduce. If I can help with debugging, I'll be happy to do that.

Anyone else able to reproduce?

Thank you very for caring. I don't know a lot about how the TB team is internally organized and how the tasks are assigned to its members, but may I ask somebody from your team to reproduce it? Perhaps Kai (because it's a PGP issue)? It's really as easy as

  • Permanently decrypt an encrypted message (received or sent); the message should contain attachments
  • Try to forward the decrypted message to somebody

Should take less than two minutes ...

As an addition, I am nearly sure that bug #1866822 has the same underlying reason and the same resolution as this one.

It is not a duplicate, though.

Tried comment 5 with daily. I can't reproduce any issue. The decrypted message forwarded just fine.

See Also: → 1866822

Let's summarise this one more time. The messages that fail to be forwarded are those were permanent decryption creates a multipart/mixed container for the decrypted message. That doesn't always happen. We've just permanently decrypted some older PGP-encrypted messages from the reporter and the decrypted result is wrapped into multipart/mixed and can hence not be forwarded.

So in order to reproduce it, you need to decrypt a variety of messages and check whether one of them has multipart/mixed.

(In reply to Magnus Melin [:mkmelin] from comment #6)

Tried comment 5 with daily. I can't reproduce any issue. The decrypted message forwarded just fine.

Thank you very much for trying. But just to be sure: Did your test message include an attachment?

To assist you, I have the following idea:

I cound send such a message to myself to verify that it can be used to reproduce the problem. Then I could send the very same message to you (if I can find your public key). If you then permanently decrypt that message, the problem will arise. I hope that it's OK to send you a message directly :-)

Best regards, and thank you very much,

Binarus

(In reply to Binarus from comment #5)

As an addition, I am nearly sure that bug #1866822 has the same underlying reason and the same resolution as this one.

I apologize. This statement is wrong. Something has changed in TB since I have investigated the problem with the missing signature last time.

In comment #2, I have explained that the MIME part that contained the signature still did exist in the decrypted message, but that TB didn't display the signature because it obviously had problems with the outer MIME multipart/mixed container the actual message was wrapped in. After having removed this (superfluous) outer container, the signature was displayed as expected, and "Edit as New" and "Forward" worked as expected. This means that the failure with "Edit as New" and "Forward" as well as the missing signature display at that time definitely was due to the outer multipart/mixed container.

However, that was the situation 5 months ago, and because there was no further activity in this bug report until yesterday, I had assumed that the situation was still the same. Obviously, that assumption was wrong. We are currently investigating whether "Edit as New" and "Forward" now works correctly, but what we have already found out is this:

Now, when permanently decrypting a message, the MIME part that contains the signature is stripped completely. That is, there is no trace of a signature in the decrypted version of the message any more. This situation is fundamentally different from the situation five months ago, where the signature part still did exist in decrypted messages.

As a spoiler, Magnus is eventually right and we now can "Edit as New" and "Forward" messages even if their outer container is multipart/mixed. We'll conduct further tests today and ask a few other persons to confirm. I'll write another comment in a few hours.

Sure you can email me at mkmelin at thunderbird.net. Key is on the keyserver.

Did this work correctly with TB 102.x ?

No.

Here are the STR:
In TB 102, create a PGP encrypted+signed message with subject encrypted and public key attached. Send to yourself.

In TB 102, 115 or Daily, decrypt permanently and try to forward the result. This fails.

The permanently decrypted message has a combination of multipart/mixed with multipart/signed and those messages can't be forwarded as already reported in bug 1731109 which was incorrectly closed as invalid.

Even when originally creating the message in TB 115, the permanently decrypted message has an unnecessary outer multipart/mixed part, however, for some reason, no multipart/signed past is present, so the message can be forwarded.

(In reply to Kai Engert (:KaiE:) from comment #11)

Did this work correctly with TB 102.x ?

I did not work correctly with TB 102, and also did not work correctly with TB 91. I don't remember the situation with TB 78, though.

(In reply to Binarus from comment #13)

(In reply to Kai Engert (:KaiE:) from comment #11)

Did this work correctly with TB 102.x ?

I did not work correctly with TB 102, and also did not work correctly with TB 91. I don't remember the situation with TB 78, though.

Well, I meant "It" instead of "I". Was this a Freudian slip? Maybe.

(In reply to Magnus Melin [:mkmelin] from comment #10)

Sure you can email me at mkmelin at thunderbird.net. Key is on the keyserver.

Thank you very much.

I have sent you an encrypted message that will help you in reproducing the problem. If you didn't receive it, could you please check your spam folder?

To reproduce the problem, please decrypt that message permanently, then try to forward the decrypted copy, or try "Edit as New Message" on the decrypted copy. Both will fail, creating an empty message with exactly one attachment with a weird name, instead of creating a new message that is a copy of the original decrypted version.

At least, that's the situation here. Could you please shortly confirm that this happens on your side, too?

Best regards,

Binarus

(In reply to Kai Engert (:KaiE:) from comment #11)

Did this work correctly with TB 102.x ?

If it would help, I could send you an encrypted message that triggers the problem. Please let me know whether you would like me to do that (I don't want to bother you with unsolicited email messages).

Best regards,

Binarus

Thanks, for that test message I can reproduce.

(In reply to Magnus Melin [:mkmelin] from comment #17)

Thanks, for that test message I can reproduce.

Thank you very much. If I can assist you further, please let me know.

Status: UNCONFIRMED → NEW
Ever confirmed: true

Thanks Magnus for looking into it. It sounds like the issue might be unrelated to encryption, because it happens after decryption?

If you want me to look into it, please assign to me and share the test message.

Assignee: nobody → mkmelin+mozilla

It probably isn't PGP related, I reproduced it also with the SMIME, bug 1935898.

See Also: → 1935898

Hello,

Since Thunderbird 128.6.0, the commands "Edit as new", "Forward 'inline'" and "Redirect" no longer work with signed messages (using the SMIME encryption method). With this type of messages, the original message is always missing from the composition window.
These commands work fine in all other cases: unsigned, encrypted or not.

Everything was fine up to and including Thunderbird 128.5.2.

It's a fact, it's really a problem.

Thanks in advance.

Thanks

We will wait for now ;-)

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: