Closed Bug 1653651 Opened 4 years ago Closed 4 years ago

Unable to open encrypted messages since version 78

Categories

(MailNews Core :: Security: OpenPGP, defect)

Product:
MailNews Core
Component:
Security: OpenPGP
Type:
defect

Tracking

(thunderbird_esr78 fixed, thunderbird79 fixed)

Status:
RESOLVED FIXED
Milestone:
Thunderbird 80.0
Tracking Flags:
Tracking Status
thunderbird_esr78 --- fixed
thunderbird79 --- fixed

People

(Reporter: christoph.korn, Assigned: KaiE)

Details

Attachments

(1 file)

Bug 1653651 - Hotfix for RNP failure to process long Armor Header lines. r=mkmelin,benc
47 bytes, text/x-phabricator-request
wsmwk
: approval-comm-beta+
wsmwk
: approval-comm-esr78+
Details | Review

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.119 Safari/537.36

Steps to reproduce:

Got an update to Thunderbird 78 on Solus Linux today.
Was informed that I need to import my GPG keys, so did I.
But cannot read any encrypted messages now.

Actual results:

Encrypted messages are just shown blank.
The encryption icon opens a message:
"Message cannot be decrypted
There are unknown problems with this encrypted message".

Expected results:

I expected to read encrypted messaged just like before the update.

Component: Untriaged → Security: OpenPGP
Product: Thunderbird → MailNews Core

Your distribution updated Thunderbird to 78? The Thunderbird project had recommended to NOT take this upgrade yet for users who might have been using Enigmail, because the integrated OpenPGP feature isn't yet considered stable. It was advised to stay with TB 68.x until the release to TB 78.2

The OpenPGP functionality in 78.0 is incomplete and there are known bugs, that we're still working on.

It may be difficult to diagnose remotely what happened. I'll have to ask lots of questions.

You were a previous user of Enigmail, correct?

When you started TB 78 the first time, did Enigmail upgrade to version 2.2 ? Did Enigmail suggest that you need to migrate your data?
If not, then you didn't go through the recommended migration of settings.

If you didn't see a migration window, did you manually set the configuration setting to enable OpenPGP in TB 78 ?

You said you tried to fix it by importing your key. Did you use menu "tools / openpgp key management" and "file / import secret key" for that?

Does the OpenPGP key management window show your secret key?
If you double click it to view its details, does it say the type is "key pair (secret key and public key" ?

Did you confirm "yes, treat this key as a personal key"?

Do you know if the message you have received is a PGP/MIME message (view source, you see "Content-Type: multipart/encrypted") and additional text that says the message is PGP/MIME.

Or is it an INLINE message (no headers mentioning encryption, but a message of type text/plain with the body simply being "-----BEGIN PGP..." ?

If it is inline, maybe you're running into bug 1651896 ?

Flags: needinfo?(c_korn)

You were a previous user of Enigmail, correct?

Yes, I used it for a long time.

When you started TB 78 the first time, did Enigmail upgrade to version 2.2 ? Did Enigmail suggest that you need to migrate your data?
If not, then you didn't go through the recommended migration of settings.

Enigmail is on version 2.2 and it says last updated today, so it seems yes. There was a dialog telling me to import my keys yes. I entered my passphrase but there were also keys which could not be imported. But I only saw the checksum so I cannot tell if these were old keys.

If you didn't see a migration window, did you manually set the configuration setting to enable OpenPGP in TB 78 ?

I think I did not manually enable GPG.

You said you tried to fix it by importing your key. Did you use menu "tools / openpgp key management" and "file / import secret key" for that?

No, I did it with Account Settings->End to End Encryption. There I added my existing keys as personal keys. I selected the one with no expiration date. Should be the correct one.

Does the OpenPGP key management window show your secret key?

Yes

If you double click it to view its details, does it say the type is "key pair (secret key and public key" ?

Yes

Did you confirm "yes, treat this key as a personal key"?

Yes

Do you know if the message you have received is a PGP/MIME message (view source, you see "Content-Type: multipart/encrypted") and
additional text that says the message is PGP/MIME.

It says "Content-Type: multipart/encrypted" yes.

Or is it an INLINE message (no headers mentioning encryption, but a message of type text/plain with the body simply being "-----BEGIN PGP..." ?

My message window is completely empty. Only when I click the encryption info button on the left a dialog opens which says:
"There are unknown problems with this encrypted message."

Flags: needinfo?(c_korn)

Can you please open the Error Console and check if there are any reports in red? If yes, can you please paste the details here?

Additional info for getting more info is here:
https://wiki.mozilla.org/Thunderbird:OpenPGP#Debugging_.2F_Tracing

When I open a message there is no error in red, but this message does not seem correct:

rnp_op_verify_execute returned unexpected: 268435457

At this location:
RNP.jsm:740:17

Thanks that code means RNP_ERROR_BAD_FORMAT.

Can you please quit Thunderbird, then open a terminal window, and run this command:

export RNP_LOG_CONSOLE=1

Then please start thunderbird again from the same terminal. When you open the message in thunderbird, you may get diagnostic messages from the RNP library. If yes, please paste them here. Maybe that will us what RNP doesn't like about the message.

You probably have many encrypted messages. Does this happen with all the messages that were encrypted for you? Or only with some?

This is the output:

[armor_parse_headers() /home/build/YPKG/root/thunderbird/build/thunderbird-78.0/comm/third_party/rnp/src/librepgp/stream-armor.cpp:587] failed to peek line
[init_armored_src() /home/build/YPKG/root/thunderbird/build/thunderbird-78.0/comm/third_party/rnp/src/librepgp/stream-armor.cpp:663] failed to parse headers
[armor_parse_headers() /home/build/YPKG/root/thunderbird/build/thunderbird-78.0/comm/third_party/rnp/src/librepgp/stream-armor.cpp:587] failed to peek line
[init_armored_src() /home/build/YPKG/root/thunderbird/build/thunderbird-78.0/comm/third_party/rnp/src/librepgp/stream-armor.cpp:663] failed to parse headers
[armor_parse_headers() /home/build/YPKG/root/thunderbird/build/thunderbird-78.0/comm/third_party/rnp/src/librepgp/stream-armor.cpp:587] failed to peek line
[init_armored_src() /home/build/YPKG/root/thunderbird/build/thunderbird-78.0/comm/third_party/rnp/src/librepgp/stream-armor.cpp:663] failed to parse headers

This happens with every message.
Every message in my inbox is encrypted because I ordered my mail provider to automatically encrypt every unencrypted message.
This is why I cannot read any incoming message currently.

Sorry to hear that.
Would you be willing to share one of the messages? (I won't be able to decrypt it, that's fine.)
If yes, you could use File Save As in Thunderbird.
If you're concerned about headers, feel free to remove any email addresses and from/to/subject lines, and routing headers from the message.
Please keep the content-type headers and the body of the message.
After you trimmed your saved message file, please zip it, and attach the zipped file to an email to me, kaie at kuix dot de.

No problem.
Mail is out.

Thank you very much. I was able to identify the part that's causing the RNP library to fail. It's a very trivial detail.

Your message contains:

-----BEGIN PGP MESSAGE-----
Version: posteo-encryption-milter v1 (Posteo encryption milter for OpenPGP v1)

Apparently RNP has trouble to process that Version line. I don't know why, I would expect that this comment should be simply ignored, if it cannot be understood - but apparently RNP decides to stop the processing with a failure.

After manually removing the Version line, the rnp command line tools are willing to analyze the message further, and an attempt to decrypt gives me the expected result (secret key not available).

Flags: needinfo?(o.nickolay)
Status: UNCONFIRMED → NEW
Ever confirmed: true

Reported to RNP at https://github.com/rnpgp/rnp/issues/1208

Assignee: nobody → kaie
Status: NEW → ASSIGNED

I have a hotfix attached. It fixes this bug, and it passes the RNP test suite.

Description of the patch:

src_peek_line wants to scan a buffer to find a complete line, terminated by line end character.

Because peeking into the line can cause read-ahead processing in a subsequent function, the function wants to minimize the size peeking.
Therefore it wants to peek in increments of 64 bytes.

I have described at https://github.com/rnpgp/rnp/issues/1208 how the function is failing, and explained why I don't want to change function src_peek. (If that approach is considered better, I want to leave that to the RNP developers, but they might need more time to develop the "best" fix.)

Instead, this hotfix uses the existing API of src_peek, and therefore requires a different implementation strategy.

The idea is, each time we have read 64 additional bytes without finding end-of-line, it requests a larger buffer, again increasing by 64 bytes - or whatever smaller number of bytes is remaining.

Christoph, if you are able to rebuild your thunderbird package, you could apply this patch locally, and test if it helps you.
Or you could ask your distribution to include this experimental patch.

Another alternative is that I create an experimental Linux build, that you could run, instead of the binary provided by your distribution.

Nickolay, what do you think about the suggested hotfix patch?

Thanks for the quick help.

I opened a request to include the hotfix in the package for my distribution here:
https://dev.getsol.us/T9202

Kai, this fix looks good and all of our tests are passing. But, still, it will not handle the case with 1024+ chars line, breaking the things. As RFC says 'any line length may be used', I'll work on solution which would handle this case as well.

Flags: needinfo?(o.nickolay)

Kai, thanks, but I was able to build and install a Solus package of thunderbird with the hotfix applied.

And it works! Finally I can read my encrypted messages again.
Thanks for your quick and professional help.

Pushed by kaie@kuix.de:
https://hg.mozilla.org/comm-central/rev/f94d1b1bef0a
Hotfix for RNP failure to process long Armor Header lines. Limited temporary fix. r=Nickolay

Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED

Christoph, thanks a lot for the quick testing and feedback, I'm glad it's working for you!

Comment on attachment 9164486 [details]
Bug 1653651 - Hotfix for RNP failure to process long Armor Header lines. r=mkmelin,benc

[Approval Request Comment]
Regression caused by (bug #): OpenPGP code
User impact if declined: cannot decrypt messages
Testing completed (on c-c, etc.): yes manually
Risk to taking this patch (and alternatives if risky): Minimal, strictly limited to experimental OpenPGP code

Attachment #9164486 - Flags: approval-comm-esr78?
Attachment #9164486 - Flags: approval-comm-beta?

Comment on attachment 9164486 [details]
Bug 1653651 - Hotfix for RNP failure to process long Armor Header lines. r=mkmelin,benc

Approved for beta & esr78

[Triage Comment]

Attachment #9164486 - Flags: approval-comm-esr78?
Attachment #9164486 - Flags: approval-comm-esr78+
Attachment #9164486 - Flags: approval-comm-beta?
Attachment #9164486 - Flags: approval-comm-beta+

Upstream also included a fix for this bug, which we should pick up with bug 1654768.
https://github.com/rnpgp/rnp/commit/37463f90e26a12fe494df51fcff45a6cb0812364

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: