1629292 - encrypted/signed email processing fails with mail.server.default.mime_parts_on_demand=true

Status: RESOLVED FIXED

(MailNews Core :: Security: OpenPGP, defect)

Description

[Kai Engert [:KaiE:]]

Background: Thunderbird has a pref mail.server.default.mime_parts_on_demand which is set by default to true. This means, if a message is bigger than a threshold, and and the message is on an IMAP server, and the message consists of multiple MIME parts, the additional MIME parts may not be downloaded immediately, but only on demand.

While "on demand" sounds great, we don't have an implementation that downloads on demand when a encrypted or signed message is detected.

Bug: If the pref is false, and only a subset is downloaded, and only that subset is given to the PGP message processing, processing will fail (because the full contents are required to process it correctly). No message contents are shown, and signature/encryption icons inform that something is broken.

Which of the following strategies should we use to fix this bug?

(1) Set default to false

This is what the Enigmail Add-on did. It just changed the default value to false, which means, all messages all always downloaded completely.

This fixes the problem, but obviously is a regression for users in low-bandwidth environments.

(See also Patrick's comments in bug 1593733.)

(2) Detect, inform and require the user to trigger a full download

We could try to add new code, that detects if a message couldn't be processed correctly, because only a subset was available. (I don't know yet how to detect that reliable, distinguishing it from other failures.)

If that happened, show some feedback in the UI. Offer the user to click a button to trigger a full download/reload of the whole message.

(3) Make the "on demand" smarter

Could we enhance the existing message download code to peek into the content type, and if a encryption/signing content type is detected on the outer layer, we trigger a full download automatically?

(4) full download if account has encryption configured

The IMAP protocol code could query the account/identity configuration. If any identity for an account has either S/MIME or OpenPGP data configured, then we switch this server to always use full downloading automatically.

---

1 is easiest to do, but with regressions.
4 is probably still easy to do, and only gives performance regression to those actively using encryption, but for all of their messages.
3 would be the smartest approach, but I don't know how difficult it is to implement.

S/MIME does 2, at least for signed messages. The code to detect the incomplete download is done in C++, so maybe that code has more information available than we have at the JS layer?

Lots of questions. What do you think?

Comment 1

[Kai Engert [:KaiE:]]

Magnus, we need to make a decision here, and fix it soon. It's a major issue that makes OpenPGP unusable.

Which approach should we use?

Comment 2

[Magnus Melin [:mkmelin]]

In bug 1593733 comment 1 Patrick says for S/MIME the not-fully-downloaded doesn't happen. I wonder if that's the case and where we do that...

In general I think always doing full downloads of messages would be totally ok, though it might require more wide discussions. It's not only about IMAP but we also have the POP (mis)feature that you can download only headers or up to a certain size. (With really awful UX!).
Messages aren't usually too many MB and just a single load of Facebook will put you at the same bandwidth usage. People don't seem to have much problem with that so I kind of doubt downloading messages would be much of a problem.

Comment 3

[Kai Engert [:KaiE:]]

(In reply to Magnus Melin [:mkmelin] from comment #2)

In bug 1593733 comment 1 Patrick says for S/MIME the not-fully-downloaded doesn't happen. I wonder if that's the case and where we do that...

I don't understand why Patrick made that observation.

I remember from the time when we had implemented S/MIME, that this was a common scenario.

We had specifically introduced the "signature with question mark icon" to signal that we're uncertain, and when clicking it, we inform the user that the message hasn't been downloaded yet, and we offer the user to manually trigger a full download of the message, to fix the status display.

I don't remember what we did for encrypted message.

In general I think always doing full downloads of messages would be totally ok, though it might require more wide discussions. It's not only about IMAP but we also have the POP (mis)feature that you can download only headers or up to a certain size. (With really awful UX!).

We could argue, if someone needs bandwidth preserving behavior, that's rather difficult to achieve with IMAP.

Consequently, we could recommend that everyone requiring strict bandwidth control should use POP servers only, with the explicit configuration to limit amount of data downloaded by default.

Based on that, here's my suggestion:

• let's change the on-demand pref to false on nightly for Beta 77
• start the discussion you're suggesting in parallel
• wait for feedback during the Beta 77 and Beta 78 periods, if necessary, we can adjust the plan later

Comment 4

[Kai Engert [:KaiE:]]

Attachment: Bug 1629292 - Change mail.server.default.mime_parts_on_demand to false. r=mkmelin

Comment 5

[Magnus Melin [:mkmelin]]

We should probably also change mail.imap.mime_parts_on_demand_threshold to something much higher than 30KB. This pref controls what's too little to use mail.imap.mime_parts_on_demand_threshold at all and just download it all. Maybe 500KB?

Comment 6

[Kai Engert [:KaiE:]]

Ok, patch updated.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/5c4a3edcdb08
Change mail.server.default.mime_parts_on_demand to false. r=mkmelin