Open Bug 1627956 Opened 5 years ago Updated 1 year ago

Guided "Trust on First Use" (TOFU) for OpenPGP keys in composer

Categories

(MailNews Core :: Security: OpenPGP, enhancement, P1)

Product:
MailNews Core
Component:
Security: OpenPGP
Type:
enhancement

Tracking

(Not tracked)

Status:
REOPENED

People

(Reporter: KaiE, Assigned: KaiE)

References

(Blocks 1 open bug)

Details

Attachments

(3 files, 19 obsolete files)

Bug 1627956 - Add API to retrieve overall OpenPGP key availability status for a given email address. r=mkmelin
48 bytes, text/x-phabricator-request
Details | Review
already-landed-with-api.patch
1.32 KB, patch
Details | Diff | Splinter Review
fix-unaccepted-filter.patch
1.55 KB, patch
Details | Diff | Splinter Review

The initial public key trust management from bug 1626683 requires an explicit opt in, before a key is used for sending encrypted email.

Magnus is worried this might be too complicated for most people, and suggests that we should automatically accept it. Could we improve that?
[edit: changed "automatically trust" to "automatically accept"]

If Alice sees a public key for Bob's email address for the very first time (Alice has never before seen any other public key for Bob's address), then we could accept it automatically as "unverified", without explicit opt-in.

If we did (and assuming our default configuration allows to encrypt using unverified keys), then Alice could immediately send encrypted email to Bob, without confirming/accepting Bob's key.

This has another consequence, though.

If receiving a signed email from Bob is the first event in which we learn about Bob's (allegedly owned) key, we'd also have to treat it as "unverified".

The received signed message would then be shown as "good signature from unverified identity". (In contrast, with the initial work from bug 1626683, we'd treat the key as "undecided" and the signed message would be shown with "questionable signature".)

Do we think that's reasonable?

I'd personally prefer to treat it as "undecided". The user could click the signature, sees the undecided status, and be required to deliberately change it to "yes, accepted (unverified)", to use the key.

I suggest that we postpone the decision, and try to collect more arguments for one or the other default behavior.

(In reply to Kai Engert (:KaiE:) from comment #0)

If Alice sees a public key for Bob's email address for the very first time (Alice has never before seen any other public key for Bob's address), then we could accept it automatically as "unverified", without explicit opt-in.

If we did (and assuming our default configuration allows to encrypt using unverified keys), then Alice could immediately send encrypted email to Bob, without confirming/accepting Bob's key.

This would be close to the idea of Autocrypt.

This has another consequence, though.

If receiving a signed email from Bob is the first event in which we learn about Bob's (allegedly owned) key, we'd also have to treat it as "unverified".

The received signed message would then be shown as "good signature from unverified identity". (In contrast, with the initial work from bug 1626683, we'd treat the key as "undecided" and the signed message would be shown with "questionable signature".)

Do we think that's reasonable?

I'd personally prefer to treat it as "undecided". The user could click the signature, sees the undecided status, and be required to deliberately change it to "yes, accepted (unverified)", to use the key.

I suggest that we postpone the decision, and try to collect more arguments for one or the other default behavior.

The arguments depend on the use-case you focus on. I think you should define what you want to achieve. If that's clear then you also know which way to go forward. The main question to ask is which of the following scenarios you intend to achieve:

  • If your goal is to make it easy for everyone to use encryption, then this is clearly the right direction.
  • If your goal is high security for those who use encryption, then you should not do this - or have at least an option to turn off automatic acceptance.

In other words, you have to find your answer for the balance between easy-to-use and highest security.

I very agree to the decision finding approach with the achievement definition and the two use-cases/scenarios Patrick mentioned.
Maybe there could be names established for these two mentioned use-cases, and these names get profiles in TB, which the user can select and set some key trust management behavior accordingly (I even added a third profile here):

  1. Profile (extremely easy for every one, Autocrypt type approach): Profile name "I have nothing to hide" :-)
    This profile would be without public key trust management, just for untrusted end2end encryption for widespread use, very user friendly, radically accepting/updating every foreign key without question(maybe only a hint "Foreign key was updated. [OK]" , explicitly no protection against active attacks (the user means he would not have to hide the cat pictures sent to his grandma anyway), only privacy protection enhancement to protect against low cost passive mass eavesdropping to make the digital world better for the non-technical people. If with this approach the count of daily emails not e2e encrypted would decrease from the 99,99% been reality for decades now to e.g. 75%, this would be a great success. Mails are locally stored PGP-unencrypted, so no risk of data loss. Public and private keys come and go, so no hazzleing with them, only used for transport encryption. Possibly the only messages would be e.g.
    "Could not decrypt received encrypted message due to missing appropriate decryption key. Please answer the sender via mail, so the sender gets Your current key and can re-encrypt"
    and
    "Cannot encrypt outgoing mail due to missing appropriate receivers encryption key. Shall the mail be sent unencrypted? If not, please send key request mail to recipient first [here] (Button) to receive the recepients encryption key within the answer mail and compose mail again."

  2. Profile (more complicated to use): "Key protection"
    This profile would be with public key trust management for authenticated end2end encryption, meaning the foreign keys have to get assigned a trust level, they are not accepted when expired, warning/question when foreign key changed, warning when signature is invalid (or no signature), key length is too short, key cipher is too weak etc. This profile would claim to protect against active key attacks on the wire (e.g. MITM) to break the encryption on the wire.

  3. Profile: "High security" (slightly ironically here)
    This profile claims to protect against active attacks circumventing the encryption, like installing a local key logger on the user device or stealing the private key etc.
    When the TB user selects this profle, only a warning is displayed: "Please create Your private key, compose and encrypt Your text on a plain installed seperate offline device (with a secure operating system like Tails and the recipients public key verified in person), and use internet connected TB only for transporting the offline encrypted attachment file."

Maybe users could switch between profiles 1. and 2., and if profile 2. is too complicated for a user, he could easily switch back to profile 1 and all the bells and whistles would disappear?

Edit: In profile 1 for every recipient, for which it's public key is missing, the mail is silently sent unencrypted (but with the senders public key attached/in the Autocrypt header like with every mail). So the only new error message compared to status quo all unencrypted would be the stated "could not decrypt" message.

Additionally, profile 0 would be "Never encrypt or sign" which would disable signing, sending the senders(users) public key and the encryption of mails even if the recipients (guessed) public key is present from possible usage in the past. But maybe, if a received mail is encrypted and can be decrypted because the users(recipients) private key is still present from past usage, mail would be decrypted. And if the received mail would be signed and the signature could be validated with a senders public key present from past usage, there could be a green "Signature OK" mark? As You see and know, this is really complicated, and maybe the concept of having different profiles with different security/usability ratio available, could be an approach, so the user can select and try the profiles, switch forth and back, until the appropriate is found as sufficient.

Let's revisit these ideas at a later time, I think it's too late to consider them for TB 78.

Changing the summary a bit and hijacking this bug to handle the guided TOFU setup flow.

I think the happy path could actually not be too complicated. I'd suggest to have a guided dialog to show what's going on and there allow TOFU + accept all. If someone needs special control, like individual acceptance, they should do that in the key manager. This way we could make the sending process fairly simple, and yet informative. I would also leave out verification from this flow: that will only confuse the user, and there's no need to do that straight away - well there could, but if there is the user needs to be aware of this and take appropriate measures.

After some iterations, this is what I came up with. The dialog would come up if you chose to encrypt, also in the future if we have encrypt-if-possible and the keys were available but not yet configured.

+----- Setting Up End-to-end Encryption-- -------------------------------------------+
|
|  The public key of each recipient is required for end-to-end encryption. 
|  For 3 of the recipients keys have not yet been used.
|
|    Thunderbird is discovering possible keys ...
|    Found the key 0xABCDEF1234567890 for "Homer <homer@example.com>" from an email.
|    Found the key 0xABCDEF1234567891 for "Marge <marge@example.com>" from WKD.
|    The key 0xABCDEF1234567892 for "Moe <moe@example.com>" was already imported,
|    but not accepted.
|
|  Found possible keys for everyone. Authenticity of the keys can't be established.
|
|  By continuing you're accepting the to use these keys for this and future
|  encrypted messages to the specified recipients.
|
|                                                            [ Cancel ] [ Continue ]
|
+------------------------------------------------------------------------------------+

The middle of this dialog is supposed to show a log (with scrollbars) of what happened. We would link the key ids, as well as the email and wkd source, for debugging/trust purposes.
Summary: Should we automatically accept an OpenPGP public, if it's the first ever seen for an email address? → Guided TOFU setup for OpenPGP
Priority: -- → P1

In reply to Patrick and Arvidt:

With the OpenPGP implementation in TB 78 and 91, we had already taken the path of "don't do things automatically", but rather, try to assist the user, and require the user's participation.

I'm not in favor of having different "profiles" as Arvidt suggested, parallel implementation of UI flow seems to be a maintenance burden and can be confusing. I think it's ok to have individual prefs, but the set of possible combinations of prefs is probably higher than 3, so I think a profile wouldn't sufficiently serve all user requirements.

I had originally asked in this bug, should we automatically accept keys on the very first time, for a particular email? I tend to say "no", I prefer to at least tell the user that we COULD accept a key, and still require confirmation.

Looking at Magnus' proposal, he also seems to have this opinion. He wants to assist (automatic discovery), and inform (here is what we discovered), but he wants to require the user to confirm (even if it's just an overview with limited details).

Summary: Guided TOFU setup for OpenPGP → Guided "Trust on First Use" (TOFU) for OpenPGP keys in composer

Magnus' suggestion isn't bad. But I think it would be very rare to hit the "happy path".

I think that in most situations, you'd encounter at least one problem. A flowing text box might be difficult to grasp.

I'd like to propose a slightly different approach - a "wizard-like" approach.

Today, if we cannot send an encrypted email, we show a brief error message, and then we jump to the recipients dialog.

Instead, I suggest we use a series of wizard steps to assist the user to accept the keys, one step at a time.

In the first step, we could show the error message as today. Something like "cannot send encrypted, problems with keys for N recipients. Click NEXT to get assistance.

The dialog would change to show the first problematic email address.

It would summarize what we have currently, e.g.

  • (1) no key available yet

  • (2) a key that you previously accepted has expired

  • (3) one new key was found, you may accept and use it
    showing Identity from key, showing fingerprint
    [ accept (unverified) ] [view details]

  • (4) multiple new keys were found
    [view list of keys]

  • (5) all keys are either revoked, rejected or expired

[discover more keys online]

The dialog would start by showing either (1), or (5), or (2), or "(2) combined with (3)" or "(2) combined with (4)".

Initially, we'd offer the button "[discover more keys online]". If clicked, we can show some progress on the dialog (and disable all keys except CANCEL). Once done searching, we'll update the summary (and remove the button to discover keys).

So for example, in the "happy path", we'd initially say (1). The user clicks "discover". We only find one key. We show (3). User clicks [Accept unverified].

Once the user has resolve the situation (either by clicking (3) "accept" directly, or by using (3) "details" to verify and accept, or by using (4) to view the list of keys, and marking one of them as accepted, and the user returns back to this primary view, the display could change to
"ready to encrypt"

The dialog would also show a button [NEXT RECIPIENT] at the lower right corner. For the last recipient, the button could say something like "done" or "complete".

With this approach, we have the same flow for any kind of problems.

In the happy path, the user would get one simple dialog for each recipient, click discover, click accept, click next recipient, until done.

If we had shown (2), the user clicks discover, and we find a refreshed (extended) copy of the already accepted key, then we can immediately switch to "ready to encrypt" for that user.

My suggestion is to require an explicit click to "discover". The idea is to avoid the leaking for queries to keyservers, for users who don't want that to happen (don't want anyone on the network to see that you query the keyserver for a particular email address).

Initial WIP patch with some partial UI for the key assistant.
Tomorrow I'll finish all the other views and fix some sizing issues, but I just wanted to upload it so you can check the direction and maybe can visually help Kai to see how we can start getting the data from the back-end.

Assignee: nobody → nf

Here is my updated Frankenstein. A few comments:

  • The thing I couldn't get to work is to unfold the encryption options when clicking on the body (and not the arrow) of the encryption technology split button. I marked this with a XXX line 627.
  • The notification logic in checkRecipientKeys is probably more complicated than it should. I left it like this because it's a stripped down version of the more elaborated notifications that I had in October. The notification blinks when updating and I might try to fix this if time allows.
  • I'm not handling yet the case when changing from an identity with encryption by default to another identity with no encryption configured. We shouldn't disable encryption silently, as discussed today. I left it like this for now because I'm not sure that it matters for the usability tests.
  • I have a known bug that I don't think matters for now: 1. open on an identity with only S/MIME configured but disabled by default, 2. click "Encrypt", 3. switch to an identity with both S/MIME and OpenPGP, 4. switch from S/MIME to OpenPGP. The subject should be encrypted but it's not.

Let's see tomorrow how we can combine both!

I applied Nico's patch first, and rebase Alex' patch on top. The only place that need merging was this line, which I changed to include the words from both of you.

defaultset="button-send,separator,button-address,spellingButton,button-security,button-save,buttonKeyAssistant,button-encryption-toggle,button-encryption-options,ring,button-attach"
Attachment #9260016 - Attachment is obsolete: true

(In reply to Kai Engert (:KaiE:) from comment #7)

I had originally asked in this bug, should we automatically accept keys on the very first time, for a particular email? I tend to say "no", I prefer to at least tell the user that we COULD accept a key, and still require confirmation.

I am asking me how TB users that have absolutely no idea what a "key" is at all, react when they are asked if they would accept such a "thing". Either they click accept, or they are feared and click "not accept", in either case they do not know what they do. When they rejected the key and keep getting mails from the partner again and again with the question if they would accept the "key", maybe at some point they would say "Well, do not bother me with this stuff anymore, ok I accept".
Negative: User is (possibly infinite repeatingly) feared or plagued about things that the user does not know about. The user is forced to do something.
Postive: User gets aware about "keys" and encryption, and can learn about it. When user accepts only after repeatingly getting key offered, chances are good, that the key is authentic when possibly finally accepted.

Attachment #9259880 - Attachment description: WIP: Bug 1627956 - WIP UI for OpenPGP Key Assistant. → WIP: Bug 1627956 - UI for OpenPGP Key Assistant.

I've updated the "API" patch. Please see the updated comments in my patch. It now depends on Magnus' patch from bug 1667564.

You can interactively test and inspect the data that is returned by the API patch.
Compose an email, select "manage keys for recipients", select the email address you would like to inspect, click "manage keys for selected recipients".

This triggers a dump to the error console, it will contain the data returned by API getKeyStatusForEmailRecipient() for that email address.

I recommend to include the patches from bug 1751883 and bug 1751885 when testing.

We also need bug 1751869 and bug 1634524.

Depends on: 1751869, 1634524

Comment on attachment 9261095 [details] [diff] [review]
allow-classic-resolve-for-debugging.patch

Integrated in the UI wip patch.

Attachment #9261095 - Attachment is obsolete: true
Attachment #9259880 - Attachment is obsolete: true
Attachment #9260188 - Attachment is obsolete: true
Depends on: 1752428

Comment on attachment 9261140 [details]
WIP: Bug 1627956 - Silent OpenPGP key discovery and collection.

Revision D137211 was moved to bug 1752428. Setting attachment 9261140 [details] to obsolete.

Attachment #9261140 - Attachment is obsolete: true
Depends on: 1752436

Here is a bug report for the current state:

  • enter email with a key that is expired, and which has an updated validity on the keyserver
    (in my example set, this is bob)
  • click resolve in notification
  • click resolve next to email
  • you'll see section "problematic keys"
  • (I don't get any key details in this scenario)
  • click "discover"
  • after it's done, there's no longer any key section shown
  • click back
  • bob is still shown with "issues"
  • click close
  • click the resolve notification again
  • now bob is shown as good.

In other words, if discovery in the one-email resolve screen fixes the issue, the keys in the main view aren't updated at all - and updating inside the resolve screen seems to have a bug.

Attached patch ontop-assistant.patch (obsolete) — — Splinter Review

Alex, I suggest to apply this patch to phab D137205.

It hooks up additional backend functionality, fixes use of the key status API, and disables the additional prompt on failing to send an email caused by problematic encryption keys.

Here is a summary of the changes since Alex had attached the latest patch, what needs to be done to to pull in the various fixes I've added:

(1)
We should make a change to Magnus' key collection patch, from bug 1667564, to phab revision D135074.
See this patch in bugzilla, I've asked Magnus to apply it:
https://bugzilla.mozilla.org/attachment.cgi?id=9261424&action=diff
When making experimental builds for Nico, if the patch hasn't yet been applied to phab, it should be manually included.

(2)
Fixes to my work in bug 1752428.
I have already updated phab revision to include them.
If necessary, you can see the changes here:
https://phabricator.services.mozilla.com/D137211?vs=533151&id=533868#toc

(3)
The changes to Alex' code which I mentioned in the previous comment 28 in this bug.

(4)
A fix for new bug 1752718 that Nico reported, we could backport this to TB 91, so I've filed it as a separate bug and patch.
Please include phab D137386 when preparing updated builds for Nico, at any point in the patch stack.

Comment on attachment 9261425 [details] [diff] [review] ontop-assistant.patch >diff --git a/mail/components/compose/content/messengercompose.xhtml b/mail/components/compose/content/messengercompose.xhtml >+ if (!("collected") in key) { oh, this should be if (!("collected" in key)) {
Comment on attachment 9261425 [details] [diff] [review] ontop-assistant.patch Review of attachment 9261425 [details] [diff] [review]: ----------------------------------------------------------------- ::: mail/extensions/openpgp/content/ui/enigmailMsgComposeOverlay.js @@ +1897,1 @@ > } There's a bug here due to an accidentally delete bracket. I fixed it when merging it on top of my patch.

Comment on attachment 9261425 [details] [diff] [review]
ontop-assistant.patch

Marking obsolete as this was merged on top of D137205

Attachment #9261425 - Attachment is obsolete: true
Attached patch fixes.patch (obsolete) — — Splinter Review

Comment on attachment 9261768 [details] [diff] [review]
fixes.patch

marking obsolete, needs a better patch

Attachment #9261768 - Attachment is obsolete: true
Attached patch fixes2.patch (obsolete) — — Splinter Review

Comment on attachment 9261773 [details] [diff] [review]
fixes2.patch

Marking obsolete as it was merged on top of D137205

Attachment #9261773 - Attachment is obsolete: true
See Also: → 1755281

The key assistant implementation MUST consider the scenario explain in bug 1627956 comment 1. (The offered key already has a positive or negative acceptance decision for a different email address, but not for the email address the user is currently trying to resolve.)

Here are my notes and recommendations about the results of the
usability tests that directly affect our work on this patch.

Bugs:

  • Missing "Subject Encryption" in OpenPGP menu

    I had it in my code and tested it but, for some reason, it was not
    in the prototype that I tested with users.

  • Security menu is available but all options are unavailable until
    OpenPGP is configured

    I recommend making the display of the Security menu consistent with
    the display of the Encrypt toggle if OpenPGP is not configured for
    the current identity or not configured at all.

  • P7 got an error message related to S/MIME when trying to send an
    email but OpenPGP was not configured yet (even though they already
    have a personal key)

  • Subject encryption icon is black on black in dark theme

Higher priority usability improvements:

  • Improve the layout and the signal/noise ratio of the recipients list

    During the usability tests, 4 participants out of 7, closed away the
    Key Assistant when they could have used it to solve a key
    issue. Several of them were confused by an empty list of recipients
    when there were no issues. Other participants got confused by the
    change of structure of the Key Assistant when recipients moved from
    1 section to another and disappeared in the section without issue
    after resolution or the list of recipients appears empty.

    It seemed to me than the Key Assistant was sometimes too unclear and
    noisy at first sight. People might get used to it, though.

    We could improve on this by removing every bit of information that's
    not critical on this screen and insisting more on the important
    parts.

    I also think that we optimized the layout of the Key Assistant too
    much for long list of recipients, when most encrypted emails (I
    guess) would only have 1 or few recipients.

    Recommendations:

    • Add a "Resolve" button also for recipients that have a missing
      key. Have a variant of the resolution screen for missing keys.

    • Implement the "Resolved" label when coming back from the
      resolution screen (designed but not implemented).

    • Assuming 1 or few recipients in the vast majority of cases, we
      could list all recipients by default. Dividing the screen in 2
      sections might not be useful for a few recipients and a scrollbar
      could be enough in the rare cases where the recipients don't fit
      on 1 screen. We could replace the 2 sections, their headings, and
      introductory sentence with more visual labeling of each recipient.

    • Move the tip on key aliases “Thunderbird normally...” at least to
      the resolution screen and ideally replace it there with some
      advanced options UI.

    • The warning about rogue keys was not useful to prevent using an
      actual rogue key or to encourage better verification.

      5 of the 7 tests participants used deprecated or rogue keys
      anyway. The 2 test participants who didn't had other strategies in
      place to prevent that outside of the Key Assistant.

      I think that it's a bit of noise that could be removed from this
      screen and rather moved to the resolution screen (where people
      actually accept keys) and the import process (from file or
      servers) outside of the Key Assistant.

    See this mockup:

    https://raw.githubusercontent.com/nfalquet/thunderbird-openpgp/main/mockups/recipients-list.png

  • People are extremely confused about Thunderbird encrypting to 1 key
    randomly when they have several keys for a given email address.

    Test participants thought that Thunderbird would either encrypt to
    both or make a smart choice magically for them. Thunderbird does
    neither of this and don't tell the user what it actually does.

    Thunderbird should help people avoid having several accepted keys
    for a given email address as, most of the time, only 1 key will be
    the correct one (a new key replacing an old key), though they are
    legitimate corner cases to have 2 keys for 1 email address.

    Recommendations:

    • When importing a key, Thunderbird should warn if there is already
      an accepted key for the same email address. That's outside of the
      scope for this patch.

    • In the Key Assistant, the "View Key..." button should be a "View
      Keys..." button instead and open the current interface to view
      keys for 1 recipient (designed but not implemented).

    • Thunderbird should encrypt to all accepted keys for an email
      address instead of randomly and silently encrypting for 1 but not
      the other keys.

  • Remove the radio button when there's a single key in the resolution screen

    3 test participants had a hard time understanding the single radio
    button in the resolution screen. The widget was particularly
    problematic on macOS where it looked like a stop sign, and not so
    much on Windows.

    Recommendations:

    • Remove the radio button where there's only 1 key available.

    • Rename the button as "Accept Key".

    • Make the "Accept Key" available by default when there's only 1 key
      available.

  • "Encrypt" and "OpenPGP" buttons until OpenPGP is configured

    For a first release and until you have time to design a good
    onboarding experience, I recommend hiding the "Encrypt" and
    "OpenPGP" buttons (when no identity has OpenPGP configured) to
    prevent having many people wonder about it.

    Optionally, you could show the "Encrypt" and "OpenPGP" but make them
    unavailable as soon as the user has at least 1 public or private
    key, even if OpenPGP is not configured for the current account or
    for any account. It might have helped P6 and P7 understand that they
    still had something else to configure.

    If you do so, make sure to fix the display of the Encrypt
    toggle. With the current code, P7 had a toggle with a plain lock but
    unavailable even when the message couldn't be encrypted.

Lower priority usability improvements:

  • Feedback of online discovery from Key Assistant is too quick and unclear

    What I originally designed is to replace the "Discover" button by
    progress indication on the same screen. The result of the search
    would stay on the same screen. It would also replace the need for
    the green notification on success.

  • Green check mark on Encrypt toggle

    During the usability tests, P8, the most novice user, was slightly
    unsure whether the email was actually going to be encrypted or
    not. They mentioned missing the "green sign from Enigmail". Adding a
    green check to the padlock when encryption is Enable would add an
    extra visual indication (also accessible to colorblind people) and
    be more consistent with the icon in the Reader.

    The padlock could be plain once you use it to onboard people to
    configure OpenPGP.

  • Rename “Require Encryption” as “Encrypt”

    During the usability tests, P8, the most novice user, wondered what
    the difference was between "Encrypt" and "Require Encryption".

    See this video clip:

    https://un.poivron.org/~nf/thunderbird-openpgp/clips/require-encryption.webm

    I recommend renaming this menu entry to "Encrypt". It's shorter,
    simpler, and more consistent.

  • Tooltip on problematic recipients is not helpful (and actually misleading)

    Right now, we don't update the tooltip on the pill of problematic
    recipients. This means that the tooltip might be "xxx is not in your
    address book", which users might relate to our warning.

    During the usability tests, P1 (and also P0 from the pilot test)
    didn't notice the key notification at the bottom of the window (it's
    far away and looks like many other notifications), but tried to get
    more info from the recipient pill.

    Recommendations:

    • As a minimum, we should remove the misleading tooltip about the
      address book.

    • We could also warn whether it's about a missing key or another
      issue:

  • Keep the menu entry "Attach My Public Key" in the OpenPGP menu

    During the usability tests, several people had no problem finding it
    in the Attach menu. P3 even commented that "it seems logical". But
    the 3 most novice users had a harder time finding it there.

    It might be worth having it in the 2 places, even temporarily. If
    you only want it in 1 place, the OpenPGP menu might be superior.

  • “Send Public Key(s) By Email” from Key Manager should attach keys to
    the Composer from which the Key Manager was open from, instead of
    opening a new Composer like it's the case right now. 4 test
    participants run into this.

  • Recipient pills are not updated when coming back from Key Manager.

    It might be easier to at least trigger refresh the recipient pills
    when coming back from a Key Manager opened from the Composer.

  • Resolution screen not updated after accepting or verifying a key
    from the Key Properties

Not from the usability tests:

  • Prevent key notification from blinking

    Right now the key notification blinks when the list of recipients is
    edited because it's removed, recalculated, and added again.

  • Display OpenPGP menu when the body of the split button is clicked.

    Right now the menu is only displayed when the arrow of the split
    button is clicked. It worked fine during the usability tests,
    though.

See Also: → 1669788

Reference of what we designed before implementing the software prototype and doing the usability tests:

https://un.poivron.org/~nf/thunderbird-openpgp/mockups/key-assistant-for-prototype.png

https://un.poivron.org/~nf/thunderbird-openpgp/mockups/key-assistant-for-prototype.png

Spreadsheet listing the usability issues identified during the tests in February:

https://un.poivron.org/~nf/thunderbird-openpgp/results.fods

See Also: → 1677709
See Also: → 1680810
See Also: → 1681938
See Also: → 1683332
See Also: → 1710453
See Also: → 1723694
See Also: → 1744835
See Also: → 1630395
See Also: → 1630405
See Also: → 1630416
See Also: → 1595226
See Also: → 1616525
See Also: → 1630433
See Also: → 1732275
See Also: → 1752777
See Also: → 1752436
See Also: → 1651045
See Also: → 1667254
Assignee: nf → nobody

I don't like the idea of a key assistant, when users want to compose an email. I tested version 97.0a1 of Thunderbird and there I had to open several windows until I could receive a public key for an email address. But, there are two problems:

  1. The Usability suffers. When a product has a good usability users don't have to think much about how they accomplish a task. But when they have to go through these steps they are distracted from their task.
  2. Users have to read more text, but many people don't do that when they browse through the internet or use a software. When they don't read the texts it could lead to errors or unsafe behaviour.

That's why I strongly suggest to retrieve keys automatically in the compose window. Additionally, it would be good to display how trustworthy a key is because this would help advanced users to use only keys for encryption which are good enough. Maybe this way novice users could be motivated to concern themselves with encryption.

To improve the security Thunderbird could choose and retrieve the keys like this:

  • When there is a key which was accepted (after validating the fingerprint) take this key. If there isn't such a key continue.
  • Use WKD to retrieve a key and use this key. If this action isn't successful continue.
  • Contact keyservers to retrieve a key and use this key.

When Thunderbird has multiple public keys for one email address it would also make sense to automatically prefer an accepted key and then a key retrieved via WKD. WKD has the advantage that keys come from an email provider who can confirm that a key belongs to an email address. Usually, email provider also offer a better security than private persons who maintain a keyserver.

Hi Christoph, thanks for your participation but I would like to kindly ask you to not use bugzilla to write your feature requests.
A more appropriate location for this would be the UX mailing list, where you already started a thread, or the e2ee mailing list.

Comments on bugzilla should be limited to technical feedback after testing the full stack of patches available in the bug, as well as the linked dependent bugs, otherwise this thread turns into an open forum and it's hard for developers to stay on track.

To quickly give you an overview, all of the things you mentioned have already been discussed at length in the stack of bugs connected to this one, and will be address, or have already been addressed, in patches and fixes that will be available in the next ESR.

Assignee: nobody → kaie
Attachment #9260218 - Attachment description: WIP: Bug 1627956 - Add API to retrieve overall OpenPGP key availability status for a given email address. → Bug 1627956 - Add API to retrieve overall OpenPGP key availability status for a given email address. r=mkmelin
Status: NEW → ASSIGNED

@aleca: FYI, your revision D137205 no longer works, but I'll help to get it updated.

First, bug 1756665 makes changes that overlap with your patch.
I've just completed merging that, and will attach the result FYI.

Second, we're changing the API that this code queries.
Once we're done with the API patch, I'll update your patch to use the new API, and attach it here, too.
I want to do that myself, I want to use this for testing that the new API actually works as expected.

Then we'll have to get your patch reviewed.

Attached patch 1627956-ui-merged-20220315.patch (obsolete) — — Splinter Review

First step as described in previous comment.

Attachment #9260188 - Attachment is obsolete: false

Second step as described in above comment 43.

Attachment #9261132 - Attachment is obsolete: true
Attachment #9260188 - Attachment is obsolete: true
Attachment #9267955 - Attachment is obsolete: true
Attachment #9268109 - Attachment is obsolete: true

Pushed by kaie@kuix.de:
https://hg.mozilla.org/comm-central/rev/08cb06ba7d28
Add API to retrieve overall OpenPGP key availability status for a given email address. r=mkmelin

Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

aleca: prior to landing the API patch, I had added this small fix.

I had previously accidentally added that to your UI patch, so please remove it from your code, prior to rebasing, to prevent that you get a merge conflict.

In addition, I made this mistake when rewriting your UI code to the new API, please apply this patch on top of your patch.

With the UI patch applied, we fail in mail/test/browser/openpgp/browser_editDraftTemplate.js

See Also: 1630405 →

The UI patch doesn't consider the selected encryption technology.

If S/MIME is configured/selected for the message, then you must not run any of the OpenPGP related checks.

Either the UI must show information about available or missing OpenPGP keys, or, if you want to postpone that, it shouldn't show any warnings at all (previous behavior, only get error message at send time).

(In reply to nf from comment #38)

  • Missing "Subject Encryption" in OpenPGP menu

    I had it in my code and tested it but, for some reason, it was not
    in the prototype that I tested with users.

Alex, please clarify if you want to fix this in this bug, or in a follow-up bug.

Assignee: kaie → alessandro

(In reply to nf from comment #38)

  • People are extremely confused about Thunderbird encrypting to 1 key
    randomly when they have several keys for a given email address.

This isn't a new behavior, it's not influenced by the work in this bug. A separate bug should be filed for this suggestion. There might already be a bug open.

I looked through Nico's long summary from comment 38.
IMHO the following are the most important issues, which we need to fix before releasing this assistant in summer 2022:

  • Recipient pills are not updated when coming back from Key Manager.
  • Resolution screen not updated after accepting or verifying a key from the Key Properties

It's very confusing when making changes elsewhere, and then having outdated problem reports (or wrong success) shown in the composer.
Can we introduce a global broadcast, that is sent whenever we're changing any OpenPGP key in any way? Whenever the composer sees it, it should fully refresh all current displays.

I'll file a follow-up bug to track this.

He also said:

  • Rename “Require Encryption” as “Encrypt”
  • Keep the menu entry "Attach My Public Key" in the OpenPGP menu

I'm ok, and they are likely very easy to do as part of this patch, if Alex agrees.

Everything else from Nico's list seems to require more work.
I'll leave it to Alex which of the additional should be done in this bug, or in follow-up bugs.

Blocks: 1760150

(In reply to Kai Engert (:KaiE:) from comment #56)

  • Recipient pills are not updated when coming back from Key Manager.
  • Resolution screen not updated after accepting or verifying a key from the Key Properties

I specifically remember fixing this things in my WIP patch. That's strange.

Anyway, I'm a bit overwhelmed with other priorities at the moment.
I should be able to jump back into OpenPGP tomorrow and do a proper overall usability walkthrough of all the features to be sure everything works as intended.

Apologies for the temporary blockade.

Attachment #9268203 - Attachment description: Bug 1627956 - OpenPGP composer and key assistant implementation. r=kaie,mkmelin → Bug 1627956 - OpenPGP composer and key assistant implementation. r=mkmelin
Attachment #9268203 - Attachment description: Bug 1627956 - OpenPGP composer and key assistant implementation. r=mkmelin → Bug 1627956 - OpenPGP composer and key assistant implementation. co-author=kaie r=mkmelin
Attachment #9268203 - Attachment description: Bug 1627956 - OpenPGP composer and key assistant implementation. co-author=kaie r=mkmelin → Bug 1627956 - OpenPGP composer and key assistant implementation. co-author=aleca r=mkmelin

The current patches introduce a functional regression.
We need a follow-up patch that needs to be done together with landing the other parts.

Alice has two email addresses. She has an OpenPGP key that lists both email addresses.

Bob receives email from Alice with address 1. Bob replies. Bob uses the key assistant to resolve. Bob accepts the key for Alice address 1.

The code as implemented calls PgpSqliteDb2.updateAcceptance with the single email address 1.
This will cause only address to be marked as accepted.

So far, I think this makes sense. We shouldn't automatically accept the other email addresses, to ensure the user cannot be easily tricked to accidentally accept it for email addresses of unrelated people.

Later, Bob receives an email from Alice address 2. The key is already accepted, just this second email address isn't accepted yet.

The key assistant reports that a collected key is available. Accepting that key will call PgpSqliteDb2.updateAcceptance again, now with only address 2.

This function call isn't incremental. It completely replaces all addresses on store.
As a result, after accepting address 2, the address 1 is no longer accepted.

I'll make a follow-up patch to fix that the acceptance logic.

However, we should also check if the wording needs to be tweaked for this scenario.

FYI, I'm working on code in bug 1755281 that makes it simpler to work with partially accepted keys.
(Previously, all key acceptance went through UI that showed all email addresses, and marked all of those as accepted at the same time.)

I'm pasting several observations from Magnus, to ensure they won't get lost (I think phabricator doesn't get looked at much after a patch is accepted):

Observations (could maybe fix later instead of now):

  • notification about key problems, should have close button as well (to just ignore the problem for now)
  • key discovery closes before you have time to read what's there (should stay open, I think, or at least give a sec to glance what it did before closing)
  • that there is no individual "resolve" button for no-key-available is confusing. It's quite more understandable when the "resolve..." button is shown next to the recipient
  • we should allow individual key discovery in the resolve screen, not just discover all. (And it's odd that they are links on the resolve screen, but buttons on the main screen)
  • When you have keys for all and the message is set to be encrypted, use the menu to open the key assistant. It shows a disable encryption button. Now do the converse, it will not show an "enable encrypt" button

I found another bug that should be considered a blocker for landing this.

I used this code with my primary work profile. The composer window was broken. The error console told me that the button-encryption-options couldn't be found, and that caused the code to throw, and the remaining processing of the init code was skipped.

Thanks for Magnus for pointing me to the cause: In the past I probably had touched/customized my toolbar in some way. After I used customize again to add the two buttons to the toolbar, the code works.

Does that mean we should enforce that these two toolbar buttons are always visible? Or do we need to adjust the code to allow for the buttons to be absent?

(Because I have already difficulties merging, I suggest a new patch on top of the existing patches, as part of this bug.)

The code needs to consider that the user may have removed a button (if someone never wants to encrypt, they may well do that).

Two additional changes are necessary:

We need to load alias definitions earlier, on composer window open, to ensure the recipient key checks can find them.

Regarding keys that were modified in other windows, I think it's easiest to re-check whenever focus is returned to a composer window. The check is cheap if encryption is disabled. And it avoids a lot of complexity we'd have to invent, as there are many places that could affect what keys are available.

Compose email to alice
key assistant shows issue
click resolve
key assistant shows alleged key available
click resolve again
click on "view key"
click on "yes" to accept, then ok
you're back in the view "public keys for alice"

key assistant doesn't notice that the key is already accepted, and continues to offer to accept it.

I think we should change the code in the following way:

Prior to opening the key properties (view key), we should remember the current acceptance state for this key and email.
After the dialog gets closed, we should again look at this specific combination.

If the key has been changed to accepted, it makes no sense to keep that view open, the problem is solved.
I suggest we automatically close the resolve view for that email address.

Attachment #9271458 - Attachment description: Bug 1627956 - Load OpenPGP aliases early in composer. Re-check recpient keys on regaining focus. r=mkmelin → Bug 1627956 - Load OpenPGP aliases early in composer, required for correct key problem notifications. r=mkmelin

(In reply to Kai Engert (:KaiE:) from comment #69)

Created attachment 9271745 [details]
Bug 1627956 - Refresh key assistant views after viewing key details. r=mkmelin

This revision addresses comment 67

We have decided that we want to enable the changes to the composer window (button changes, menu changes, notifications), but we want to take more time for improving the key assistant, and NOT yet enable the new key assistant by default.

We are ready to commit parts of this work, however, this work is still incomplete, as the list of dependent and related bugs shows. This means we shouldn't yet close this bug.

Because we usually want to have a tracking bug per commited change, I will file a new bug for landing the patches that we're ready to land today, and will mark that one fixed (and as a blocker to this one).

I would like to keep this bug open, and convert it into a primary tracking bug for this feature.

Depends on: 1764324

Comment on attachment 9268203 [details]
Bug 1627956 - OpenPGP composer and key assistant implementation. co-author=aleca r=mkmelin

Revision D141329 was moved to bug 1764324. Setting attachment 9268203 [details] to obsolete.

Attachment #9268203 - Attachment is obsolete: true

Comment on attachment 9270611 [details]
Bug 1627956 - Add certificate status feedback for S/MIME encryption in composer. r=mkmelin

Revision D142766 was moved to bug 1764324. Setting attachment 9270611 [details] to obsolete.

Attachment #9270611 - Attachment is obsolete: true

Comment on attachment 9271120 [details]
Bug 1627956 - Don't erase existing accepted email addresses in openpgp key assistant, keep existing level of key acceptance. r=mkmelin

Revision D143075 was moved to bug 1764324. Setting attachment 9271120 [details] to obsolete.

Attachment #9271120 - Attachment is obsolete: true

Comment on attachment 9271247 [details]
Bug 1627956 - Gracefully handle a customized composer toolbar with removed encryption buttons. r=mkmelin

Revision D143146 was moved to bug 1764324. Setting attachment 9271247 [details] to obsolete.

Attachment #9271247 - Attachment is obsolete: true

Comment on attachment 9271744 [details]
Bug 1627956 - Rename button-encryption-toggle to button-encryption. r=mkmelin

Revision D143384 was moved to bug 1764324. Setting attachment 9271744 [details] to obsolete.

Attachment #9271744 - Attachment is obsolete: true

Comment on attachment 9271458 [details]
Bug 1627956 - Load OpenPGP aliases early in composer, required for correct key problem notifications. r=mkmelin

Revision D143270 was moved to bug 1764324. Setting attachment 9271458 [details] to obsolete.

Attachment #9271458 - Attachment is obsolete: true
Depends on: 1765974

I've filed bug 1765974 as a blocker bug for this one.

If we aren't able to fix bug 1765974 in time for the TB 102 release, we must disable the key status notifications in composer window.

(Showing no status is better than showing incorrect/misleading status.)

Depends on: 1765977

Dear all,
first of all thank you so much for finally improving the openpgp-encryption worflow! I had a look at the daily 101.0a1 and simply the yellow color, and the button "do not encrypt" at the bottom is helping a lot for normal users sending unencrypted emails to some recipients, while automatically encrypting to other recipients.

Just one remark, to make the workflow as easy as it was in TB68, and I think this will not threaten the currently followed security model:

  1. add a button which says "send unencrypted" or similar in the OpenPGP Message Security popup. Maybe, there can be another warning afterwards like "the email will be sent UNENCRYPTED", and with a "send UNENCRYPTED" button, but important is, that users can click through a linear workflow, and not have to go back to the composer again.

I made a small screenshot and uploaded it here: https://ibb.co/YD938m2

This would be a replacement for bug https://bugzilla.mozilla.org/show_bug.cgi?id=1710453
It would be great if that can be landed in TB102, because then, after 2 years of still using outdated TB68, we can finally make the switch, as the workflow is acceptable for our common users (we are not a tech-org!).

(In reply to Chaos Prevails from comment #80)

  1. add a button which says "send unencrypted" or similar in the OpenPGP Message Security popup. Maybe, there can be another warning afterwards like "the email will be sent UNENCRYPTED", and with a "send UNENCRYPTED" button, but important is, that users can click through a linear workflow, and not have to go back to the composer again.

Hi, we're working on a new key assistant dialog to replace the current dialog, which comes with a "disable encryption" button in it.

(In reply to Alessandro Castellani [:aleca] from comment #81)

Hi, we're working on a new key assistant dialog to replace the current dialog, which comes with a "disable encryption" button in it.

See here for more information:
https://thunderbird.topicbox.com/groups/e2ee/T79006a5b486f6945/openpgp-enhancements-to-composer

thanks for the topicbox link, I'm also following the discussion there.

It would be really really great if the current Daily features (having the "encryption" button in the composer, having the "do not encrypt" yellow button at the bottom would make it into TB102 (which I assume would be the official release this summer/autumn, is this correct?). If the key assistant dialogue can get an upgrade as well, that would be swell. However, for usability, buttons which are visible (and not hidden behind the "security" dropdown), and considerable less clicks to send an unencrypted email, make a world of difference for us (being an org with non-tech common users who need to send an mixture of encrypted, and unencrypted emails, but can't opt-in to encryption, because every non-tech human being will simply forget (for the same reason that nobody has to opt-in for https: if https would need a user-induced opt-in, it would not be adopted as widely as it is now)

(In reply to Kai Engert (:KaiE:) from comment #79)

I've filed bug 1765974 as a blocker bug for this one.

If we aren't able to fix bug 1765974 in time for the TB 102 release, we must disable the key status notifications in composer window.

(Showing no status is better than showing incorrect/misleading status.)

I understand the argumentation, but from my experience (being IT officer for a non-tech org with about 100 staff members): a very small fraction of users actually understand (or cares about) the process of GPG, and the same little amount of staff members will care/dare to open the OpenPGP key manager (or any settings/configuration related).

100% of my colleagues want to write encrypted/unencrypted emails in an as simple as possible way.
2% of my colleagues want to understand the GPG process, or would want to open any PGP related settings (thus potentially updating/adding/removing keys, which would need a visual update in the composer).

Showing the buttons in a TB release as soon as possible increases the usability of the encryption functionality, and leads more people to actually use encryption (because it can be disabled easily).

Maybe this bug can be treated as less important, so it does not block such a tremendous increase in usability when it comes to the visible buttons?

Comment on attachment 9271745 [details]
Bug 1627956 - Refresh key assistant views after viewing key details. r=mkmelin

Revision D143385 was moved to bug 1765974. Setting attachment 9271745 [details] to obsolete.

Attachment #9271745 - Attachment is obsolete: true
No longer blocks: 1760150
See Also: → 1760150
Depends on: 1767592

Comment on attachment 9271907 [details]
WIP: Bug 1627956 - Adjust tests when using key assistant by default.

Revision D143482 was moved to bug 1767592. Setting attachment 9271907 [details] to obsolete.

Attachment #9271907 - Attachment is obsolete: true
See Also: → 1768153
Depends on: 1768926

For those of you who would like to test the current improvements, but who are currently using beta with your primary profile, and who don't want to use the Daily build for whatever reason:

Here is an experimental build based on today's beta snapshot, adding most OpenPGP related changes, that are not yet in the official Beta:
https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=d28c48263f9f357451be2d546dd5a9c0439ce613

(Click on a green B, click artifacts, and download the appropriate target.* file for the platform)

Kindly assigning this bug to Kai since he's been doing almost all of the work :D

Assignee: alessandro → kaie
Depends on: 1770939
Depends on: 1771122
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: