1615994 - [Thunderbird Telemetry] collect usage statistics of secure email

Status: RESOLVED FIXED

(Thunderbird :: Security, task)

Description

[Magnus Melin [:mkmelin]]

It would be useful to understand how much secure email is used, and how that usage will vary over time.

ATM, we should add

• How many read signed emails S/MIME.
• How many read encrypted emails S/MIME.

Xref bug 1615989 and bug 1615990.

Once we have OpenPGP, same probes for that.

Comment 1

[Magnus Melin [:mkmelin]]

[We now have OpenPGP too.]

Comment 2

[Ping Chen [:rnons] (away)]

Attachment: 1615994.patch

When an email is both encrypted and signed, it seems the MIME content type can only tell it's encrypted, but not clear if it's signed. This patch took the easy path by only reporting it as an encrypted mail, is this alright?

Besides, this patch only reports when an unread secure mail is displayed in messagepane. No report when mails or folders are marked as read.

Will add tests after your confirmation.

Comment 3

[Magnus Melin [:mkmelin]]

Comment on attachment 9152603 [details] [diff] [review]
1615994.patch

Review of attachment 9152603 [details] [diff] [review]:
-----------------------------------------------------------------

Since the UI will know the security info like (both encrypted and signed) you should probably hook it up from there.

::: mail/base/content/folderDisplay.js
@@ +16,5 @@
>  var { MailUtils } = ChromeUtils.import("resource:///modules/MailUtils.jsm");
>  
> +var EnigmailMime = ChromeUtils.import(
> +  "chrome://openpgp/content/modules/mime.jsm"
> +).EnigmailMime;

Please use the more common patter for these:

var { EnigmailMime } = ChromeUtils.import("chrome://openpgp/content/modules/mime.jsm);

(There's a bunch to clean up in the imported Enigmail code.)

@@ +97,5 @@
> + *
> + * It's more involved when a mail is both encrypted and signed. Currently, it's
> + * only reported as an encrypted read.
> + */
> +function readSecureMailProbe(msgHdr, uriSpec) {

I'm not sure where, but I think we probably get this information somewhere else already, and/or should just grab it then. Getting the mime tree is after all not a super cheap operation I think.

@@ +105,5 @@
> +    mimeMsg => {
> +      let matches;
> +      let key;
> +      if (
> +        (matches = /application\/pgp-(signature|encrypted)/.exec(

add i to make it case insensitive

@@ +109,5 @@
> +        (matches = /application\/pgp-(signature|encrypted)/.exec(
> +          mimeMsg.fullContentType
> +        ))
> +      ) {
> +        key = matches[1] === "signature" ? "pgp-signed" : "pgp-encrypted";

could be better to switch the name around. It should also be openpgp instead of pgp (which is a trademark). So signed-opengpg, encrypted-openpgp etc.

::: mail/components/telemetry/Scalars.yaml
@@ +135,5 @@
> +
> +  read_secure:
> +    bug_numbers:
> +      - 1615994
> +    description: How many secure emails are read.

Might we worth being a bit more elaborate, like "how many times different kinds of secure emails are read (for the first time)".

Comment 4

[Ping Chen [:rnons] (away)]

Since the UI will know the security info like (both encrypted and signed) you should probably hook it up from there.

I tried to do that at first, but the message has already been marked as read then. I can't tell if it is a first time read or not. I will take another look.

Comment 5

[Ping Chen [:rnons] (away)]

Attachment: 1615994.patch

Fixed review comments and added test.

I'm not sure where, but I think we probably get this information somewhere else already, and/or should just grab it then. Getting the mime tree is after all not a super cheap operation I think.

I didn't figure out a clean way to do that. Seems the signed/encrypted info are not stored anywhere, but only put onto the two icons. To get info from the two icons, the message would have been marked as read.

I found that getMimeTreeFromUrl involves a network request, I tried to disconnect my network, and it still works, so I guess the response is somehow cached. And in this patch, it's only called for the first time of reading a message, maybe it's not too bad.

Comment 6

[Ping Chen [:rnons] (away)]

Attachment: 1615994.patch

Re-upload, since the last patch missed User/Date field at the top.

Comment 7

[Magnus Melin [:mkmelin]]

Kai, do you know some better place we could tap into this info?

Comment 8

[Kai Engert [:KaiE:]]

Jon, do you have an opinion if this kind of telemetry is acceptable for everyone in the community of OpenPGP users?
Given that email produces metadata, could it be reasonably argued that this telemetry won't leak any more "this user uses openpgp" than is already available anyway?

Comment 9

[Jon]

Ideally, a lot of "this user is using PGP" metadata would be obscured by TLS usage among email servers.

I think an opt-in approach on this would be the safest path given the sensitivities of who is currently using PGP at all, and could also open the door to gathering anonymized data on specific features being used/ignored/abandoned mid-process.

Obviously the downside of opt-in is that you'll get a small percentage of users who are already pretty dedicated to privacy protections choosing to opt in, and those who do may present a skewed user group.

We can get you in touch with the Clean Insights team (https://cleaninsights.org/about) who we're working with on other privacy-forward tools and ethically collecting more usage data to improve tool development; and obviously Mozilla's Telemetry work is already advanced in similar practices.

Comment 10

[Kai Engert [:KaiE:]]

Thanks for the feedback Jon.

We should ensure that this telemetry data will never be sent to anyone but Mozilla.
Magnus, Dan, do you know if that is already guaranteed?

For example, is it guaranteed that telemetry data will only ever be sent to a Mozilla owned server?
What protection mechanism is used to guaratee that?
Is it simply done using TLS and hostname checking, and assuming there won't be a rogue server certificate involved?

(This might be sufficient, because if anyone has the ability to fake a Mozilla telemetry server certificate, they could equally fake the TLS certificate of an email gateway.)

Comment 11

[Kai Engert [:KaiE:]]

I'd be interested to know if it's reasonable for users being concerned, if we have some realistic expectation, that telemetry won't reach anyone but Mozilla.

Comment 12

[Magnus Melin [:mkmelin]]

The data is only sent to Thunderbird's servers. Where else would we send it? And yes, over TLS so server couldn't be faked or traffic intercepted.

The data doesn't contain personal information, so I don't see an issue. If someone is watching the mail traffic, much more, and personal, identifiable data is available from the mail headers alone (where encrypted or not would be obvious, and who to the email address level.)

All Thunderbird gets is: someone, somewhere, sent X amount of signed/encrypted emails.

Comment 13

[Kai Engert [:KaiE:]]

The patch adds a CC license to the new test file. Was there a special reason to do so? We generally use the MPL licenses in all TB files, please have a look at other existing files.

(In reply to Magnus Melin [:mkmelin] from comment #7)

Kai, do you know some better place we could tap into this info?

I don't know a single place.

I agree that obtaining the mime tree, and parsing it separately for telemetry should be avoided.

I suggest that we identify all places that will update the user interface to set the "is encrypted" or "is signed" state. There are just a few.

For S/MIME, this happens here:
(1) https://searchfox.org/comm-central/rev/af886b93f1a520cb98cb6c0e3eda612a1caf2560/mail/extensions/smime/content/msgHdrViewSMIMEOverlay.js#59
(2) https://searchfox.org/comm-central/rev/af886b93f1a520cb98cb6c0e3eda612a1caf2560/mail/extensions/smime/content/msgHdrViewSMIMEOverlay.js#145

If we return early from that function, don't count it.

For OpenPGP, it's done here:
(3) https://searchfox.org/comm-central/rev/af886b93f1a520cb98cb6c0e3eda612a1caf2560/mail/extensions/openpgp/content/ui/enigmailMsgHdrViewOverlay.js#522

I agree we should only count messages when they haven't been read before. I suggest to remove function readSecureMailProbe, and decouple the detection of "this is a new message" from "this is encrypted/signed".

At the position where your patch currently calls readSecureMailProbe, you could set a hidden attribute somewhere in the user interface. (An attribute that gets reset to the "not new message" state whenever we switch to a different message.)

Something like

if (msgHdr && !msgHdr.isRead) {
  document.getElementById("<id-of-an-existing-appropriate-element>").setAttribute("initial-message-read", "true");
}

Then, at the time we update the user interface to show the signed/encrypted status, query the attribute. Only if the attribute is true count it for telemetry.

Comment 14

[Magnus Melin [:mkmelin]]

Maybe we should just send out a custom event for the places a secure message is handled. Like

window.dispatchEvent(new CustomEvent("displaysecuremsg", { msg: gFolderDisplay.selectedMessage; }));

Then somewhere there would be a listener that could take care of read etc. logic, and add to Telemetry what's needed.

Comment 15

[Magnus Melin [:mkmelin]]

... and some more details of the sec state we found.

Comment 16

[Kai Engert [:KaiE:]]

Jon, Magnus, I've done more brainstorming on this topic. I'm feeling a bit uncomfortable about enabling this telemetry data. Please see what I wrote in bug 1644139, which I've currently set as a blocker to this bug. I'd like to ask that we think hard about the potential consequences, before we make a decision.

Comment 17

[Wayne Mery (:wsmwk)]

The openpgp probe may be the best proxy going forward to measure how reluctant enigmail users are to update.

Kai, can you ensure that bug 1644139 gets resolved, so we can get the probe landed?

Comment 18

[Kai Engert [:KaiE:]]

Magnus resolved it.

Comment 19

[Ping Chen [:rnons] (away)]

Attachment: Bug 1615994 - Collect read statistics of secure email. r=mkmelin

Comment 20

[Ping Chen [:rnons] (away)]

Attachment: 1615994-data-collection-request.txt

Comment 21

[Ping Chen [:rnons] (away)]

Attachment: Bug 1615994 - Collect read statistics of secure email. r=mkmelin

Comment 22

[Andrei Hajdukewycz [:sancus]]

Comment on attachment 9189684 [details]
1615994-data-collection-request.txt

This is good to go, below follows the data review response form.

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, in the probe definition files.

2. Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, the normal telemetry opt-out.

3. If the request is for permanent data collection, is there someone who will monitor the data over time?

Magnus(:mkmelin).

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction Data.

5. Is the data collection request for default-on or default-off?

Default-on.

6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No.

7. Is the data collection covered by the existing Thunderbird privacy notice?

Yes.

8. Does there need to be a check-in in the future to determine whether to renew the data?

No.

9. Does the data collection use a third-party collection tool?

No.

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/5afbcaa5a8f9
Collect read statistics of secure email. r=mkmelin, data-r=sancus

Comment 24

[Ping Chen [:rnons] (away)]

Attachment: 1615994-esr78.patch

Rebased to esr78.

[Approval Request Comment]
Regression caused by (bug #):
User impact if declined: Read statistics of secure mail are not collected.
Testing completed (on c-c, etc.): On c-c.
Risk to taking this patch (and alternatives if risky): Low.

Comment 25

[Wayne Mery (:wsmwk)]

Comment on attachment 9192974 [details] [diff] [review]
1615994-esr78.patch

[Triage Comment]
Approved for esr78

Comment 26

[Rob Lemley [:rjl]]

Thunderbird 78.6.1:
https://hg.mozilla.org/releases/comm-esr78/rev/439df6995098