776397 - privacy enhancement: prevent local timestamp disclosure via Date and Message-ID header fields

Status: RESOLVED FIXED

(MailNews Core :: Composition, enhancement)

Description

[tagnaq]

Attachment: fixes local time disclosure via Date header

Hi,

Background:
Jacob Appelbaum, Sukhbir Singh and me are working on a Thunderbird extension (TorBirdy [1]) that aims
to prevent information disclosure that could harm the privacy of a Thunderbird user when 
sending or receiving mails via Tor.
So far everything was implemented via the extension but to solve a particular local information disclosure 
we need to apply two little changes to Thunderbird because it can't be fixed via an extension only.

We choose to submit two changes in one bugzilla entry because both changes are required to fix the same underlying issue.

The Problem
As described in [2] (chapter 3.5.3, 3.5.4 and 4.2.3) a privacy adversary might use the local date information
disclosed via Date and Message-ID headers in outgoing emails to reduce the anonymity set of the sender, or to
link multiple pseudonyms to a single origin.
We were not able to modify Date and Message-ID header fields via an extension to prevent these privacy leaks
without changing Thunderbird.

The Solution
We propose the following little changes to prevent local date information disclosure via Date and Message-ID header 
entries in outgoing emails. These changes have no effect on the default behaviour of Thunderbird users. 
It just allows users to opt-in for better privacy. Everyone using the TorBirdy extension chooses to opt-in.

The two little changes are:

1) add a new pref (mailnews.local_date_header_generation, default: true) that allows the user to specify
if he wants to send emails with Date header or if that header should be inserted by the MSA.
If this pref is set to false, the Date header will be inserted by the mail submission agent (MSA) as specified
in RFC4409/RFC6409.
Besides our privacy use-case this might also be handy for users who struggle with their time settings (outgoing emails
will always have the server generated time - which is usually more accurate).

Sending emails without Date header has no negative side effect on the receiver (MSA inserts date header).
This has been successfully tested with the following freemail provider:
- gmail (ports: 25, 465, 587)
- lavabit (ports: 25, 465)
- GMX 
- yandex 
- mail.com

Wait 'Date' is mandatory according to RFC2822 [3], you can't send emails without Date header?!
MTAs must send complete emails, but for MUA -> MTA "incomplete" mails are allowed and explicitly mentioned
in the submission RFC [4]:
   "Unfinished messages may need to be completed to ensure they conform
   to the Message Format specification [MESSAGE-FORMAT] and related
   requirements. For example, the message may lack a proper 'Date'
   header field, and domains might not be fully qualified.  In some
   cases, the MUA may be unable to generate finished messages (e.g., it
   might not know its time zone)." [4]

This change is implemented in: date.patch

2) The Message-ID in outgoing emails includes the exact same local date information as the Date header,
therefore this field requires also a change to fix the local time disclosure.
To keep the patch as small as possible the new time-independent message-id format is implemented in
the extension. The patch just allows an extension to set a custom msg-id.
If you are interested on the details of our time-independent msg-id format, I can
elaborate on that.
The following patch allows extensions to modify the Message-ID to an time-independent format:
messageid.patch

Both patches have been developed by Sukhbir Singh.

[1] https://trac.torproject.org/projects/tor/wiki/torbirdy
https://github.com/ioerror/torbirdy/
[2] https://trac.torproject.org/projects/tor/attachment/wiki/doc/TorifyHOWTO/EMail/Thunderbird/Thunderbird%2BTor.pdf
[3] http://tools.ietf.org/html/rfc2822#section-3.6
[4] http://tools.ietf.org/html/rfc6409#page-4


To give you a feeling on how the new privacy enhanced header look like:

1) a mail where mailnews.local_date_header_generation was set to false:

----
[...]
Received: from 127.0.0.1 ([199.48.147.37])
        by mx.google.com with ESMTPS id bj6sm1724438vdc.12.2012.07.14.16.27.12
        (version=SSLv3 cipher=OTHER);
        Sat, 14 Jul 2012 16:27:14 -0700 (PDT)
Date: Sat, 14 Jul 2012 16:27:14 -0700 (PDT)
Message-ID: <[e-mail]>
From: alice@@example.com
MIME-Version: 1.0
To: [e-mail]
Subject: mailnews.local_date_header_generation = false
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

body line 1
----

2) a mail with custom time-independent msg-id (in addition to (1):
The msg-id format is not part of the patch (implemented in TorBirdy extension)

----
Received: from 127.0.0.1 (tor18.anonymizer.ccc.de. [31.172.30.1])
        by mx.google.com with ESMTPS id ef5sm18104076wib.3.2012.07.22.10.52.17
        (version=SSLv3 cipher=OTHER);
        Sun, 22 Jul 2012 10:52:21 -0700 (PDT)
Date: Sun, 22 Jul 2012 10:52:21 -0700 (PDT)
Message-ID: <[e-mail]>
From: [e-mail]
MIME-Version: 1.0
To: [e-mail]
Subject: custom msg-id 
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

foobar
----

We are really looking forward to your reaction to these patches!

Comment 1

[tagnaq]

Attachment: fixes local time disclosure via Message-ID header

Comment 2

[Ian Neal]

You will need to find a suitable reviewer for these patches. I would suggest bienvenu or standard8
You might need to set the default value for the prefs in mailnews.js

Comment 3

[tagnaq]

Attachment: set default values for new prefs

Comment 4

[Jacob Appelbaum]

If there are any changes required for the patches, we'll be happy to make them - this is the critical patch set remaining for TorBirdy.

Comment 5

[Kent James (:rkent)]

Comment on attachment 644790 [details] [diff] [review]
fixes local time disclosure via Message-ID header

Drive by comments, to speed you along (not a complete review):

>+  nsCOMPtr<nsIPrefBranch> prefs(do_GetService(NS_PREFSERVICE_CONTRACTID));
>+  prefs->GetBoolPref("mailnews.custom_message_id", &custom_message_id);

Normally we would also check the return value: do_GetService(NS_PREFSERVICE_CONTRACTID, &rv) followed by
NS_ENSURE_SUCCESS(rv, rv) to prevent a crash on null prefs

>+  prefs->GetCharPref("mailnews.header.custom_message_id", &message_id);

At this point, message_id contains allocated memory that never gets released, so this
is a leak. Better practice would be:

nsCString message_id;
prefs->GetCharPref("mailnews.header.custom_message_id", getter_Copies(message_id));
...
    return PR_smprintf("<%s@%s>",
                       message_id.get(), host);

and let the string class deal with the memory.

Comment 6

[tagnaq]

Attachment: fixes local time disclosure via Message-ID header

Comment 7

[Mark Banner (:standard8)]

Comment on attachment 646189 [details] [diff] [review]
fixes local time disclosure via Message-ID header

Hi, sorry for the delay in getting to look at this.

>+  nsCOMPtr<nsIPrefBranch> prefs(do_GetService(NS_PREFSERVICE_CONTRACTID, &rv));
>+  if (NS_SUCCEEDED(rv)) {
>+    prefs->GetBoolPref("mailnews.custom_message_id", &custom_message_id);
>+    prefs->GetCharPref("mailnews.header.custom_message_id", getter_Copies(message_id));

I think using preferences is the wrong approach here. From the compose window you'll probably be fine because you'll know when the message is sent. However, when you hit things like the mdn generation, then you'll find you may not know a message has been sent. That obviously could lead to you generating multiple Message-IDs that are the same.

I would suggest using nsIMsgCompFields to hold the custom_message_id, but that doesn't look like it will work in the non-compose window instance. Worst case we could probably use a service, that is only implemented by your extension, though if you can find another route, that would be good.

Comment 8

[Mark Banner (:standard8)]

Comment on attachment 644789 [details] [diff] [review]
fixes local time disclosure via Date header

So I tried this, and I think my main concern is that my MTA didn't insert the date when I sent the message.

Additionally, mailnews doesn't seem to display the date on the message header, if there is no date field inserted, so I suspect this may end up breaking things especially when saving the messages to sent mail.

I'm also not sure how the maildir based implementation would cope with a missing date header, as I suspect it isn't going to have the "From - <date>" information that mbox has.

However, I'd like to hear what David thinks about this as he has greater experience in that part of the code base than I do.

Comment 9

[Mark Banner (:standard8)]

Comment on attachment 644794 [details] [diff] [review]
set default values for new prefs

Clearing this review request for now, until the issues with the other patches are resolved (you could always just include these pref additions in those patches anyway).

Comment 10

[David :Bienvenu]

It all sounds rather dangerous to me.

Comment 11

[tagnaq]

Hi Mark,

thank you for your comment.

(In reply to Mark Banner (:standard8) from comment #8)
> So I tried this, and I think my main concern is that my MTA didn't insert
> the date when I sent the message.

I would be interested in the details, because we tested the date patch against a few freemailer. Can you tell which mailserver it was?

Comment 12

[tagnaq]

(In reply to Mark Banner (:standard8) from comment #7)
> I would suggest using nsIMsgCompFields to hold the custom_message_id, but
> that doesn't look like it will work in the non-compose window instance.
> Worst case we could probably use a service, that is only implemented by your
> extension, though if you can find another route, that would be good.

We changed our original patch (that was never submitted to mozilla) where the actual time-independent msg-id format was implemented in the thunderbird patch to a version where thunderbird would just allow extensions to set their own time-independent msg-id format (submitted patch), but to cleanly address the issues with the current msg-id patch we could submit our patch that would implement the time independent msg-id format directly in Thunderbird. The user would then be free to opt-in for the alternate msg-id format by simply toggling a bool pref. 

What do you think about that approach?

We could also formalize our time-independent msg-id format in a short spec if you want.

Comment 13

[David :Bienvenu]

Comment on attachment 644789 [details] [diff] [review]
fixes local time disclosure via Date header

you haven't added a default for the pref in mailnews.js (the default should be the current behavior).

Comment 14

[Sukhbir Singh]

Attachment: set default values for new prefs

Comment 15

[Mark Banner (:standard8)]

Comment on attachment 658360 [details] [diff] [review]
set default values for new prefs

Please just include the specific preference you're adding with the patch it belongs to. That is much easier for reviewers to work with than three separate patches, especially when one of those is still up for discussion.

Comment 16

[tagnaq]

Comment on attachment 644789 [details] [diff] [review]
fixes local time disclosure via Date header

To make it easier for Sukhbir to submit new versions of these patches I'm obsoleting this patch.

Comment 17

[tagnaq]

Comment on attachment 644794 [details] [diff] [review]
set default values for new prefs

To make it easier for Sukhbir to submit new versions of these patches I'm obsoleting this patch.

Comment 18

[tagnaq]

Comment on attachment 646189 [details] [diff] [review]
fixes local time disclosure via Message-ID header

To make it easier for Sukhbir to submit new versions of these patches I'm obsoleting this patch.

Comment 19

[Sukhbir Singh]

Attachment: fixes local time disclosure via Date header

Comment 20

[Sukhbir Singh]

Attachment: fixes local time disclosure via Message-ID header

Comment 21

[Sukhbir Singh]

Attachment: fixes local time disclosure via Message-ID header

Comment 22

[Jacob Appelbaum]

Is there anything else that we can help with? We'd really like to get these patches into Thunderbird sometime soon - I'm happy to run tests or builds or whatever needs to happen...

Comment 23

[Magnus Melin [:mkmelin]]

Having ONE patch with all the changes would be a start... it's really hard to discuss patches stretching over several attachments.

Comment 24

[Jacob Appelbaum]

A single patch for all changes? OK.

Comment 25

[Sukhbir Singh]

Attachment: prevent local timestamp disclosure via Date and Message-ID header fields

Comment 26

[Magnus Melin [:mkmelin]]

Also ask Mark for review (set the "review?" flag on the attachment, and the value of the field to his email address).

Comment 27

[Mark Banner (:standard8)]

Comment on attachment 684554 [details] [diff] [review]
prevent local timestamp disclosure via Date and Message-ID header fields

I still don't think this is the right thing to do, I think it'll break too easily when sending mail and you'll end up with mails of the same id. I think this should have an explicit extension hook rather than a back-door via preferences.

As to the date server, I don't know what my SMTP server is (its an ISP, and there's no info on what they use), and I don't see anything here addressing Thunderbird's handling missing date headers in emails.

Comment 28

[Jacob Appelbaum]

We're thinking that we're going to open two tickets - one for each issue, with a design in each bug and attach corresponding code for review/iteration.

Comment 29

[Jacob Appelbaum]

We opened two other tickets, so this one can be closed and we'll hopefully tackle these issues one by one:

  https://bugzilla.mozilla.org/show_bug.cgi?id=902573
  https://bugzilla.mozilla.org/show_bug.cgi?id=902580

Comment 30

[Arthur Edelstein [:arthur]]

Both child tickets have been resolved.