18882 - Support "plussed" addresses as described in RFC 5233

Status: NEW

(MailNews Core :: Composition, enhancement)

Description

[Phil Peterson]

Regarding the composition of From: headers on one's outgoing mail, it is
clear from the various comments attached to bug #17319 that there are
varying opinions on the advisability of allowing users excessively easy
access to the From: header of outgoing mails that they send.  However one
of the comments attached to this ``bug'' (really an enhancement request)
noted the scenario of incoming e-mails sent to <[e-mail]> and to
<[e-mail]>, where both sets of incoming e-mails are presumably
handled by the same person, and where the e-mails may perhaps even be
funneled into the same single POP3 (or IMAP) account on that person's
local mail server.

I have a concern about that situation and also about a somwhat more sophisti-
cated version of the same general sort of situation, e.g. a person who has
several different ``identities'', each of which corresponds (in some obvious
way) to one or another of that user's IMAP folders.

As far as I know, there are two flavors of IMAP server that are in wide-
spead use, namely the UW IMAP server and the CMU Cyrus IMAP server.  If
the information I have read is correct, both of these support so-called
"plussed" addresses of the form <user+tag@domain> where the `tag' portion
can be used by the IMAP server's associated local delivery agent as a
selector which determines which specific folder the incoming message
should be delivered to.

Plussed addresses are already in use by a number of people... in particular
those who use either IMAP or procmail... as a way of pre-sorting their in-
coming mail into various folders.  For example I might use the address
<[e-mail]> when I sign up for the FOOBAR mailing list.  By
using the address <[e-mail]> when I sign up, I can get my
local mail delivery agent (and/or procmail) to help me with the task of
sorting my incoming e-mail by delivering all incoming traffic from the
FOOBAR mailing list directly to my private "foobar" folder, rather than
into my default IMAP "INBOX" folder.

This sort of usage may not be widespread (yet) but it is catching on, I
think.  It is catching on, in particular, among those who are fortunate
enough to be able to fetch their mail from an IMAP server, rather than
from a POP3 server.

There is a serious problem with this usage however.  Unless the Mail User
Agent (MUA) provides some support for this use of plussed addresses, the
whole scheme becomes more trouble than it is worth.

If I sign up for the FOOBAR mailing list using the subscription e-mail
address <[e-mail]> then whenever I want to post something
to that mailing list I must do so from the address <[e-mail]>,
*NOT* from my un-tagged nominal address <[e-mail]>.  (This is required
because the FOOBAR mailing list, like most well-administered mailing lists
these days, does not accept postings from non-subscribers.)

For this reason, even if no other, I and other people would benefit from
being able to modify the From: address on our outgoing E-mails.

In fact, for us, at least, it would be greatly beneficial if the end user
mail client actually provided some support itself for this usage of plussed
addresses.  Ideally, if I am am using Mozilla as my MUA, and if I currently
have my "xyz" folder selected, and if I then begin to either compose a new
message or else I begin to reply to an existing message (in the currently
selected "xyz" folder) I would really like to have the Mozilla MUA just
automatically pop up a composed From: line that looks like:

        From: "Ronald F. Guilmette" <[e-mail]>

i.e. it should use the currently selected folder name as the "tag" part of
a plussed address... a plussed address which it composes on-the-fly at the
moment I begin to compose a new outgoing message.

Without this, any attempt to use plussed addresses becomes more trouble
than it is worth, because each and every time I try to make a posting to
any of the mailing lists I have subscribed to (using plussed addresses) I
would have to manually type in the proper (plussed) address that corresponds
to the address that is my true ``identity'' with respect to that specific
mailing list.

For this reason, it would be most helpful if Mozilla could itself be helpful
enough to both understand and properly support plussed addresses... at least
for some subset of my personal IMAP folders.

That having been said, I should also say that the automated plussed-address
generation behavior that I described above will obviously not be desirable
for all mail users or even for all of the folders of any given mail user.
Rather, the ideal mail user agent (MUA) would allow the end user to explicitly
specify his/her From: ``identity'' separately for each of that user's separate
IMAP folders.  But even this per-folder selection should be made as painless
as possible, and when the time comes to specify the From: address to be
associated with a given folder (say, for example, folder "abc") the user
should be offered a menu of reasonable pre-canned choices of possible From:
addresses for the given folder, like for instance:

        <user@domain>
        <user+abc@domain>
        <abc@domain>
        other {fill in your own text}

(The third case in the list above might work out very nicely for the postulated
case of the single user answering e-mail for <[e-mail]> and also for
<[e-mail]>... at least if those two addresses were gatewayed into
two different IMAP folders, both of which being owned by the relevant user.)

That's about all I had to say, other than to say that I seriously hope that
Mozilla 5.0 will ship with good and proper support for the increasingly
popular combination of IMAP and plussed addresses, along the general lines
I've noted above.  (I, at least, actually *cannot* use Mozilla as my mail
client at the present time because of its current lack of support for plussed
addresses.)

Thank you all for your attention.

Comment 1

[Alec Flett]

I'd just like to add that plussed addresses have been in use since around 1987,
(originally part of CMU's Andrew Messaging System) and really doesn't show any
sign of catching on any time soon.

However, I think plussed addresses could be just a special case of multiple
identities per account. There is backend support for THIS.

Here's how I could see handling plussed addresses though... and it requires some
work from MIME and mail composition.

Basically, we need MIME or mail composition to figure out, at composition time,
what the identity in the currently displayed message is. It needs to look at the
"To:" "CC:" or "Resent-To:" header to figure out who the mail was addressed
to... this can then become the default identity when the compose window opens.
I'll file a bug on that.

Once that feature is implemented, then we should just have a preference,
something like "mail.ignore_identity_chars", "+" - where the identity-detection
code would ignore any characters after the "+" when trying to determine the
identity, and would possibly tack those characters onto the identity in the From
header int he compose window

Comment 2

[Alec Flett]

adding ronald to CC

Comment 3

[Ronald F. Guilmette]

Regarding Alec's point about plussed addresses not catching on, let me
just say that I think that this is a classic chicken-and-egg problem.
Maybe plussed addresses WOULD start to become more widely
used than they already are, *IF* mail client software (e.g. Messenger)
provided what would appear to be the minimal level of support for plussed
addresses necessary to make them really usable.  (Of course, it would also
help a lot of more mail administrators started providing IMAP service,
in addition to POP3 service, and if they arranged for mail addressed to
plussed addresses to be delivered to distinct folders.  But again, its a
chicken and egg problem.  The mail admins won't do any of that until there
seems to be a demand for it, but end user demand for this functionality
won't materialize until more people know that such nice things are even
possible, and that won't happen until more mail client and more mail
server start to support plussed addresses.)

Comment 4

[Alec Flett]

that's a good point... I do like the feature, and would love it if we supported
it...

Comment 5

[Alec Flett]

ok, I filed the other bug for even-smarter-identity choice at compose time.

Comment 6

[Ronald F. Guilmette]

At the risk of boring you guys even more than I already have, I'd like to talk
in even a bit more depth about this question of plussed addresses and message
composition.  (Keep reading.  The payoff is that I think that I have a rather
simple solution that will allow me and other to use plussed addresses quite
effectively.)

First let me say that yes, Alec has a very good point about how (ideally) if
one is replying to a message that was sent to you via one of your public
E-mail aliases, e.g. <alias1@domain>, then ideally, if you then go to com-
pose a reply to that message, your default From: ``identity'' for that
specific reply should be <alias1@domain>.  That's obvious.  The bad news
(relative to that ideal situation) is twofold...

First even though RFC 1869
(ESMTP) added an (optional) ORCPT= parameter for the RCPT TO command (through
which the _original_ E-mail address to which a message was addressed could be
preserved, even across multiple .forward and alias mungings of the actual
envelope recipient address, that capability is NOT widely used/supported (yet?)
and thus, it is REALLY hard to know what the real ``original'' recipient address
actually was at the moment the original sender hit the SEND button.  (And that
is what you would like to have in order to fully implement what Alec talked
about.)

Second, and perhaps more importantly, not all of the outgoing messages I send
are replies.  Sometimes they are just brand new messages.  So you still have to
answer the question ``Where does the default From: address come from for
those?''

So anyway, setting aside (for the moment) the idea of generating a default
From: header based upon the address to which the message you are replying to
was sent, I have what I think is a reasonably simple proposal for a solution
to the problem that I first proposed, i.e. how to get the default From: address
for an outgoing message that the user is composing to be an address of the form:

    <user+folder@domain>

where ``folder'' is just the name of the most recently visited folder.  Quite
simply, I'd like to propose that whatever string the user specifies as his
e-mail address for outgoing mail (i.e. the one that gets stashed in the `email'
field of the nsIMsgIdentity.idl thingy) no longer be treated just as literal
string, but rather as a string which is routinely subjected to some minimalist
sort of INTERPOLATION.  Specifically, if the given address string contains
TWO CONSECUTIVE PLUS SIGNS, then this indicates that that portion of the
address should be replaced (nominally) by a single plus sign and then the name
of the most-recently-visited folder.  (If the most recently visited folder
is "INBOX" however, this is treated as a special case, and the two plus signs
are then replaced by nothing.)

This simple interpolation scheme for the user's e-mail address would allow me,
for example, to specify my e-mail address to Messenger as:

      <[e-mail]>

(at least for the specific IMAP server that I have an account on and that I
know supports plussed addresses)
and this alone would be sufficient to make IMAP folders and plussed addresses
work ``right'' for me.  And with this approach, plussed addresses would work
``right'' both for the case where I am replying to messages and also when I
am composing fresh messages (as long as I remember to visit the relevant
folder first, because I start to compose any message that I want to be
associated with the relevant folder's asociated ``identity'').

So?  Whadaya think?

Simple, no?

The advantages of this interpolation scheme should be obvious.  The most
important one is that the documentation of this interpolation feature could
be neatly tucked away in some ``advanced'' portion of the documentation (where
power users could find it but ordinary end lusers wouldn't see it OR become
confused by it) and best of all, you wouldn't have to make _any_ obvious
changes to your existing carefully-designed UI.

Comment 7

[Alec Flett]

ok, the plus sign is a whole lot bigger issue than I thought :)
I forgot that the default + behavior was so smart on the Cyrus server :)

(In the old AMS world, we had to write LISP scripts to parse out the text after
the + and the choose an appropriate folder from there!)

well, the other bug is file (18906) to allow for the auto choosing of reply
during compose based on the currently displayed message, and will definately
benefit the + people

So I'll try to explain what needs to be done in the Mozilla world for this to
happen.
we need to create a "phantom" nsIMsgIdentity at compose time to serve as a
temporary identity that refers to the current folder. This could work... it
would involve slight tweaks to the compose code so that we pass the identity
itself rather than the identity key (because the "phantom" identity would not
live in the account manager)this would come in to the compose window as a
parameter of openDialog() I think.

We'd of course do all of this based on some preference.

Comment 8

[Christopher W. Curtis]

I don't want this to spiral too far off topic, but I was going to suggest the
"last visited folder" approach, but a little differently.  Instead of changing
your "from" header at all, just leave it be [e-mail].  Then, in the
properties for the _folder_, have a checkbox stating "append +folder to outgoing
mail".  This way, if that folder is open, the outgoing mail header would have
the +folder automatically appended, under the assumption that if you are
composing a new message from this folder, you want replies to go to this folder
(ie, to that list).  This does not completely solve the problem, however: you
may be in a folder but not want to send it to the list.  Rather than forcing the
user to select a non-plussed folder, display the "From:" header in the messenger
window, and have the "+list" (if not highlighted by default) be editable.  This
could also apply for folders not maked as "plussed".  Display the From: line,
and let text be entered before the first @ symbol if preceeded with a "+".  I
don't know all the RFC rules, but special filtering will of course be needed (no
'!' marks, no '@', etc.)

An aside reason I like this setting being at a folder level (even if editing the
from header is allowed, I think that folder level settings are still needed) is
because I procmail incoming messages.  However, it seems that Messenger only
checks the special "Inbox", and I would like to see a folder flag stating "mark
as incoming mail folder".  Then when "Get Messages" is checked, or MozBiff runs,
it'll check all my incoming folders.  xbiff++ did this well (IIRC), with
different events based on the folder or any other criteria.

Comment 9

[Robert O'Callahan (:roc) (email my personal email if necessary)]

I'd just like to add that at CMU we have support for plussed addresses with POP
as well as IMAP, and they get used a *lot*. It's great to give out a different
email address every time you fill in a Web form --- when someone passes on your
address to someone else, you know right away who it was. They may not have
"caught on" everywhere, but I'd never want to go without them. BTW, our system
supports = as well as +, but that may be a local quirk.

Comment 10

[Matthew T (active 1999-2002)]

Re rfg's suggestion about xyz++@domain: but what happens for someone whose 
address really *is* user++@domain? They'll have all sorts of strange things 
happening to their e-mail without knowing why.

Specifying your address as xyz++@domain to activate a specific option brings to 
mind UI monstrosities like making the user enter \n for a new line or whatever. 
Better, I think, to have a pair of radio buttons in the account setup, to specify 
whether you want a normal identity set or a `plus-powered' identity set for that 
account.

Re ccurtis's suggestion of some folders being plus-powered and some not: that 
might be more trouble than it was worth. First, you'd need little plus signs 
stamped on the icons for the plus-powered folders, to stop people from wondering 
why something had or hadn't happened when they'd selected a particular folder. 
And second, you'd need a non-trivial UI design for entering the post-plus part of 
the address if a non-plus-powered folder had been selected at the time.

A nice touch would be to be able to drag a folder from a mail window onto the 
From field in a composition window, which would magically change the From address 
to <user+foldername@domain>.

Comment 11

[Ronald F. Guilmette]

I have two brief points.

First, I am forced to agree with both of the points made by
<[e-mail]>.  He is correct that my (hack?) idea of
allowing users to specify some quirky form of personal return
address in order to avail themselves of some extra, and very magic
on-the-fly personal return address interpolation is rather entirely
kludgy and sub-optimal, and that this would be better done by adding
a radio button to select this option to the Communicator user interface.
I also agree that it would be an unnecessary hassle (and also
overkill) to try to support functionality allowing some folders to be
``plus powered'' while others are not.

The only other point I want to make is that there is one big
fallacy in everything I have said so far with regards to this
Bugzilla thread, namely my implicit assumption that if a mail
message comes in for <user+xyz@domain> that it will in fact
be delivered to the `xyz' folder of `user'.  That sort of one-
to-one mapping of (what I personally call) the `tag' part of
the address to folder names may not necessarily hold in some
cases.  On some systems, now or in the future, it might perhaps
be possible for the end user to request that mail sent to
<user+xyz@domain> be delivered instead to that user's `abc'
folder (rather than to his `xyz' folder).

This possibility raises a new problem for any sort of mail
client that is going to attempt to infer a `proper' sender
address for each new reply message the end user may begin
to compose.  Obviously, if the user is replying to a message
that was addressed to <user+xyz@domain>, but one that was
actually *delivered* to that user's `abc' folder, then it would
be inappropriate to infer, automatically, a sender address of
<user+abc@domain> when the end user goes to compose a reply to
that newly received message.  Rather, because the sender of the
original message may only know the user in question by his
<user+xyz@domain> ``identity'' (and because that original message
sender may perhaps even have his message reception filters set
to reject mail from any and all other parties) it will be important
to insure that even though the message was delivered to the user's
`abc' folder, the message itself should still carry with it some
marking or indication that it was ORIGINALLY ADDRESSED to the
address <user+xyz@domain>, even though it may have ended up in
the recipient's `abc' folder... either via some automated magic
on the server side, or because the end user manually moved it to
that folder sometime after it was delivered.

Attaching this additional ``original recipient address'' info to each
message is clearly the responsibility of the MTAs through which
the message passes on its way to its destination, together with
the terminal Mail Delivery Agent, and one or another of these
programs should, ideally, arrange for the ``original recipient
address'' to be attached to the message, perhaps via an `X-Really-To:'
or an `X-Original-Recipeint:' header, or some such thing.  (I previously
described in this thread some of the behind-the-scenes machinations
in MTAs that might be necessary to make this happen.  But that stuff
is clearly beyond the scope of the discussion here about mail *clients*,
so I will say no more about that.)

So anyway, it is the responsibility of the MTAs and the local/final
MDA to somehow attach the ``original'' recipient address to each
delivered message, and this can probably be accomplished most easily
via some new X- mail header.  Once that is accomplished... and *I*
already have an MTA/MDA combination that does it... it is then the
responsibility of the mail *client* to be smart enough to snatch that
original recipient address information out of any message which the
end user has just asked to compose a reply for.  The mail client should
then use that ``original recipient'' address in both the From: header
AND also as the SMTP envelope MAIL FROM address for the reply that the
user composes.

As I understand it, this exact sort of capability/functionality is
already present and available in the Eudora mail client.  Specifically,
I have been informed that one can instruct Eudora to use the contents
of some arbitrary end-user-selected mail header (e.g. X-Really-From:)
as the From: address when composing outgoing replies.

I am currently engaged in building some fancy server-side software
which will in fact allow the end user to specify that incoming mail
addressed to <user+xyz@domain> be, in effect, redirected and delivered
to that user's `abc' folder instead.  But because Mozilla/Communicator
currently lacks the specific client-side functionality described above
(for Eudora), which is clearly the minimum functionality necessary to
make this stuff work seemlessly (from the end user's perspective) my
current plan is to just tell all of my end users to get Eudora.  I *do*
wish that it were otherwise, and I do wish that I could also recommend
Mozilla/Netscape, but without the functionality I just described, I don't
believe that will be possible.

Comment 12

[A. Craig West]

A minor note about 'plussed' addresses. Sometimes they are 'minused' as well. To
pick an example entirely at random, [e-mail].

Comment 13

[m_mozilla]

acwest: sometimes they *are* minused, or in my case underscored.

However, there is wide-spread existing support for the "plus" convention, and it
conflicts with far fewer standard email addresses. For example, many mailing
list managers use dashes (minuses) to separate list names and actions
(listname-subscribe@) and many places use underscores as part of a username
(lastname_f@) whereas the plus sign doesn't have such widespread use outside of
this convention.

-matt

Comment 14

[A. Craig West]

I hadn't seen underscore before... In the case of the hyphen separator, this is
the default on a qmail based system. The plus separator is the default on
sendmail, making it a lot more common. My thought is to make this
user-configurable, although a default separator of '+' makes sense.

Comment 15

[Serge Gautherie (:sgautherie)]

Filter on "Nobody_NScomTLD_20080620"

Comment 16

[aleth [:aleth]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=3cf31cdbe9bd

Comment 17

[u677327]

Receiving a message at [e-mail] should reply to [e-mail], but TB instead replies to [e-mail] which makes this a defect.

This has caused many problems, including email ticket systems being unable to rely on from addresses so they must use custom headers.

Whoever thought it was a good idea to tamper with the from address should have all their code reviewed.

Comment 19

[david]

How is this different from an alias, which is what it is. Thunderbird works properly when users with aliases set them up as identities. Is this a technical problem, or a documentation problem. I see it as a documentation issue, as the plus addresses work properly when so defined. The current process of requiring identities also ensures that the email address will work, whereas allowing overwriting the TO address opens the door for more problems. See https://support.mozilla.org/en-US/kb/using-identities. I have one email account (Gmail) that uses the plus addresses, and I have another email account that also allows aliases in a different form, so focusing on the 'plus' aliases alone would be incomplete, but the current structure works for all. I see this as a non-issue.

Comment 20

[Magnus Melin [:mkmelin]]

I think what's requested is that, in case an identity with plussed email wasn't set up, e.g. replies should understand to fall back to the correct identity (like comment 17). Or perhaps keep the plussed address when replying even if no plussed identity existed.

Comment 21

[david]

I will leave only one final comment: the RFC at https://www.rfc-editor.org/rfc/rfc5233 is already fully supported by Thunderbird, in that any alias, whether 'plus' or other, is supported. The minor requirement is that the user include all email addresses within the account. Doing this 'on the fly' seems unnecessary. Email providers make the feature available, and all Thunderbird needs to do is provide a simple way to use them, which it does. Since the function is there, doing this 'enhancement' only eliminates the need for the user to include aliases in the account definition. As a user, I would much prefer to see developers focus on new features or important bugs, not new workarounds. Thank you for listening, and I'll stand down.

Comment 22

[Wayne Mery (:wsmwk)]

A brief search suggests the following clients support plussed addressing:
Gmail (of course)
Outlook.com / Microsoft 365: Supports it on personal and enterprise accounts (Exchange Online). Note: It may require PowerShell to enable in some older Microsoft 365 environments.
Proton Mail
Fastmail
Yahoo
Zoho
iCloud
GMX & mail.com
Tuta

I didn't find an item for this at https://connect.mozilla.org/t5/ideas/idb-p/ideas/label-name/thunderbird, so someone is welcome to file one for everyone to vote.