120926 - New account managing structures proposed

Status: VERIFIED WONTFIX

(SeaMonkey :: MailNews: Account Configuration, defect)

Description

[Wolf Eichler]

Analysis 
--------
(reviewed Mozilla build 2002011703)
The present shape of the account system (mail) is demanding. First of all, it
does not function in sending mails from a multiple accounts setting. Second,
distribution of incoming mails is not satifying.

My assumption is that the presently implemented system does not match the
requirements of the real existing system. It does not take into account the
multitude of possible mailbox settings. The following may exits:

a) a user may have several mailboxes (= a single specific mail address) at
several hosts
   - host alpha.com -> user "justme" -> [e-mail]
   - host beta.com  -> user "justme" -> [e-mail]
   ...
b) a user may have several accounts at a specific host
   - host alpha.com -> user "Frankie" -> [e-mail]
   - host alpha.com -> user "Lola"    -> [e-mail]
   ...
c) a user may have several mailboxes within a specific account
   - host alpha.com -> user "Frankie" -> [e-mail]
   - host alpha.com -> user "Frankie" -> [e-mail]
   ...

To deal with this circumstance it does not suffice to just relate a hostname to
a Mozilla mail account. I propose in this document an account system that is
based on the real user mailbox, i.e. mail address like
"[e-mail]", as a basis for identifying and relating to accounts.
That is the closest to real needs in my eyes. In later editions - to finally
confuse people - more than one mailbox could possibly switched together into one
account if there is a need for it. But that is not recommended for now.

I hope you will consider carefully reading this paper, it could mean the basis
for a solution of the present undesirable state of the account system. I excuse
for its medium lengthiness, best thing to print it out and study it for a while.
Then let me know your comments.


Structures
----------
Account Identification Scheme:
Accounts are unique/identified only through <email-address>, multiple accounts
for a pair {<hostname>, <username>} are allowed. Functions that need to select
an account must be in hold of a related email address slot, the host name and
the user name. If a search with <email-address> does fail - as is conceivable in
incoming mail - the nearest match of {<hostname>, <username>} is selected.

It can be proven that in all cases of incoming and outgoing mail transaction a
predictable selection of an account takes place, that is: selection never fails
(provided UI garantees proper input).

Internal lists:
li-1: list of email addresses of all accounts (wi reference to single account)
li-2: list of {<hostname>, <username>} of all accounts but without double
entries of a specific pair

Procedures
----------
PA. Mail Composing:
i) as sender ident an option of li-1 must be selected
ii) the account ac1 is selected as above (possibly through li-1)
iii) the mail is sent or put into "unsent messages" of ac1

PB. Mail Sending (Multiple Account):
i) mail from all accounts' "unsent messages" are collected, i.e. an index is
built over these messages heading the pair {<hostname>, <username>} of the
account they belong to.
ii) the index is walked through with the following:
   a) at each *new* pair {<hostname>, <username>} a new SMTP session is invoked
     
      with these values
   b) the message is sent accross the currently open SMTP session

PC. Mail Sending (One Account):
i) SMTP is invoked for this account's pair {<hostname>, <username>}
ii) mail is sent from this account's "unsent messages"

PD. Mail Download (Multiple Accounts):
i) for each host (=pair {<hostname>, <username>}) in li-2, a POP3 transmission
is invoked (one transaction if mail is in the drop). The results may be stored
interim or directly processed according to ii and iii.
ii) the results of all POP3 transmissions are analysed for their receiver email
address. 
iii) the results are put into the accounts' inboxes according to the account
identification scheme (see above).

PE. Mail Download (One Account):
i) POP3 is invoked for this account's pair {<hostname>, <username>}
ii) mail is processed as in PD.ii and PD.iii.


UI Rules for accounts
---------------------

ru-1:  consistency of account data

- <pop-hostname>           must
- <smtp-hostname>          must
- <host-username>          can
- <password>               can
- <email-address>          must, unique
- <email-username>         can
- <account-name>           must

ru-2:  compose and import integrity
- the sender email address (field value) is restricted to a member of li-1

Recommendations
---------------

rc-1: any displayed list of smtp-hosts (send) should show their relevant
identity with a pair "<smtp-hostname> <host-username>"

rc-2: for the purposes of this proposal, the UI-display of li-2 (list of
smtp-hosts) is not required. In any case the field <smtp-hostname> should be
simply added to the server settings pane. The present "advanced" button and the
list there are unnecessary uncomfortable.

rc-3: please note that {<hostname>, <username>} is unique in li-2 but the link
of the user input fields is not binding (but only informational). That is,
multiple accounts with the same pair must be possible.

Comment 1

[Ben Bucksch (:BenB)]

Wolf, a few URL for you:
<http://www.mozilla.org/mailnews/>
<http://www.mozilla.org/mailnews/arch/accountmanager.html>
<http://www.mozilla.org/mailnews/specs/accounts/>
bug 44863

Comment 2

[Wolf Eichler]

Enhancement: Uniqueness of email-address can be dropped
-------------------------------------------------------
With experimenting I noticed there may arise a need in users to install more
than one account with the same email-address (for instance to handle with
different sender names or to divide the message store). This flexibility can be
added to my proposal of 2002.01.19. 

I add the following as extensions to my proposal, and so we'ld have a totally
free and well functioning account system.

Account Identification Scheme (substituted):
Accounts are not uniquely identified through user input settings. Multiple
accounts for a pair {<hostname>, <username>} as well as for a specific
<email-address> are allowed. Functions that need to select an account must be in
hold of a related email address slot, the host name and the user name. In a
search for a target account the <email-address> is compared first. If there is
more than one account matching, the first in the subset is selected. If a search
after email-address fails - as is conceivable in incoming mail - the first match
of {<hostname>, <username>} is selected.

PD.iii) (added): If more than one account is matching the value of
<email-address>, the mail is put into each of the accounts' inboxes.

Comment 3

[Ben Bucksch (:BenB)]

Wolf, did you read <http://www.mozilla.org/mailnews/arch/accountmanager.html> and
bug 44863?

Comment 4

[Alec Flett]

yeah, I have to recommend reading the architecture doc - I think you'll find
that there are more back-end capabilities than you realize - we actually split
the concept of "identity" away from the concept of an incoming mail server, and
provide rudamentary (possibly now broken) support for multiple identities per
account. 

I'd also like to add that we've been round and round on this many times and ease
of use has always been a higher priority than exposing every possible
combination of identity/mail server/inbox/etc.. the more you expose, the harder
it is to use. Not every mail user understands the difference between an smtp
server and a pop server, or even what the heck a server is in the first place.

Comment 5

[Wolf Eichler]

<.. and provide rudamentary (possibly now broken) support for multiple
identities per account.>

So you concede there is some "hang-through" in the present state?
There is some confusion around about the concepts of "accounts" and
"identities". I possibly can offer some thoughts on that later on.

Now, to improve my analysis, I admit I don't understand everything. So I don't
understand why somebody should arrive at the following concept:

START CITE (from doc "Mail: Account Setup")

* Mail from multiple accounts can be sent out at one time using one SMTP server
(user/system does not have to connect to each server separately to send mail).

END CITE
 
If you see my "Analysis" in #1 I have listed mailbox variation cases with the
assumption that the user wishes to set up an account for each single mailbox
separately within one profile. Now ISP servers normally do not allow a user to
send mail with other "from" addresses than those registered at the ISP. How will
you send a mail from the account set up for

- host alpha.com -> user "Frankie" -> [e-mail]

when you are connected in a SMTP session to server beta.com, as of account

- host beta.com  -> user "justme" -> [e-mail]

? How is that possible?

The following cites seem at first hand to contradict the first claim.

START CITE (from doc "Mail: Account Setup")
* In order to use ONE specified SMTP server for ALL outgoing mail (from
different identities), the user must be connected to the Internet via the
account paired with the specified SMTP server.

* When a user launches Mail, and has multiple accounts set up, the user will see
a login (name, password) dialog for each account (as necessary).
END CITE

Why do I need a (name, password) for each account when a single SMTP session
suffices to send all mail? (name, password) I only need to set up a SMTP session
(or a POP session).

So either I'm missing a technical innovation here or pointing to a conceptual
flaw. Some misfortunate behaviour of the mail client seems to indicate the latter.

-------------

A final question:

START CITE ("Using the Multiple Accounts API"):

The only thing that determines which SMTP server a user should use is their
physical connection to the internet. For instance, if a user is connected to the
internet through MyISP, Inc. then he or she must use MyISP's SMTP server, no
matter what identity they will be sending mail with.

END CITE

Is that feature in that very sense already/still active in Mozilla?

------------

- Wolf

Comment 6

[Ben Bucksch (:BenB)]

Wolf,
> There is some confusion around about the concepts of "accounts" and
> "identities".

If you are confused about it, you need to ask specific questions (*after*
reading the doc I cited).

I don't understand your usage of
. "mailbox" (which you seem to equate with email addresses),
- your host/username pairs (li-2), which you use for both incoming servers
  (POP/IMAP) and SMTP (although they are usually different hosts).
- "Internal lists". You shouldn't use user input to identify
  anything internally. This is, in *any* app, almost never a good thing to do.
  Mozilla currently uses a serial to identify these records internally.
- email addresses during download ("PD."). Currently, the mail from one
  incoming server is put into the corresponding INBOX. If the user wants to
  sort it after email addresses, he can do that via filters.

> Now ISP servers normally do not allow a user to
> send mail with other "from" addresses than those registered at the ISP.

This assumption is wrong. Most German ISPs' SMTP servers just look at the IP
address, and if it's from the own pool, they accept almost anything. This is
just right IMO.

> The only thing that determines which SMTP server a user should use is their
> physical connection to the internet.

Wrong. Not only is it almost impossible to determine the Internet connection
reliably, the user might also want to use different SMTP servers over the same
connection. One might be preferred, but blacklisted by the broken MAPS DUL,
another might protect the privacy more or add a PGP signature or whatever.

I think the current scheme in Mozilla (even that exposed in the UI) meets most
requirements: You have a default SMTP server (changeable). Identities default to
that server. You can, however, specify any amount of SMTP servers and
"hard-link" a specifiy identify to a certain SMTP server.

> Why do I need a (name, password) for each account

You don't. Excuse the question, but have you actually *used* Mozilla Mailnews?

Comment 7

[Ben Bucksch (:BenB)]

> > Why do I need a (name, password) for each account
> You don't. Excuse the question, but have you actually *used* Mozilla Mailnews?

I retract that, since I am not completely sure, what you mean with "account" here.

All in all, after trying to understand your proposal, I am not sure what it
would add to what is implemented in Mozilla's backend. The latter seems much
more simpler and more flexible to me.

Comment 8

[Wolf Eichler]

Ben, the following message is to answer and possibly clarify some points of your
reply. I will make a retreat this weekend and cannot do more til next week. My
feeling is that both my analysis and the Mozilla account conception need some
more work to do.


>> The only thing that determines which SMTP server a user should use is their
>> physical connection to the internet.
>
>
>Wrong. Not only is it almost impossible to determine the Internet connection
>reliably, the user might also want to use different SMTP servers over the same
>connection. One might be preferred, but blacklisted by the broken MAPS DUL,
>another might protect the privacy more or add a PGP signature or whatever.

Please note that you are referring to a statement from the mozilla.org document
mailnews/arch/accountmanager.html. If you find that cite peculiar along with me,
that's ok. From what I see implemented already,  me thinks this is an outdated
piece of text there.


>I don't understand your usage of
>. "mailbox" (which you seem to equate with email addresses),

I introduced the term "mailbox" as designating a single email-address (and its
used up spaces, if you like), what is difficult with it? This is common practice
in some of the RFCs.


>- your host/username pairs (li-2), which you use for both incoming servers
>(POP/IMAP) and SMTP (although they are usually different hosts).

Yes you are right. It was a lapsus not to differentiate between POP and SMTP
host-names. So please in PB. and PC. of the draft read "<hostname>" as
"<pop-hostname>". This is a more trivial thing here, because POP-hosts are just
a member of accounts (in my proposal).


>- "Internal lists". You shouldn't use user input to identify
>anything internally. This is, in *any* app, almost never a good thing to do.
>Mozilla currently uses a serial to identify these records internally.

As much as possible, yes. But at times new input (incoming mail e.g.) must get
related to existing internal objects and so objects have to be identified by
some semantic marker/pattern. Thus it was meant.


>- email addresses during download ("PD."). Currently, the mail from one
>incoming server is put into the corresponding INBOX. If the user wants to
>sort it after email addresses, he can do that via filters.

Have not thought of that yet. Viable, but much extra work for the user who has
to promote to "advanced" first. I would see a solution best fit which
distributes incoming mail to mail-end-points, which in my understanding are the
correspondants of mailboxes of my first definition: the email-addresses.


>> Now ISP servers normally do not allow a user to
>> send mail with other "from" addresses than those registered at the ISP.
>
>
>This assumption is wrong. Most German ISPs' SMTP servers just look at the IP
>address, and if it's from the own pool, they accept almost anything. This is
>just right IMO.

Doubt if the latter is right; would be hazardous to rely on "most" in a mail
collecting algorithm.

But something else holds true: shame on my head! I have thrown away the
illegible RFC 2821 and re-read RFC 821. Before I went from the assumption that
mail relaying is not possible in client-host connections, but this must be
wrong. Obviousely message relaying is the basis for the present one-for-all
accounts conception. But I'm not sure if that really works for all mailbox
variation cases as in my draft. At least at present Mozilla reports a host
problem during send which I have added as an attachment (Errormessage-001.jpg).
This is from a setting where I have variation cases b) and c) set up, smtp all
set to default and at least one mail of three in "unsent messages" is
cross-account .


>I think the current scheme in Mozilla (even that exposed in the UI) meets most
>requirements: You have a default SMTP server (changeable). Identities default
>to that server. You can, however, specify any amount of SMTP servers and
>"hard-link" a specifiy identify to a certain SMTP server.

I now know more about Mozilla's backend concepts than I did when writing my
proposal. In fact I allow you to call it "premature". It was a way to plunge
into the topic, out of reasons that were not trivial: my mail sending from any
set up account other than the first one didn't work at all. I would agree that
the present concept is not bad - but not yet "best practice" - if only it would
function right. If I understand right, an error message as the attatched may not
occur under the presently followed conception.

>In fact, I don't see what your proposal would improve even over Mozilla's
>current *frontend*. Enhancements as of my proposals read:

i)  sending of messages works for a multitude of accounts, comprising all
variation cases a), b) and c)

ii)  distribution of incoming mail goes automatically into identified
mail-addresses/mail-boxes (accounts)

iii)  don't bother the user with smpt hostname list (if not otherwise required);
put a smtp host field into "server settings", not into "identity"

iv)  if a smpt list is necessary, make it legible; you have to add
"host-username" to servername in order to distinguish sufficiently for a
selectable connection. At present you can setup a differentiated list for case
b) but they all show with the same entry name (hostname).

v)  make an "unsent messages" for each account (you could dispute whether this
is an enhancement)



- Wolf

Comment 9

[Wolf Eichler]

Attachment: An error message from Moz-Mailer build 20020117

Related to comment #8
Should it show at all?

Comment 10

[Wolf Eichler]

I may add here to the enhancements:


vi)  reduced privacy risk: relayed mail keeps a return path that identifies the
first sending mailbox. The name of this mailbox possibly shows up in the plain
text version of the mail available at the recipient. The user is not aware of
that, but his true name might be available in postings he wishes to be anonymous
or masked.

For instance you have a mail address "[e-mail]";
this by a chance determines your default SMTP server in Mozilla. Now you also
have a mail address "[e-mail]" that you give as sender in the composer.
The receiver will find 

return-path: <[e-mail]>

in the mail header, if he switches to full display.

Comment 11

[Ben Bucksch (:BenB)]

> Please note that you are referring to a statement from the mozilla.org
> document mailnews/arch/accountmanager.html.

Oh, sorry! I thought, you were referring to one of your own proposals!

> If you find that cite peculiar along with me,
> that's ok. From what I see implemented already,  me thinks this is an outdated
> piece of text there.

Alecf or whoever is responsible for
<http://www.mozilla.org/mailnews/arch/accountmanager.html> now, can you please
update/correct the doc?


Your usage of "Account" seems to completely mismatch that of Mozilla's current
implementation. An Account in Mozilla consists of "identities" (email address in
outgoing mails, signature etc.) plus exactly one "Incoming server"
(POP/IMAP/... server). I am not critizing that (it's just a term), but it
might clear up some confusion.

> I introduced the term "mailbox" as designating a single email-address (and
> its used up spaces, if you like), what is difficult with it? This is common
> practice in some of the RFCs.

From what I remember from RFC822, a mailbox is a message store, e.g. a POP
account. This is different from an email address:
- several email addresses might map to one mailbox (e.g.
  the "catch-all"/default domain rules)
- an email address might forward to another arbiatry address.
- an email address might expand to a mailing list
etc.

> would be hazardous to rely on "most" in a mail collecting algorithm.

s/collecting/sending/

We don't *rely* on it, do we?

> At least at present Mozilla reports a host
> problem during send which I have added as an attachment 

Just assign a certaim SMTP "server" (hostname/username pair) to that identity.
Acount manager|main account pane|Advanced.

(Note that this case is indeed uncommon.)

> i)  sending of messages works for a multitude of accounts, comprising all
> variation cases a), b) and c)

The SMTP setup (in contrast to mutiple identites per account) is quite flelxible
in Mozilla's current frontend already. If you work more with it, you might find
out that it covers (almost?) all cases already.

> ii)  distribution of incoming mail goes automatically into identified
> mail-addresses/mail-boxes (accounts)

I think that just getting mail from one POP server and throwing all that mail
into one certain folder is the most straight-forward (and thus easiest to
understand for users) way and that it will work for most cases. (Although some
people prefer having only one global inbox.) Doing some matching on the To
addresses will only confuse users. Those few users who really need it can still
do it with filters.

> iii)  don't bother the user with smpt hostname list (if not otherwise
> required); put a smtp host field into "server settings", not into "identity"

Maybe "Server setting" is a miswording in Mozilla's current UI - it refers to
Incoming servers. I do think that (*especially* in your case) an SMTP server has
more to do with an identity than an incoming server. Identites are about sending
mail, incoming servers are about getting mail.

> you have to add "host-username" to servername in order to distinguish
> sufficiently for a selectable connection. At present you can setup a
> differentiated list for case
> b) but they all show with the same entry name (hostname).

You shouod be able to give "SMTP servers" (in Mozilla's setup) "nicknames", yes.
(Currently, we use only hostnames to represent "SMTP servers" in the UI, which
is a problem, if 2 "SMTP servers" are only different by the user name on the server.

You should file a new bug about that. "Allow user to give SMTP servers nicknames"

> v)  make an "unsent messages" for each account (you could dispute whether
> this is an enhancement)

That might be an improvement as well, yes.

(But no need to overthrow all of Mozilla's account structure for that either.)

> vi)  reduced privacy risk

I can't follow you here. I selected an arbitary identity in Mozilla's composer,
sent it to one of my mailboxes and read the message source. The identity I
selected is used as "return-path" (top line of msg source).
In any case, what you describe sounds like a bug, not a structural problem.


So, if you proposal doesn't add that much over the current Mozilla frontend, and
it substracts some thing, can we mark this WONTFIX?

Comment 12

[Alec Flett]

ok, this is rediculous. The mail team is not going to go uproot the entire
account management software just for a few small gains for a small group of
power users.

(That said, Virtual folders or combining multiple accounts into one INBOX is
still workable with the given architecture, with a few tweaks of course, and
does not require rewriting the account management interfaces)

Basically this is a value vs. cost. Perhaps if we were writing a new mail client
from the ground up, we might be considering these issues but there are a whole
lot more interesting things that people should be spending their time on. 

I'm removing myself from the CC, and recommending that someone mark this WONTFIX.

Comment 13

[Ben Bucksch (:BenB)]

doing so.

Comment 14

[Ninoschka Baca]

Verified Wontfix.