128944 - [RFE] Message filters should include standard rule "to+received"

Status: NEW

(MailNews Core :: Filters, enhancement)

Description

[André Langhorst]

b: 2002-03-02-08
os: all

Unfortunately I have some experience creating message filters and to almost
every message I blocked using "To" I added an "Received" block in addition.
For convenience "to+received" should be added to.

I guess this is high gain with low impact.

Comment 1

[Brant Gurganus]

[RFE] is deprecated in favor of severity: enhancement.  They have the same meaning.

Comment 2

[resigned]

*** Bug 190681 has been marked as a duplicate of this bug. ***

Comment 3

[resigned]

Suggest that this filter should be given a simpler name, for the benefit of all
those who do NOT understand email-headers, and are not aware of the relevance.

Perhaps call the filter "Destined for" or "Intended recipient" or similar?

c..f 190681

Comment 4

[Vaclav Dvorak]

I'm opposed to this. The method is very unreliable, and Mozilla would have to
show a very long warning explaining the perils of using it. See fetchmail's man
page for an explanation. If you really want this, use fetchmail.

I'm not marking WONTFIX to let other people say their opinion, but I'm strongly
in favor of WONTFIXing this.

Comment 5

[resigned]

Very kindly, you neither mention a particular version of fetchmail (so I can't
be sure I'm looking at the source you cite) nor do you bother to say which part
of the rather long man page you are referring to. This makes discussion
difficult - in fact, since I'm having to guess what you're talking about, I
don't see how your last comment helps at all?

Having found a part of the manpage which says something akin to "its hard
working out who the intended recipient is" I have only one conclusion: the
authors aren't up to date on email header RFC's - which include this rather
useful "Received" field, which is the subject of this RFE.

So, it would seem that you are suggesting that because the authors of a
different email program didn't know about the fields mentioned in this
particular RFE, we shouldn't do it? Um, I think that's what you're saying (but
as I said, we're having to guess here!).

Of course, there is the possibility that a mail MTA (note: this is independent
of clients, the Received field is generated by servers) doesn't use the Received
header at all - but then, it becomes impossible to actually deliver an email
ANYWHERE, AFAICS - and I've never seen an email that was successfully delivered
without it including a note of who on earth it was destined for!

As for your suggestion "if you really want this, use fetchmail", what on earth
are you on about? Perhaps you also believe "if you really want a web browser
that works, use lynx"? I really don't understand how your attitude relates to
attempting to IMPROVE Mozilla?

I'm genuinely confused and bemused by what you are saying. Please explain.

Comment 6

[Vaclav Dvorak]

Sorry, I guess I was indeed a bit too brief in my comment. Let me clarify.

> Very kindly, you neither mention a particular version of fetchmail (so I can't
> be sure I'm looking at the source you cite) nor do you bother to say which part
> of the rather long man page you are referring to.

The relevant section is "THE USE AND ABUSE OF MULTIDROP MAILBOXES" (their caps,
not mine) and it's present in any version that I have seen. You can read it
online at http://catb.org/~esr/fetchmail/fetchmail-man.html#25

> the authors aren't up to date on email header RFC's - which include this
> rather useful "Received" field, which is the subject of this RFE.

Now, it seems you are being even more vague than I was. Could you point to a
specific RFC that says that an email has to include the recipient's address,
whether in a "Received" header or elsewhere?

> Of course, there is the possibility that a mail MTA (note: this is independent
> of clients, the Received field is generated by servers) doesn't use the
> Received header at all - but then, it becomes impossible to actually deliver
> an email ANYWHERE, AFAICS - and I've never seen an email that was successfully
> delivered without it including a note of who on earth it was destined for!

This seems to be the source of the misunderstanding. The fact is, a mail _can_
be successfully delivered without it containing the recipient's address. In
fact, it is quite a common case. The authoritative information about an email's
destination is contained in its envelope, which is _not_ part of the email.
Instead, it is something that the various MUA's, MTA's and MDA's communicate to
each other with the SMTP protocol. In principle, this information is _lost_ once
the email reaches its destination mailbox. Some MTA's and MDA's may, under some
conditions, add the destination email address (or some modification of it) in
the email's headers, but that is just an option. The fetchmail man page explains
this in much more detail.

If you want an RFC, here's a quote from RFC 2821, section 4.4 (Trace Information):
> The FOR field MAY contain a list of <path> entries when multiple
> RCPT commands have been given.  This may raise some security
> issues and is usually not desirable; see section 7.2.
The key word here is MAY. Not MUST, and not even SHOULD.

> As for your suggestion "if you really want this, use fetchmail", what on earth
> are you on about? Perhaps you also believe "if you really want a web browser
> that works, use lynx"? I really don't understand how your attitude relates to
> attempting to IMPROVE Mozilla?

Oh, I'm all for improving Mozilla, by all means. I just happen to think that
including this feature would not be an improvement.

I can almost hear someone say: how can including a new feature NOT be an
improvement? Well, for me, there are two main reasons in this case:
1) This feature, while useful in some cases, has its pitfalls and must be used
with care and understanding. Exposing it in the UI would require some big red
warning, and/or it would generate many false bug reports and create a bad
impression of Mozilla.
2) Sorting mail to mailboxes is a function of an MDA (Mail Delivery Agent), not
an MUA (Mail User Agent). [You could argue that it's actually the MTA's (Mail
Transport Agent) job, and you may even be right, but that's irrelevant for this
discussion.] Mozilla is an MUA. The function, if it were to be implemented
properly, is not as trivial as searching one additional header for the email
address. There is also "X-Envelope-To" (which can have a different name), and
"Delivered-To" (in which the email address is probably mangled). While it is
conceivable that Mozilla would re-implement what Fetchmail already does well, I
think the job's best left to the specialized tool. Furthermore, sorting mail at
the time when you open the inbox is too late. You can forward the mail to the
other person, but then he is dependent on you opening your mail before he can
read it...

There, I hope I made myself clear this time.

Comment 7

[resigned]

Thanks for the explanation; I think I understand now.

Re-familiarising myself with RFC 822, and quoting relevant section (note, this
is obseleted by 2822, but I've checked and the result is the same, but 822 is a
bit clearer to read):

message     =  fields *( CRLF *text )       ; Everything after
                                            ;  first null line
                                            ;  is message body

fields      =    dates                      ; Creation time,
                 source                     ;  author id & one
               1*destination                ;  address required
                *optional-field             ;  others optional

source      = [  trace ]                    ; net traversals
                 originator                 ; original mail
              [  resent ]                   ; forwarded

trace       =    return                     ; path to sender
               1*received                   ; receipt tags

return      =  "Return-path" ":" route-addr ; return address

received    =  "Received"    ":"            ; one per relay
                  ["from" domain]           ; sending host
                  ["by"   domain]           ; receiving host
                  ["via"  atom]             ; physical path
                 *("with" atom)             ; link/mail protocol
                  ["id"   msg-id]           ; receiver msg id
                  ["for"  addr-spec]        ; initial form

It is clear that "Received" is a REQUIRED part of an OPTIONAL field: read the
full RFC for syntax details.

So, although I remembered the required aspect, I'd forgotten that the
higher-level field is an optional.

However, the suggested RFE is not in any danger of contravening the RFC; nor
will it "break" on emails wihtout the Received header. Laying aside for the
moment your concerns about roles of MTA/MUA/MDA, there is no damage that would
be caused by offering this filter. Clearly, there is a potential problem in that
it is not guaranteed to work!

Perhaps we could name it "Probable Intended Recipient"?

The problem here is that the functionality is currently completely absent for
all those who are NOT highly technical, who generally don't know what RFC stands
for, let alone read them. For me, personally, this isn't a problem (uexcept for
all the people I persuade to use mozilla who then complain that the filtering
doesn't work or does "weird things").

But it would, as the original reporter mentioned, provide "high gain with low
impact" for the majority of non-RFC readers. In fact, in the light of your
observations w.r.t other headers, they too could be included in the filter, perhaps?

Now, on to your concerns with MUA/MDA etc. I think the most important point is
"Mozilla mail is a MUA, NOT a MDA". This is perfectly valid, except that in a
way it IS an MDA (from my limited understanding). Since most users use mozilla
mail to collect mail from a remote mailbox, and bring it in locally, with
Mozilla fetching mail for multiple accounts simultaneously, and delivering to
(possibly) multiple local mailboxes (stored as files on the hard drive, IIRC),
filtering as it goes - sounds very much like an MDA to me? (even if not quite
what MDA really means, surely there is a big overlap in roles here?)

I think the problem is partly that the traditional separation of MDA/MUA has
been blurred, largely by the desire of people to have one tool that does all
(less hassle, easier to manage, and with microsoft its basically the only
way...)? It seems to me that the proposal would make lives easier, at the cost
of contributing (slightly) to the blurring of the MDA/MUA difference - which is
of precisely NO concern to the user whatsoever.

Although I appreciate what you're saying now about all else, I still don't see
why you are so afraid of this, to the extent of saying "we need a big red
warning"? No protocols are broken, no standards are broken. Mozilla mail's
behaviour is not altered. All that happens is that an additional (extremely
useful) feature which takes almost no programming effort to implement (in its
BASIC form - although your comments suggest a more complex form could be more
useful) might be added.

I too am uncomfortable about contravening the intentions for MUAs - but as I
said this doesn't seem (to me) to in any way negatively affect the rest of the
world of mail agents.

Also bear in mind that mozilla is meant to be a standalone program - if I have
to get other apps to make it work properly, what's the point in it having about
80% of the features it already has? (FTP download, bookmarks, tabbed browsing,
etc). In fact, why on earth does it have ANY filtering at all? I think that if
you are against this particular RFE, you HAVE to be against the existing
filtering on the same grounds. No?

Comment 8

[Vaclav Dvorak]

> It is clear that "Received" is a REQUIRED part of an OPTIONAL field: read the

It's actually the other way around: the "for" field is an optional part of a
required "Received" header. But I suppose that's what you meant.

> However, the suggested RFE is not in any danger of contravening the RFC;

I didn't say that it was. No argument here.

> nor will it "break" on emails wihtout the Received header. Laying aside for

Well, that depends on what you mean by "breaking"...

> the moment your concerns about roles of MTA/MUA/MDA, there is no damage that
> would be caused by offering this filter. Clearly, there is a potential problem
> in that it is not guaranteed to work!

That's the point. (Or, rather, the more important part of the point.)

> Perhaps we could name it "Probable Intended Recipient"?

Oh, I'd love to hear what the UI experts think about this! :-))) That's _very_
vague. Not even the experts would understand what it is supposed to mean, let
alone non-technical users.

> The problem here is that the functionality is currently completely absent for
> all those who are NOT highly technical, who generally don't know what RFC
> stands for, let alone read them.

And I say, good, let it stay that way!

About the MUA/MDA difference: let's ignore any philosophical views. What matters
is the practical consequences. If you use Fetchmail to do the multi-drop mailbox
sorting, it's not reliable because of the reasons we discussed above, but when
the header _is_ present, the result is as good as if you really had separate
mailboxes: the resulting mail is stored in separate mailboxes, which can be
accessed with POP3 or IMAP; the users can use any client they want to access
their mail in a standard way. You can even alias a username and let the mails be
resent elsewhere. BUT if you use Mozilla to do the same thing - even if it was
implemented as well as in Fetchmail - it's too late: when Mozilla filters the
mail, it has already been retrieved from the POP3 server. There is not much you
can do at this point: you could forward the mail to a different email address
(which isn't implemented, and which could take long if it was implemented), or
just move it to a folder. All the users of the mailbox would then have to use
the same copy of Mozilla, even the same profile. Mozilla is just not in the
_position_ to sort mail into mailboxes for different people.

About the "big red warning": you can ignore all the above, this is the most
crucial argument. The warning would have to be something like this:

You have selected filtering criteria that can sometimes be used to sort mail
from one mailbox for several people, based on message headers. While this kind
of filtering has its uses, you should be aware that it is not perfect. Some of
its problems are:
1) Some messages don't contain the information required for recipient
identification. These will not match the filter, so they will stay in the
default inbox.
2) Some messages contain recipient identification that is ambiguous. Such
messages may match the filter for one person, without any warning that other
people may have also been intended recipients.
3) Privacy breaches are unavoidable.
These problems are not caused by shortcomings of Mozilla: they are inherent in
the method. The only reliable way to avoid them is using an email routing setup
that provides a separate mailbox for each user. Contact your email provider for
more information.

There. Would you want such a warning in Mozilla? (The wording is not perfect, of
course, but I doubt that it can be made much shorter while staying useful.)

Comment 9

[(not reading, please use [e-mail] instead)]

mass re-assign.

Comment 10

[(not reading, please use [e-mail] instead)]

sorry for the spam.  making bugzilla reflect reality as I'm not working on these bugs.  filter on FOOBARCHEESE to remove these in bulk.

Comment 11

[Serge Gautherie (:sgautherie)]

Filter on "Nobody_NScomTLD_20080620"