Open Bug 105169 Opened 23 years ago Updated 2 years ago

Filter for Attachments

Categories

(MailNews Core :: Filters, enhancement)

Product:
MailNews Core
Component:
Filters
Type:
enhancement

Tracking

(Not tracked)

People

(Reporter: Lee.Jnk, Unassigned)

References

(Depends on 1 open bug, Blocks 1 open bug)

Details

(Whiteboard: [patchlove])

Attachments

(1 file, 1 obsolete file)

Enable filtering for POP3 and IMAP
2.28 KB, patch
Bienvenu
: review-
Details | Diff | Splinter Review 
It would be nice if I could filter my messages which have Attachments, to a folder.
i.e.
In new message filter I could select 
if "Message has Attachment" then "Delete the Message"

It would better still if I could select like
if "Message has .wav Attachment" then "Move to Folder 'Sound'"or
if "Message has .vbs Attachment" then "Move to Folder 'Virus'"
Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: [Feature] Filter for Attachments → RFE: Filter for Attachments
Target Milestone: --- → Future
Adding my $.02:

This is possible right now (Build 2001121003) by customizing filter "look-ins"
to include "Attachment." I just did it and it worked. But, putting "Attachment"
as one of the standard places to look, along with "To," "Sender," etc. would
perhaps make it easier to set an important filter like this, and maybe even help
suggest to users that this is possible. I just now set a filter so that messages
with .scr or .vbs attachments go directly to trash. I wish they could get
immediately expunged, but I am NOT going to put in an RFE for that. If I had any
friends left in IT support, I would show this capability to them and perhaps
they would decide to ditch Outlook for Mozilla. |-) <-- me dreaming!
*** Bug 149407 has been marked as a duplicate of this bug. ***
What is the status of this bug ?.
Now that 1.0 is out,
Which release approximately is 'Future'.
1.3 ?
The status "Future" basically means that the assigned engineer has no plans to
fix this soon, but as always, anyone can submit a patch.
This should include an option to save the attachment to a specified directory,
most likely different from where the message is saved.

Eudora has this capabilty, but I had to write a VB program to get Outlook to do it.
*** Bug 181418 has been marked as a duplicate of this bug. ***
mass re-assign.
Assignee: naving → sspitzer
Is there any progress on this RFE?

All that would need to be done is add a message header of
"X-Mozilla-Attachments" (just guessing at what it would be called) that would
contain a comma-separated list of the attachments to the email.  Let users
figure out how to filter .wavs, .vbss, .exes, etc. on their own, but give them
the tool to do so.
John Fredlund said:
'This is possible right now ... by customizing filter "look-ins" to include
"Attachment."'

Is "Attachment" the standard header for attachments? I've seen "X-Atteched" used
as well.

Prog.



*** Bug 219319 has been marked as a duplicate of this bug. ***
Re: comment 9:  There is no standard Attached or X-Attached header, that I am 
aware of.  One thing that does work to detect the presence of attachments is to 
filter on a header of 'Content-Type' containing 'multipart' (or on 
'multipart/mixed' if you don't want to filter combination plain/HTML emails sent 
as 'multipart/alternative').  You add the Content-Type header from the 
'Customize' option in the first dropdown in the rule editor: 
Message Filters | New (or Edit)

Determining the type of attachment is not feasible because that info is not part 
of the header block.

Adding an xref to bug 175384, rfe for filter on size of attachments.
What would be much more useful would be the ability to filter not just by 
attachment type, but by any criteria. So mail from certain senders or certain 
domains could be sent directly to related directories, while mail with certain 
headers could be sent to their related directories.
Summary: RFE: Filter for Attachments → Filter for Attachments
There is also a security consideration under UNIX/Linux. Resource files often
start with a dot. As an example, if an attachment is saved in the users home dir
as  .profile then the contents of the attachment will be interpreted every time
a new shell is opened. It would be a good idea if files starting with a dot
could be filtered out.

Additionally there is a problem with permissions on files. If a file already
exists and has the executable bit set, then a file saved on top of it will also
have the executable bit set. Any file saved should have it's access permissions
set to those defined by the user's umask default. This should be true even if
the file is saved to a file name which already exists and does not already have
the default permissions.
*** Bug 233862 has been marked as a duplicate of this bug. ***
I tried the suggestion made in Comment #1 to "customiz[e] filter "look-ins"
to include "Attachment."" and it did not work. That is, I used "customize" to
add an item "Attachment", then added a condition to my supplemental "to_junk"
filter (for recognizable virus-carrying messages the Junk filter doesn't
identify), when [Attachment contains .scr], but it does not mark the message as
junk when run. Am I missing something, or does this no longer work? If it
doesn't, then it needs to be restored. I want to filter out all messages
containing attachments *.exe, *.com, *.scr, *.pif, etc.
This is a very useful bug, considering the amount of worms on the net today. 

Whoever implements this should make sure to look at the message body rather than
the headers to spot the files, since viruse messages often omit the header
declaring the attachments they contain.
I simply can't believe this isn't in the Mozilla mail client at the moment.
Filtering based on the type of attachment (best example is nuking .PIF, .EXE,
.SCR, etc for common virii) is to me as commonly needed as filtering by sender
or subject. Other mail clients have this, please can we have it in Mozilla!
Whiteboard: Please no useless comment. Read http://bugzilla.mozilla.org/page.cgi?id=etiquette.html
It would seem that this problem could be solved by setting up a kind of mini
mail server through which incoming (and perhaps outgoing from virus) email could
be filtered, borrowing code from things like milter or procmail to serve as
virus filters, with provisions for using commercial virus filters written for
Windows to be made plugins for it.

Is anyone working on this. Virus-carrying and -complaining messages are costing
me more than an hour a day of personal time. I can't edit and expand filters on
Subjet, Sender, etc., fast enough to keep up, in the absence of the ability to
filter on attachments using wildcards.
After further research I have found that what seems to be needed is a convenient
way to integrate Mozilla, at least on Linux/Unix machines, to tools like Clamav
(for virus filtering), spamassassin (for spam filtering), and milter, and for
those of us who run local client machines that access a remote ISP's mail
server,  a quick way to set up to locally use fetchmail, sendmail, procmail, and
anything else needed for a complete solution. See
http://www.constitution.org/comp/linux.htm for links to these and related tools.
At this point I haven't figured out how to connect Mozilla to fetchmail for the
first step, and I propose that Mozilla account setup at least offer that option
as part of a setup wizard or list of options, which should ultimately enable the
novice user to set up those tools with little more difficulty than now attends
setting up email retrieval directly from an ISP's POP server. The wizard should
download, install, and configure all the above utilities to receive updates of
the virus db, have spamassassin learn to recognize spam from clicking the junk
button, edit the conf files, and leave the overall operation of Mozilla mail
appearing to work the same way, except that the spam and viruses no longer
appear in the Inbox (and hopefully also no more of the warning messages that
"you sent a virus-infected message" when you did not -- standard prepended
strings like "[infected mail]" or "[returned mail]" that could be filter-moved
to the Junk or Bounce folder would help a lot for that).
Just something simple would be fine.  Filter on the name of attachments,
including wildcards.  So I can filter out anything with "*.pif" or "Important.*"
 Of course size of attachment, and "has attachment" and "does not have
attachment"  also.  This would take care of at least 80% of the the virus email
I get these days.

Of course it needs to be smart enough to be able to filter out attachments that
dont really show up as attachments, as in the following message snippet.

----------------------------------------------
Translated message has been attached.<br>

Or you can view the message at:<br><br>

<a href=3Dcid:121401Mfdab4$3f3dL780$75387018@57W81fa70Re height=3D0
width=3D0>www.mywebiste.com/inmail/webmaster/mread.php?sessionid-24321</a>
<iframe
src=3Dcid:121401Mfdab4$3f3dL780$75387018@57W81fa70Re height=3D0 width=3D0></iframe> 
<DIV>&nbsp;</DIV></BODY></HTML>

------=_NextPart_001_001C_01C0CA81.7B015D10--

------=_NextPart_000_001B_01C0CA81.7B015D10
Content-Type: audio/x-wav;
	name="message.pif"
Content-Transfer-Encoding: base64
Content-ID:<121401Mfdab4$3f3dL780$75387018@57W81fa70Re>

TVqQAAMAAAAEAAAA//8AALgAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
-----------------------------------------

Note that this only has the "name=" tag and not the additional "filename=" tag
that is usually there.  Obvious this is a virus also.
While we are considering this one, I suggest we also get a filter on character
encoding. Much spam and some viruses come in messages with a foreign langage
code, such as Chinese. It would be helpful for English readers who don't want to
receive foreign language messages to be able to filter them out. Trying to train
the spam filter on them doesn't work, only results in Bayesian poisoning. But
perhaps this should be a separate bug/rfe.
*** Bug 249525 has been marked as a duplicate of this bug. ***
I'm trying to write this for Thunderbird which is my client of choice (but my
Mozilla coding experience is low).

First problem I came up against is that I think I need to add another option
called 'Attachment Filename' to
mozilla/mailnews/base/search/public/nsMsgSearchCore.idl

but that has an enum 'nsMsgSearchAttrib' that seems to be 'full' - nothing can
be added after 49 according to the comments and all earlier options are taken. 
Anybody familiar or could suggest who I can collaborate with on coding this?
Blocks: 66425
I have coded a patch for this, but for Thunderbird, where a similar request was
made.

The patch meets the original request (filter on attachment filename only), but
not the further requests added in additional comments.

For the patch to Thunderbird 0.7.2, see
See http://bugzilla.mozilla.org/show_bug.cgi?id=224392
 
A corollary to this request would be the ability to filter on messages with an
SMIME attachment; in the UI it might be presented simply as "Message is/isn't
cryptographically signed"
Flags: blocking-aviary1.0mac?
Flags: blocking-aviary1.0?
Flags: blocking-aviary1.0mac?
Flags: blocking-aviary1.0mac-
Flags: blocking-aviary1.0?
Flags: blocking-aviary1.0-
*** Bug 225602 has been marked as a duplicate of this bug. ***
Product: MailNews → Core
*** Bug 271520 has been marked as a duplicate of this bug. ***
Using the existing filters, one could filter on 
a) whether the message has attachments 
b) what 'kind' of attachments they are

for a)
body...contains...Content-Type: multipart/mixed;
( I know this will catch plaintext messages that have a html part as well )

for b)  ( as an example jpg files )
body...contains...Content-Type: image/jpeg

also b) ( this time exe files )
body...contains...Content-Type: application/x-msdownload

Good luck
I have added "Has Attachments" filter criteria in my patch to bug #196036.
Depends on: 196036
I think this is a dup of bug #83969

*** This bug has been marked as a duplicate of 83969 ***
Status: NEW → RESOLVED
Closed: 19 years ago
Resolution: --- → DUPLICATE
It seems to me that bug #83969 is different from this one - this requests a
filter on attachments, and #83969 asks for the ability to search them manually.

While both may be able to be implemented by the same bit of core-level code,
they aim to achieve two different sets of functionality and clearly require
different UI implementations. 
(In reply to comment #32)
> It seems to me that bug #83969 is different from this one - this requests a
> filter on attachments, and #83969 asks for the ability to search them manually.
> 
> While both may be able to be implemented by the same bit of core-level code,
> they aim to achieve two different sets of functionality and clearly require
> different UI implementations. 

Good point. OK, I'm unmarking the DUP.
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
Since the fix for bug #241203 makes this info available, we can use it in POP3
filters now.
It actually works for IMAP too, so enabling that as well.
Assignee: sspitzer → hyc
Attachment #188287 - Attachment is obsolete: true
Status: REOPENED → ASSIGNED
No longer depends on: 196036
Attachment #188289 - Flags: review?(bienvenu)
*** Bug 321295 has been marked as a duplicate of this bug. ***
Comment on attachment 188289 [details] [diff] [review]
Enable filtering for POP3 and IMAP

Again, the has attachment flag is only a guess at filter execution time, based on the content type of the message. We correct it when a message is displayed, if we've gotten it wrong. But it would be a bit dangerous to allow filters on a flag which is just an approximation when filters run on incoming messages. Thoughts?
(In reply to comment #37)
> (From update of attachment 188289 [details] [diff] [review] [edit])
> Again, the has attachment flag is only a guess at filter execution time, based
> on the content type of the message. We correct it when a message is displayed,
> if we've gotten it wrong. But it would be a bit dangerous to allow filters on a
> flag which is just an approximation when filters run on incoming messages.
> Thoughts?
> 
I'm not sure about "dangerous" but I suppose it can be less than 100% reliable. What is the danger from false positives, and what is the danger from false negatives? Since there is no such feature at all today, I would say false negatives is a moot point. I'll note that I've been running with this patch in my own builds since July, and there have been a few false negatives but not false positives.
(In reply to comment #37)
> Again, the has attachment flag is only a guess at filter execution time, based
> on the content type of the message. We correct it when a message is displayed,

Would it be possible to make the flag reliable/more than a guess? 

If not, maybe do a further examination of the matched messages. Just being able to trust the flag would be preferred of course...
Blocks: 83969
you pretty much have to parse the message body to know for sure...still, this might be better than nothing, I'm not sure.
Search improvements -> cc Bryan.
Flags: wanted-thunderbird3?
QA Contact: laurel → filters
Target Milestone: Future → ---
Product: Core → MailNews Core
Depends on: 224392
I'd like to propose a compromise here.

First, the issue is real that the attachment status available to the filter code is unreliable. For example, I have junk emails sent with Content-Type: multipart/mixed (but the content is a base64 encoded message.) The attachment icon appears in the thread pane for the message, until I select it. Then the message is parsed, mime figures out that there really is no attachment, and the attachment icon goes away in the thread pane. That's just one example, there may be others.

So the simple approach of just enabling the hasAttachmentStatus search attribute for junk will not give reliable results. Yet we put up with this already in the thread pane display. The user should be able to decide for themselves for filtering whether the reliability is adequate for their purposes - as long as they have some reason to be a little cautious.

My suggestion is that the "reason to be a little cautious" is to disable this by default, but allow it to be enabled through a hidden preference. Then an extension (such as my FiltaQuilla) could provide the UI to enable it. That then dumps the reliability problem onto the extension, which I think is the appropriate place for it rather than in the main program.

I'm going to grab the assignment to follow that approach, unless there are objections.
Assignee: hyc → kent
Sorry for the bugspam, but in looking at this again, including the use cases in the comments, it looks like what people are really after is something that could filter on the names of the attachments. So I am going to instead propose an alternate approach.

In the Bayesian filter code, we are already running downloaded messages though libmime, and getting information about attachments. But we are currently throwing that away. So I think what I need to do is to add a method to persist the attachment names that comes from that functionality. Then we could actually do some filtering of attachment names post-analysis. In bug 198100 I already intend to add the capability to run filters post-analysis, so the extension filtering capability desired here could then be done then.

So I will abandon the simpler approach, and instead pursue that direction. There's some interesting interaction though with whitelisting. A message that is currently whitelisted bypasses the expensive download and tokenization process, which also is where I would get the attachment names.
Whiteboard: Please no useless comment. Read http://bugzilla.mozilla.org/page.cgi?id=etiquette.html
It's not just filtering on the name we need, but on other attributes of interest, such as the attachment being executable code, the type of character encoding (which would also be useful for the main message and subject line), little HTML spots that can conceal spyware, and so forth. In other words, all the attributes of malware and spam that might make it past the malware and spam filters operating on the servers. It is unlikely that users can anticipate the names of attachments, but they can anticipate that they don't want anything that will execute without giving the user the chance to quarantine them and give them careful treatment.
Well filtering on name is (dependent) bug 224392.
(In reply to comment #45)
> Well filtering on name is (dependent) bug 224392.

Thanks for pointing that out, Magnus - I guess I need to look over the full range of bugs in the "filter/search by attachment" cloud.
> It's not just filtering on the name we need, but on other attributes of
interest, such as ...

Don't forget file size. (And maybe unzipped file size for .zip/.bz2/.gz/.7z/...)
Although I appreciate the problems of viruses in files with extensions to their names, for those operating systems that use file name extensions. My concerns, raised more than 5 years ago, for UNIX/Linux, of not saving a file with a dot at the start of the name, and saving a file without the executable bit set should be much easier to implement (as it is based on two simple rules) and although it does not fix all the security problems highlighted here, it does fix a security problem for UNIX/Linux systems.
Here's an update on this bug.

There was a thread "running new messages through MIME immediately" in m.d.a.thunderbird beginning 2009-06-19 that discussed the backend work that would be needed to fully implement this and related bugs. In summary, in order to prevent multiple code segments from repeating the same mime processing, we would need to agree on how mime results will be available to a variety of different consumers if we try to do it upfront. That is clearly not going to happen in the TB 3.0 timeframe. I'm also not sure I am the right person to do that, so I am unassigning myself from this for now.

However, bug 198100 unintentionally snuck in the ability to specify Attachment Status during an incoming filter in the post classification context, since it uses the same validity tables as after-the-fact filters. This has the dangers discussed here in comment 37, but does provide a partially working attachment status filter. That may or may not be a good thing.
Assignee: kent → nobody
Status: ASSIGNED → NEW
Attachment #188289 - Flags: review?(bienvenu) → review-
Comment on attachment 188289 [details] [diff] [review]
Enable filtering for POP3 and IMAP

minusing - I think we need a more accurate solution, per the discussion in the bug.
See Also: → 224183, 224392, 83969
I think wanted-thunderbird3='?' is nonsensical today.
Flags: wanted-thunderbird3?
Summary: Filter for Attachments → Filter for Attachments (by attachment type/extension/content-type/mime type)
Well, actually do we need this bug anymore?
Filtering by the existence of attachment is bug 83969.
Filtering by attachment name/extension is bug 224392 (has also some patch).

Can this be closed? Or is this now some general "all attachment properties" bug?
Summary: Filter for Attachments (by attachment type/extension/content-type/mime type) → Filter for Attachments
Have the security problems for UNIX/Linux type systems been fixed?
(In reply to :aceman from comment #52)
> Well, actually do we need this bug anymore?
> Filtering by the existence of attachment is bug 83969.
That's SM

> Filtering by attachment name/extension is bug 224392 (has also some patch).

should the patches join forces?
Whiteboard: [patchlove]
(In reply to Wayne Mery (:wsmwk) from comment #54)
> (In reply to :aceman from comment #52)
> > Well, actually do we need this bug anymore?
> > Filtering by the existence of attachment is bug 83969.
> That's SM
I'd say the backend for this is the same for TB and SM (see the attached patch here). Also bug 83969 is SM only since comment 15. Also the only difference I see is that this one is for Filters and bug 83969 is for Search. But again, the backend is mostly the same between these components. But we can keep them both if there really are any differing UI tweaks needed.

> > Filtering by attachment name/extension is bug 224392 (has also some patch).
> 
> should the patches join forces?
It seems they are different.

It seems to me this bug and bug 83969 both mention the lack of searching for attachments online (IMAP/News) as it already is implemented for local searches/filters.
(In reply to :aceman from comment #55)
> (In reply to Wayne Mery (:wsmwk) from comment #54)
> > (In reply to :aceman from comment #52)
> > > Well, actually do we need this bug anymore?
> > > Filtering by the existence of attachment is bug 83969.
> > That's SM
> I'd say the backend for this is the same for TB and SM (see the attached
> patch here). Also bug 83969 is SM only since comment 15. Also the only
> difference I see is that this one is for Filters and bug 83969 is for
> Search. But again, the backend is mostly the same between these components.
> But we can keep them both if there really are any differing UI tweaks needed.

Thanks. My meaning was only that the bug was FILED as SM - I don't know that there is a SM-only fix.  Feel free to do the dup or set dependency as you see it.
Have the security problems for UNIX/Linux type systems been fixed? Can an attachment still be saved by default to a an existing file name which has an execution bit set? Can an attachment with a dot as the first character in the name still be saved by default with a dot as the first character? It is now more than 10 years since I first mentioned this security hole, and there has not been one reply in this thread to my questions.
HI, please file it in a separate bug and we can look at it. Your concerns get lost in this unrelated bug. Thanks.
I think the comment at the start of this thread "It would be nice if I could filter my messages which have Attachments, to a folder." should cover UNIX as well as DOS -- so it is related. However I will post a separate bug report, as this issue is/has not being addressed here.
@Philip Baird Shearer: This bug is not about moving attachments to a folder, but about the ability to move e-mails to a folder based on the condition weather they have an attachment or not.
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: