257415 - Add filtering for IMAP folders other than the INBOX

Status: RESOLVED FIXED

(MailNews Core :: Filters, enhancement)

Description

[Miguel]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040803
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040803

Filters don't work for IMAP subfolders and when you go to some folder and select
Tools > Run filters on Folder, this doesn't work bug #253368. So there's a major
loss of function with filters for IMAP accounts.

Reproducible: Always
Steps to Reproduce:

Comment 1

[Miguel]

Maybe I didn't explain it very well. The bug would be that Mozilla doesn't
filter IMAP subfolders.

So this bug and bug #253368 produces a major loss of function for the message
filters for IMAP account, it's similar to not having filters for IMAP.

Comment 2

[Mike Cowperthwaite]

This bug is basically about having Mozilla's filters run automatically on mail 
that's arrived in the IMAP folder due to server-side filtering?  That would be 
an enhancement.

Comment 3

[Miguel]

Yes, since that is new mail and according to the option "Check this folder for
new messages", filtering the mail should also be done for those messages.

Should I change the severity to Enhancement? I truly thought new mail had to be
filtered no matter which folder it is. Are you sure it isn't a bug? :-)

Comment 4

[Miguel]

Glubs, I changed the severity without waiting you confirmation it's the desired
behavior and not a bug

Comment 5

[Marc Elser]

(In reply to comment #4)
> Glubs, I changed the severity without waiting you confirmation it's the desired
> behavior and not a bug
You cannot be serious. How can the behaviour of not beeing able to run mail
filters in imap subfolders be desired? In my opinion this is either a lack of
functionality or a bug. It don't see any advantage of not beeing able to do
this! For example my virus alert mails are automatically incoming in a "virus
alert" mail folder and all suspected spam mails in the "spam" folder, this is
done by the mail server and I certainyl want to be able to run filters on those
mails too and not only on Inbox.

Comment 6

[David :Bienvenu]

It's definitely an enhancement request, and it's a feature that would be off by
default - in the reporter's world, there are server-side filters, so you might
want to run filters on folders other than the inbox. But for most users, there
are no server-side filters, so you run your filters on new mail in the inbox,
and not on headers that are new in other folders, since the filters have already
seen all incoming mail. 

I also changed the summary - I guess the reporter uses a server where all other
folders are sub-folders of the INBOX, but that's only true for a couple imap
servers (Courier and Cyrus-derivitives) and is not true in general.

Comment 7

[Miguel]

(In reply to comment #6)
> I also changed the summary - I guess the reporter uses a server where all other
> folders are sub-folders of the INBOX, but that's only true for a couple imap
> servers (Courier and Cyrus-derivitives) and is not true in general.

Yes, I use FastMail as IMAP provider...

Comment 8

[Howard Chu]

*** Bug 281300 has been marked as a duplicate of this bug. ***

Comment 9

[Howard Chu]

This is a special case of the more general problem I described in bug 294632.

Comment 10

[Eric Hattemer]

This is incredibly important for those with server-side filters who want to automatically label messages.  There are no server-side filters for creating thunderbird labels.  I go through several different folders and hit "run filters on folder" several times a day because for some reason I can't set it to do this automatically.  

Comment 11

[(not reading, please use [e-mail] instead)]

sorry for the spam.  making bugzilla reflect reality as I'm not working on these bugs.  filter on FOOBARCHEESE to remove these in bulk.

Comment 12

[Magnus Melin [:mkmelin] (away, back Aug 4)]

I'm duping this to bug 294632 - i can't see this being implemented without doing the general approach in that bug.

Comment 13

[Kent James (:rkent)]

(In reply to comment #12)
> I'm duping this to bug 294632 - i can't see this being implemented without
> doing the general approach in that bug.
> 
I'm not sure I agree with the assessment. It seems to me that all that is needed is to move the ApplyFilterToHdr call outside of the if statement here in nsImapMailFolder::NormalEndHeaderParseStream:

  if (mFlags & nsMsgFolderFlags::Inbox)
  {
      ...
      rv = m_msgParser->GetAllHeaders(&headers, &headersSize);

      if (NS_SUCCEEDED(rv) && headers && !m_msgMovedByFilter)
      {
        if (m_filterList)
        {
          GetMoveCoalescer();  // not sure why we're doing this here.
          m_filterList->ApplyFiltersToHdr(nsMsgFilterType::InboxRule, newMsgHdr, this, mDatabase, headers, headersSize, this, msgWindow, nsnull);
        }
      }
    }
  }
  // here we need to tweak flags from uid state..
  if (mDatabase && (!m_msgMovedByFilter || ShowDeletedMessages()))
  {
    mDatabase->AddNewHdrToDB(newMsgHdr, PR_TRUE);

This could be set as a binary preference, disabled by default.

Comment 14

[Magnus Melin [:mkmelin] (away, back Aug 4)]

Reopening. Though I'd much rather see the bug 294632 implemented. The same filters are maybe not likely to be relevant for the non-inboxes as the server side filters has done the first sorting to begin with.

Comment 16

[Kent James (:rkent)]

I'm going to take this for now, proposing that initially all that I do is to add an inherited folder property that, if set, will force the server-level IMAP filters to be applied to a folder, regardless of whether it is an inbox folder. I won't add any UI for now, so the option would either need to be set in an extension or though advanced editing of a preference.

I suppose that a variation of this would be to also add an inherited property that enables or disables a filter by name. That would allow any specific filter to be applied (or disabled) fairly precisely to any folder, at least from the backend perspective.

Comment 17

[Kent James (:rkent)]

Attachment: Sometimes works but mostly crashes

This patch allows an extension to add additional folders that incoming filters apply to. However, about 90% of the time it crashes, deep in javascript. It is the checking of the string property that is the most important driver for whether the crash occurs, not the filtering - but the crash is occurring later in the code. I've tried a variety of things, none of them work. At this point, I am going to upload the patch, and maybe try this again in a few weeks, hoping it is a temporary javascript bug that will get solved by others. The crash is a fatal assertion in a debug build, involving STOBJ_GET_CLASS, so perhaps it is related to bug 490768.

Comment 18

[Kent James (:rkent)]

Attachment: Minor correction.

The crashing I experienced earlier is now recognized as a javascript problem and is being investigated elsewhere. Also, the crashes mysteriously stopped for some reason. This patch corrects a small typo in the previous patch.

Anyway, this patch allows an extension to add filtering of non-inbox folders.

Comment 19

[David :Bienvenu]

Comment on attachment 385042 [details] [diff] [review]
Minor correction.

This comment doesn't read well - Perhaps the second sentence could be something like "Or if the inherited folder property "appplyIncomingFilters" is set"

+  // If this is the inbox, try to apply filters. We use an inherited folder
+  // property "applyIncomingFilters" to allow override of the default, which is
+  // that only the inbox gets filtered.
+  if (mFlags & nsMsgFolderFlags::Inbox || m_applyIncomingFilters)


can you move this with the packed bools?

+  // apply filters to this folder, even if not the inbox
+  PRBool m_applyIncomingFilters;

r/sr=me, with those fixed.

Comment 20

[Kent James (:rkent)]

Attachment: [checked in] Fixed nits, patch to checkin

Carrying over reviews, patch to checkin.

Please do not close this bug. The patch provides backend support, but any front end would need to be provided by an extension (or by setting a hidden preference). The bug is still valid as a request for FE support for this. I expect to provide a simple UI for this in my FiltaQuilla extension though.

Comment 21

[Mark Banner (:standard8)]

Comment on attachment 386461 [details] [diff] [review]
[checked in] Fixed nits, patch to checkin

Checked in: http://hg.mozilla.org/comm-central/rev/921049691081

Comment 22

[Kent James (:rkent)]

I'm reversing my comment 20 about resolution. I will include some UI for this in my FiltaQuilla extension. I doubt if it is worth including that in the core, given that this is a rather specialized application of filters.

So I am going to resolve this as FIXED. If anyone feels that UI should exist in the core, feel free to file a new bug.