1975717 - Implement message filters for EWS

Status: ASSIGNED

(MailNews Core :: Networking: Exchange, task, P2)

Description

[Brendan Abolivier [:babolivier]]

We want to eventually support client-side message filters for EWS accounts. Currently, the state of message filters makes this fairly non-trivial, as every protocol pretty much maintains their own implementation. There might be some things we can reuse from nsMsgFilter.cpp but overall it sounds like we might need to reinvent a big chunk of the wheel.

This is probably a good opportunity to look into how we could write a protocol-agnostic implementation of message filters, first for use with EWS, but with the goal of using it for existing protocols in the future too. I think this is a good time to kick off the conversation around what this might look like.

Comment 1

[Ben Campbell]

Attachment: Bug 1975717 - Apply filtering and spam classification to incoming messages on EWS. r=#thunderbird-back-end-reviewers

Just a first baby step, but seems to work.
You can set up filters and when a matching message arrives it'll print "BING!" on stdout.
I think the next thing to try is applying the filter after the message has been added to the database.
Once the message is in the db, applying actions is much simpler and more reliable.
POP3 and IMAP filter handling jumps through awful hoops to move incoming messages to other folders.
They implement their own bespoke message-move routines. Badly.
So it'd be nice to avoid that.

Comment 2

[Ben Campbell]

Updated my patch. It now applies filters to newly-received messages at the end of a sync operation.
Seems to work OK...
Caveats:

• I'm just passing a null window to the filter system, so any error messages probably won't pop up.
• EWS doesn't download the full message during sync, so filters which match on the body won't work yet.
• Not quite about junk classification yet. More investigation required...
• For simplicity, we add the new messages to the folder, then applying filtering.
  This means that for "move" filter actions, messages appear briefly in the folder GUI before being moved out...

Comment 3

[Ben Campbell]

Attachment: Bug 1975717 - Move some overly-verbose does-this-filter-need--the-message-body checks out of IMAP folder. r=#thunderbird-back-end-reviewers

Comment 4

[Ben Campbell]

Attachment: Bug 1975717 - Disallow null filters in nsIMsgFilterList. r=#thunderbird-back-end-reviewers

Comment 5

[Ben Campbell]

Attachment: Bug 1975717 - Move filterlist needs-message-body code out of IMAP folder. r=#thunderbird-back-end-reviewers

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/71c3d475249c
Move some overly-verbose does-this-filter-need--the-message-body checks out of IMAP folder. r=darktrojan
https://hg.mozilla.org/comm-central/rev/c66c59dfb921
Disallow null filters in nsIMsgFilterList. r=darktrojan
https://hg.mozilla.org/comm-central/rev/825d6b9c0b19
Move filterlist needs-message-body code out of IMAP folder. r=darktrojan

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/ac8e5b5b460a
Apply filtering and spam classification to incoming messages on EWS. r=edicharry

Comment 8

[Ben Campbell]

Attachment: Bug 1975717 - Add a basic test for filtering incoming messages on EWS. r=#thunderbird-back-end-reviewers

Comment 9

[Takanori MATSUURA]

Since bug 1981208 blocks bug 1847846, there is no need to set both in "Blocks."

Comment 10

[Brendan Abolivier [:babolivier]]

(In reply to Takanori MATSUURA from comment #9)

Since bug 1981208 blocks bug 1847846, there is no need to set both in "Blocks."

There is, because bug 1847846 allows us to get a better grasp on the outstanding Exchange work needed as a whole (more easily than with the full dependency graph), whereas bug 1981208 gives us a filtered-down version of the current phase (and also mostly exists to help with integration into external tools).

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/8af6c84c7272
Add a basic test for filtering incoming messages on EWS. r=babolivier

Comment 12

[Ben Campbell]

If any filters require the message body, we need to download the message first, which is happening over in Bug 1986399.

Comment 13

[Ben Campbell]

Attachment: Bug 1975717 - Apply full-body filters to EWS messages upon download. r=#thunderbird-back-end-reviewers

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/e6cb1e4a67da
Apply full-body filters to EWS messages upon download. r=babolivier

Comment 15

[Ben Campbell]

Attachment: Bug 1975717 - Remove unused param from IEwsFolder.handleDownloadedMessages(). r=#thunderbird-back-end-reviewers

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/e994959257d2
Remove unused param from IEwsFolder.handleDownloadedMessages(). r=mkmelin