206679 - junk Mail filtered from an IMAP to Local or pop folder does not get analyzed when that folder is opened

Status: RESOLVED FIXED

(MailNews Core :: Backend, defect)

Description

[esther]

Using trunk build 20030521 on winxp, mac osx and linux, it you filter mail from
an IMAP folder to a POP or local folder, that mail will not get analyzed for
Junk Control when it's opened (the messages are still listed as new and unread).
 Note,  mail filtered from a POP account to a local folder or other pop folder
will get analyzed as that scenario is fixed in bug 180153. 

1. Launch mail and set a filter for an IMAP mail account to filter into a Local
Folder. Make sure JMC is set to ON for IMAP and Local Folders and whitelisting
against Address book is off.  
2. Send messages that would be be filtered (as set up in step 1) to the local
folder and would be analyzed as junk when that folder is opened. 
3. Get Msg for the IMAP account, messages should filter at this point.
4. Open the local folder

Result. messages don't get automatically analyzed as junk
Expected  messages are new to that folder and are unread, they should get
automatically analyzed when the folder is opened.

Comment 1

[esther]

nominating, this should work for our new JMC feature, it works for POP to POP. 

Comment 2

[Samir Gehani]

adt: nsbeta1-

Comment 3

[Spundun]

Still having this problem with TB 0.8 on macosx panther.

Comment 4

[Ingo Fischer]

problem still exists in TB 2.0.0.4 :-(

Comment 5

[David :Bienvenu]

I bet this is simply because the filtered messages in the local folder don't have the new flag set, so we don't try to analyze them for spam. I'm working on a possible fix (this will fix some other issues as well)

Comment 6

[David :Bienvenu]

Attachment: [checked in]proposed fix

sucks that this is so tricky but there you go...this will also fix the new mail alert and folder tooltips for these filtered messages, which is my primary motivation for fixing this. A fair number of people have these kinds of filters, it seems.

Comment 7

[Wayne Mery (:wsmwk)]

nice (tho i'm clueless about the code)

Comment 8

[Ingo Fischer]

(In reply to comment #6)
> Created an attachment (id=269284) [details]
> proposed fix

Can I use a nightly build or something to verify the fix ?!

Comment 9

[Ingo Fischer]

One question to the fix:

Will the junk be process when I enter teh folder or after the message-filters are done (maybe see bug 329569) ... ?

Comment 10

[David :Bienvenu]

you could use a nightly, once I check the change in. 

The junk should be processed after you open the folder...

Comment 11

[Ingo Fischer]

what about to fix it like described in the references bug 329569 where it was fixed for POP that it is done automatically after the message-filter are finished?

that would be the best!

Comment 12

[Ingo Fischer]

Hi !

I have downloaded and installed "version 2.0.0.5pre (20070624)" (a nightly). As I saw on http://forums.mozillazine.org/viewtopic.php?t=560012&sid=8d4ef106a52e21193411673c1261205e you have commited the fix.

Unfortunately I don't fixes the issues I have. The messages are (still) sorted correctly, but no junk-processing happends. Not after applying al filters and not when I open a folder :-(

Any idea?

Comment 13

[David :Bienvenu]

The fix isn't in 2.0.05 - it's in nightly trunk builds.

Comment 14

[David :Bienvenu]

Attachment: run junk filters on destination local folder

Comment 15

[Ingo Fischer]

(In reply to comment #13)
> The fix isn't in 2.0.05 - it's in nightly trunk builds.
> 

Can you commit it into 2.0.0.5 branch too ?! Would be great... else I have to wait till 3.0 ;(

Comment 16

[David :Bienvenu]

Comment on attachment 269284 [details] [diff] [review]
[checked in]proposed fix

this fix should improve the new msg experience for people who use filters like this.

Comment 17

[David :Bienvenu]

fixed on trunk, second patch landed.

Comment 18

[Ingo Fischer]

should the secod flag get the "approval 1.8.1.5"-flag too? ;-)

Comment 19

[David :Bienvenu]

Comment on attachment 269675 [details] [diff] [review]
run junk filters on destination local folder

Requesting approval. It would be nice if someone could try these fixes on the trunk build and verify that they address your issues.

Comment 20

[Scott MacGregor]

Comment on attachment 269675 [details] [diff] [review]
run junk filters on destination local folder

a=mscott for mailnews only change to 1.8.1.5.

Comment 21

[Ingo Fischer]

I will be on vacation this week, but when the patch is in 2.0.0.5 nightly I will be able to test and verify it ...

Comment 22

[David :Bienvenu]

both fixes checked into 1.8.1.5

Comment 23

[Ingo Fischer]

I have installed the latest nightly and will test it now. I seems to work ok ...

The "big" test will be tomorrow when I get all mails from over-night. After this I will know exactly if it works

Comment 24

[Ingo Fischer]

I have some first test-results: sometimes it works and sometimes not :-(

1.) I have configured the IMAP account in thunderbird to look for new messages every 5 minutes and download all new mails.
When thunderbird is running and gets new mails this way (every 5 minutes) your patch seems to work. The messages are sorted into the folders and when I enter the folder then junk-processing is done.
But I have to enter the folder - the junk-processing will not occur automatically after all messages filters have finished to run.

2.) When I start thunderbird completely new then the account in thunderbird is configured to look for new messages directly on startup.
In this scenario (when thunderbird gets the messages directly after the start) no junk-processing is done after I enter the folders :-(

Do you have any idea why?

Additionally it would really be great to have the junk-processing running directly after the message-filters were done as it is for POP3-mails.

Comment 25

[Ingo Fischer]

I think I have an idea for the problem:

It would be great to get your opinion about this:

I have configured my account to get new mails every 5 minutes.
When I get emails (unimportant if at startup or later) they are sorted into the folder, marked as "new mail" and averything is ok. When i get into these folders directly after the emails arrived there everything is ok!
Lets name this "cycle 1".

But when I do not enter the folders directly after the mails arrive and thunderbird gets new mails 5 minutes later ("cycle 2") it seems that the "new mails" status of the mails received during "cycle 1" is reset and only the mails that were received at "cycle 2" are marked as new.

This repeats with every new 5 minutes with more cycles were everytime the mails from the last cycle loose the "new mail" flag and only the mails from the latest cycle withh get this flag.

When I now enter the folder (where, when I understand everything correctly in this bug all mails having a "new" flag will be checked for junk) only the mails received during the latest cycle will be checked for junk and will be sorted out. All mails received in the cycles before will not be checked (because they don't have this new-mail-flag.

Is this possible?
It would make sense for anything I have seen during my tests.

When the junk-processing runs automatically after the message-filters have finished (as it is for pop3) this problem would not happend because the junk-processing would run after any receiving-cycle.

Ingo F

Comment 26

[Ingo Fischer]

PS: maybe comparable fixes as in bug 180153 are possible for this too ?!

Comment 27

[Ingo Fischer]

I have read again the details of bug 180153 and in comment #23 in this bug a problem is described which is the same for IMAP!
When messages are filtered to the current displayed folder then they are not scanned for junk when you leave and re-enter the folder ... maybe it would be possible for fixing this for IMAP too ...

And this "run junk processing directly after message filters" where done for POP in bug 329569 ...

Comment 28

[Stephen Donner [:stephend] Not actively reading bugmail]

I've tested this exactly as the scenario was originally written in comment 0 by Esther, and verified that it's FIXED on the 2.0.0.5 branch.

Ingo, please do file followup bugs for the spin-off issues, to keep each individual case easy to reproduce, fix, and verify.

Replacing fixed1.8.1.5 keyword with verified 1.8.1.5.