180119 - When spam filter is checking spam status, can't grab new email

Status: VERIFIED FIXED

(MailNews Core :: Filters, defect)

Description

[Michael Baffoni]

I have spam filtering active.  When I connect to my IMAP (over TLS) account that
has several hundred messages to check, I can't read any messages until the spam
filter has finished running spam check agains the new messages in the mail
store.  When I click on a new message (while spam check is running), it opens a
new window that is completely blank.  Mozilla/5.0 (Windows; U; Windows NT 5.1;
en-US; rv:1.2b) Gecko/20021108

Comment 1

[Dan Mosedale (:dmosedale, :dmose)]

Taking... this makes sense, because only one connection to an IMAP folder at a
time is permitted by the back end code.  Serializing the classifications at the
message DB level (rather than allowing the IMAP code to do the serializations
later) seems like it might fix this.  David, do you think it would?  Although I
seem to recall someone saying that that would slow down the overall
classifications...

Comment 2

[David :Bienvenu]

this should be serialized, but I don't see how it would happen at the message db
level.

Comment 3

[Dan Mosedale (:dmosedale, :dmose)]

bienvenu: well, right now nsMsgDBFolder is passing a null callback to
ClassifyMessage.  Instead of iterating through all the messages in
CallFilterPlugins, it could conceivably start the first message, and have a
callback that chains the second message, which would have a callback...etc. 
Thoughts?

Comment 4

[David :Bienvenu]

you should ignore the nsMsgDBFolder call. nsImapMailFolder passes in itself as
the callback, as does nsMsgLocalMailFolder. What confused me was your reference
to the db. This has to do with the folder, not the db. Yes, the folder could
chain the uri's. However, we're going to need to change the spam filter plugin
anyway to get it to run the url in the right msg window, so that stop will work,
etc, so we should do both at once, since I believe the latter is going to
require changing the spam filter interface, and the spam filter itself.

Comment 5

[David :Bienvenu]

To summarize more of this discussion, Dan argues that we should put the chaining
in the core libmsg code because we might have multiple spam filter plugins and
having the plugins do the chaining will lead to code bloat. My response is that
we already have three places in the core libmsg code that invoke the spam filter
plugin, so those three places would have to do chaining. If we had four spam
filter plugins, then, yes, perhaps that would be a little bit of bloat. An
additional response, we would need to add state to nsMsgLocalMailFolder and
nsMsgImapMailFolder to handle the chaining (probably a pointer to an array of
msg keys to categorize, and the index of the current message), and this state
would only be used when categorizing messages, but we would pay the price for
this state in every single folder in memory.

There are a few other possibilities - add an interface to the msg service that
knows how to stream multiple messages to a stream listener, and have the
implementations of this interface handling the chaining, and checking for the
stop key. Or, similarly, add a utility class that does something similar that we
instantiate only when needed. Ultimately, we do need to go through the msg
service instead of just fetching the uri directly, so that we can set up the
msgWindow on the url, so we'll get progress and status, and enable the stop
button handling. We'll need to add a new method to nsIMsgMessageService so we
can add "header=filter" to the url.

Adding an interface to the msg service that knows how to stream multiple
messages to a stream listener would probably still require creating a helper
class to do the chaining.

Looking at the bayesian spam filter plugin again, I see that it goes through a
stream converter before tokenizing. I think this means we'll need to have a
special method in the msg service that knows it needs to run through a converter
when fetching the message. So either way, we're going to need to add an
interface to the msg service. So the question remaining is: do we want to do the
chaining in the msg service (or helper class), or in the filter plugin? I still
find the possibility of having four filter plugins rather remote, so I'm going
to think about this a little more.

Comment 6

[David :Bienvenu]

I fixed this a while ago, reassinging.

Comment 7

[David :Bienvenu]

fixed

Comment 8

[esther]

Using build 20030414 on winxp, macosx and linux it a large folder is being
evaluated by Junk Mail Control, I can read a mail message while the JMC is
running. I can also stop the JMC using the Stop button.  Verified