Closed Bug 196732 Opened 21 years ago Closed 13 years ago

Some messages correctly identified as junk are not moved to the Junk Mail folder

Categories

(MailNews Core :: Filters, defect)

Product:
MailNews Core
Component:
Filters
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: erice, Assigned: emaijala+moz)

References

(Blocks 1 open bug)

Details

(Whiteboard: [see comment 60] [also mozilla-1.9.1])

Attachments

(6 files)

IMAP log: after reading a valid new message, a junk message is not moved
182.22 KB, text/plain
Details
IMAP log: part 2 of 2
149.26 KB, text/plain
Details
The junk email that was not moved. See log file, above.
1.39 KB, text/plain
Details
another imap logfile, zipped. No. of IMAP connections now set to 4.
27.12 KB, application/zip
Details
remote image in junk mail is shown
105.61 KB, image/jpeg
Details
Possible fix - checked in
2.57 KB, patch
Bienvenu
: review+
mscott
: superreview+
Details | Diff | Splinter Review 
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3b) Gecko/20030210
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3b) Gecko/20030210

On an average day I might receive about 100 pieces of junk mail.  ~90 are
immediately moved to the "Junk" folder, which rocks.  Another 5 are pesky
all-image junk mail that slip through.  All fine.  Here is the problem:

About 5 a day are correctly marked as junk but remain in my inBox.  


Reproducible: Sometimes

Steps to Reproduce:
1. Have an email account that a bunch on spammers have gotten a hold of.
2. Turn on junk mail filtering (with move to folder)
3. Watch for the occaisional message in your InBox with the junk mail bit set.

Actual Results:  
I saw a few Junk messages

Expected Results:  
moved them to the junk mail folder.

Junk mail filtering is saving me tons of time.  Thank You!!!!!

I am using POP agianst one main account that received 99.99% of the email and
another that receives a message or two a month.

I haven't bee able to discern any real pattern, but it might have to do with
manually poking the "GetMsgs" button shortly before Mozilla automatically checks
for new messages.
On my Mandrake 8.2 system with Moz 1.3b, it *never* moves marked junk mail to
the junk folder.  All junk ends up in the inbox and most of it gets tagged as
junk, but it simply refuses to move it automatically.  I have to run the filters
manually to make it go away.
Mozilla 1.3b, Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.3b)
Gecko/20030210, Windows XP Pro, Build 2600.xpsp1.020828-1920

Confirmed in Win XP. Junk mail control correctly sends many spams to junk
folder, and on some, it simply marks them as junk, but leaves them in the inbox.
Also, selecting Tools>Run Junk Mail Controls on Selected Messages... does
nothing to the emails marked as junk (but left in the inbox), even if I
"re-mark" them manually as junk and have highlighted them.

If this isn't a duplicate of bug 196213, then someone should mark Hardware/OS ->
All/All
Status: UNCONFIRMED → NEW
Ever confirmed: true
As per my comments in bug 196213, this now WFM...
*** Bug 197542 has been marked as a duplicate of this bug. ***
I have never had any junk mail messages actually moved from my Inbox to the junk
folder automatically, even though I configured the junk mail controls as such. 
The messages are identified as junk correctly, but just not moved.  I have to
manually move them or run "Delete Mail Marked as Junk in Folder" from the Tools
menu to deal with junk mail.

This problem has not occurred since I downloaded 1.4a; however, several other
people have complained about it on other platforms.  

Should I mark this "resolved"?
I'm using 1.4a and still am having this problem.  When retrieving from my POP
account, all of the mail goes into my inbox.  After all the mail arrives, the
filter then correctly marks all junk mail.  However, it does not move it even
though I have that option specifically selected in the junk mail controls.
mass re-assign.
Assignee: naving → sspitzer
Reporter says this bug works.

Michael Sherman, have you upgraded to 1.4 Final?  (It's possible your 1.4a was 
from an earlier build than Eric's.)  
Is the bug still a problem for you?  One possible cause may be the name of the 
junk folder name contains a space (see bug 194061).
See a similar report in bug 196706.
I occasionally see this in new Thunderbird 0.1 installations as the training is 
just beginning. Should I open a new bug?

One way I've found to repro this is to:

1) get new messages (hopefully when more than a few messages are waiting)
2) stop the transfer before all messages are downloaded
3) highlight the inbox
4) click Tools>Run Junk Mail Controls on folder

Results: only one of the new junk messages will be marked junk, and it will be 
left in the inbox...the other junk messages won't even be marked. Maybe that 
should be opened as a separate bug(?).
I've recently started seeing more of these cases. I'm using UW-IMAP server.
Often the received message is correctly classified but not moved to junk until
it's either clicked on the header pane or Get Msgs has been click (sometimes
multiple times).
Now I have two correctly classified junk mails sitting in my inbox no matter
what I do so even the tricks don't work all the time. This happened upon Mozilla
startup. 
The reason this bug happens is Mozilla is compressing Junk folder at the time it
is supposed to move spams into it. 

The way to fix it is probably to make the spam filter wait until Junk folder
compression is finished.
Now that you mentioned it, I saw "Compacting folder" in the status bar when it
happened.
Workaround: Use "Tools -> Run Junk Mail Controls on Folder".

Stupid workaround: Switch off the automatic folder compacting (they will grow ad
infinitum unless you compact them manually)
Ere, aren't you talking about IMAP, and Jacek, aren't you talking about
pop3/local folders?
Yes, I'm talking about a POP3 server. I see this problem every morning unless I
remember to compact the folders manually the evening before.
And I'm talking about IMAP and there's nothing to compact in IMAP, right? So
please pardon my dust, it was probably another POP3 account that it was compacting.
This started happening to me some time ago, and happens systematically: all
messages marked as junk remain in the Inbox folder. I've tried de-activating the
option to move them to the Junk folder and then reactivating it. I've also tried
telling Mozilla to move them to another folder instead of Junk. No matter what I
do, the junk messages always remain in the Inbox folder.

I'm not using automatic compaction. So, no folder compaction is taking place
when Mozilla gets the messages from the POP3 server and classifies them as junk.

I'm now using Mozilla 1.5, but this behavior started happening before, with
version 1.5a, and remained with versions 1.5b and 1.5.
Possibly related to bug 168648?
not sure this is the right bug to write this in, but maybe it is. If not, pls
reply and I'll open a new one:

every once in a while a message automarked as "junk" is moved to the trash
instead of the junk. I don't have any mail rules that move stuff to junk (well,
not anymore) - so either this is a bug with the junk mechanism or the message rules.
Uri, have you checked "Automatically delete junk messages older than x days.."?
It uses the message date which might just be wrong.
(In reply to comment #23)
> Uri, have you checked "Automatically delete junk messages older than x days.."?
> It uses the message date which might just be wrong.

No it doesn't. It uses the date when the message was read. At least on POP3.
Either it has recently changed, or it works differently with IMAP.
Jacek might be thinking of the "delete msgs more than XX days old from pop3
server" feature, which uses the local machine date when the msg was downloaded
from the pop3 server. Ere is correct that junk mail purging uses a date search
to find old messages, where the date is the date stored in the msg.
David: You're right.
So far this appears to be fixed in 1.7rc2, at least with IMAP.

Alan
Sorry, I meant rc1 - there is no rc2 yet. 

I just had the problem again, unfortunately.

Further info:
- Mozilla 1.7rc1
- Windows 98
- Multiple accounts on the same IMAP server
- A few message filters in use
- My IMAP server reports that it is "Courier-IMAP", "Copyright 1998-2002 Double
Precision, Inc".

Should help identify when this bug happens. Under 1.7RC1, Win98. Part 1 of 2.
Attached file IMAP log: part 2 of 2 — 
part 2 of 2
I suspect in this case the problem is that your server dropped the connection
when we tried to append the message to the junk folder...

0[781d50]: creating protocol instance to play queued
url:imap://bugzilla@jynxy.net:143/customKeywords>UID>.INBOX>3849>Junk>
0[781d50]: playing queued
url:imap://bugzilla@jynxy.net:143/customKeywords>UID>.INBOX>3849>Junk>
-3504295[2d92ce0]: 2d91c70:jynxy.net:S-INBOX:ProcessCurrentURL: entering
-3504295[2d92ce0]:
2d91c70:jynxy.net:S-INBOX:ProcessCurrentURL:imap://bugzilla@jynxy.net:143/customKeywords%3EUID%3E.INBOX%3E3849%3EJunk%3E:
 = currentUrl
-3556831[302f940]: ReadNextLine [stream=302f5f8 nb=0 needmore=1]
-3556831[302f940]: ReadNextLine [stream=302f5f8 nb=0 needmore=0]
-3556831[302f940]: 302e980:mail.jynxy.net:NA:CreateNewLineFromSocket: clearing
IMAP_CONNECTION_IS_OPEN - rv = 80470002
-3556831[302f940]: 302e980:mail.jynxy.net:NA:TellThreadToDie: close socket
connection
-3556831[302f940]: 302e980:mail.jynxy.net:NA:CreateNewLineFromSocket: (null)
-3556831[302f940]: 302e980:mail.jynxy.net:NA:ProcessCurrentURL: aborting queued urls

Have you set the number of connections that we'll cache to this server to 4?  By
default, Courier servers only allow 4 connections, and by default, Mozilla tries
to cache 5. I would have thought you'd see other strange things happen if you're
running into this limit, but the symptom of having a new connection attempt
rejected often indicates this problem.

I suspect that the number of cacheable connections is set to the default of 5.
There is a usability question there if the default value upsets a common IMAP
server. But I will try playing with that number again.

When this problem occurs, I believe that I typically see the following
additional problems thereafter:
- inability to move the junk mail manually to the junk folder (which is on a
separate imap account). I am sure I have seen a bug report on this (not specific
to junk mail) but can't find it. I have this problem often - a message that I
try to move appears to move (perhaps an action is queued or cached somehow?),
but does not actually move. All moves between folders then show the same problem
while the status bar displays "Connected to myserver.net...". The problem
persists until I restart Mozilla.
- bug 196095 or something like it - failure to copy to sent folder.

Alan
I experienced this bug frequently with version 0.5.  (So far I haven't seen it 
in version 0.6.)  If I then manually marked a different message as junk, all 
the junk mail would be moved, but a list of blank messages equal to the number 
of junk messages moved would appear at the bottom of the mailbox.  The problem 
went away when I went clicked on a different IMAP folder and then went back to 
the Inbox.  I regret not testing it more thoroughly when it occurred, but part 
of it might just be a screen refresh problem.


Well, blow me down! Now that you mention it, I've had LOADS of apparently empty
emails consisting only of half-formed headers lately! I had never thought to try
and correlate the headers with junk messages that were being moved (or not).
What an interesting theory! Again, this is with 1.7RC1 on Win98.
Do you enable both automatic mail download and automatic junk purge(automaticaly
delete Junk older than X days)?
If yes, Junk folder busy may be cause of failure in automatic move to Junk folder.

Since "Junk Mail Purge" is kicked each "N" minutes after 5 minutes(not user
modifiable) later from start up of Mozilla Mail&News, 
where N is value of "check for new message every N minutes",
N=5 may cause contention of "Junk" folder on every automatic download.
Specifying prime number greater than 5 will reduce probability of this Junk
folder contention.

See Bug 228512 for "after 5 minute" of Junk mail purge.	
See attachment of Bug 242744 for NSPR log for Junk Mail Purge.
As per a previous post, I have reduced the number of permitted connections from
5 to 4 for all my accounts. The problem still occurs - IMAP log attached
(zipped this time). All I do in this log is start Mozilla, wait for junk
filtering (which marks but does not move), and quit.

> Do you enable both automatic mail download and automatic junk
> purge(automaticaly delete Junk older than X days)? If yes, Junk
> folder busy may be cause of failure in automatic move to Junk folder.

I use "check for new messages at startup" and "check for new messages every 10
minutes". I have my inbox marked for offline use. I can't see a setting called
"automatic mail download". I do not automatically delete any junk mail, it's
moved to a separate imap account which happens to be on the same server.

> Since "Junk Mail Purge" is kicked each "N" minutes after 5
> minutes(not user modifiable) later from start up of Mozilla
> Mail&News, where N is value of "check for new message every N
> minutes", N=5 may cause contention of "Junk" folder on every
> automatic download. Specifying prime number greater than 5 will
> reduce probability of this Junk folder contention.

This problem often occurs when I start Mozilla. Thus the proposed solution will
not help, unfortunately.

Thank you all so far...

Alan
(In reply to comment #38)

Alan: Sorry for my unclear word "Automatic maill download".
I meant "check for new messages at startup" and/or "check for new messages every
10 minutes".
Your case seems to be server connection loss during Junk move, as David
Bienvenu's Comment #33 says.
(Both Inbox folder read failure and Junk folder write failure during Junk move)
My comment is to clarify problem in other than your case.

Questions on cases other than Alan's case , durig Junk folder compaction case.
(Q1) If IMAP, same situation as Comment #33 ?
(Q2) What Junk log says about unmoved Junk mails and some mails moved
successfully just before & just after the unmoved mails.
(Q3) Manual "re-mark" on the unmoved Junk mails still does not work?
     I do not know manual junk move(including "re-mark") damage of POP3
     on other than ;
     (a) Whole Junk Control is damaged by mail server definition inconsistency
         made by other softwares(eg. anti-virus) or by manual prefs.js editing.
         Note: In this case, automatic Junk Move also does not work any more.
     (b) Trash folder is busy due to hang in other trash folder acess by bug.
     My Mozilla moves mails to Junk even from Trash if I "re-mark" them :-)
(Q4) Contention of Junk has relation to this problem?
     If contention on Junk folder is one of the causes, other prtocol logs
     such as MsgPurge:5(for Junk Purge) in addition to IMAP:5(or POP3:5)
     may help problem analysis.
     Sorry but I don't know about NSPR log parameter for Junk Move or Filter.
     (Maybe Junk log only for Junk Filtering)
    This is meaningless if cause is in reading from server,
    Inbox folder related problem.
on a pop connection, it seems the processing does not work when downloading down
lots of (100+) messages, but does trigger when only a few.
I'm also wondering that could the messages be processed one-by-one as they
arrive (instead of waiting for all of them downloaded and batch processed)?
One-by-one processing would eliminate this issue.
At the momement, messages are not classified by the junk mail plugin when they
arrive, but rather when you open the folder. See bug 

http://bugzilla.mozilla.org/show_bug.cgi?id=200788

[Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8a2) Gecko/20040714] (release) (W98SE)

I'm using POP3 only, and I have started seeing this bug for +/- a week.
(Or did v1.8a2 caused this for me ? I don't remember so.)

The 'Tools > Run Junk Mail Controls on Folder' workaround moves the Junk
messages to Junk...

I believe that it's "profile" related, because all was working well, and now all
of my currently installed releases are affected:

[Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8a1) Gecko/20040520] (release) (W98SE)
[Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7.1) Gecko/20040707] (release) (W98SE)

I may try to figure it out...
(In reply to comment #43)
> [Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8a2) Gecko/20040714] (release)
(W98SE)
> 
> I may try to figure it out...

Well, it seems that I had some luck in my testing:
first try, first success ;-)

What did I do ?
I renamed <junklog.html> in my profile, to let MailNews create a new one on next
start/run ... and, yeah, my messages are moving to junk again !

Data:
*File size: 311.294
*Dates: Create 2003.08.17; Modified: 2004.07.24
*First & last log: 04/14/2003 & 2004-07-21

No error/whatever in JS.C. or "DOS" console.

I'll try some more tests ... but it takes time, as I don't now how to generate a
Junk mail myself...

Could someone try and test this trick ?
Please, keep your original file around, as I do: in case we should upload it at
some point.
And comment on whether it works or not for you.
Serge, how big was your old junk.html?
(In reply to comment #45)
> Serge, how big was your old junk.html?

These were the figures of my old <junklog.html>;
The new/recreated one is 1 Kb now.

Bad news: I had only renamed the old file ...
and I guess the folder compact feature reduced it down to 0 Kb :-(
So much for its content. (which seemed just right at the time)
After removing the junk.html (and disabling the logging too), messages are
regularly delivered into the Junk folder (it seems regardless of the POP batch
size).
I can see the same behavior with 1.8a4, build ID 2004092716 on Windows XP.
Sometimes, messages marked as junk are not correctly moved into the junk folder.
The problem is that in those cases you have to click on it and move it manually,
thus you have to open the message - and this can give the sender the idea that
his spam is read (at least when a remote image is opened).
if the messages are junk, clicking on them won't load the images.

I know of one way to get new mail not to be classified, but it's probably
different from what you're seeing:

1. Get new mail in your inbox (either at startup or by pressing get new mail
with the inbox selected).

2. Select another folder while the download is going on.

That will clear the new flag on the new messages in the inbox, and we won't
analyze them for junk status. It might be related to what you're seeing.
re: comment #49 : This is not true for my build. I'll create an attachment from
an image in a mail marked as junk, located in the junk mail folder. This image
is a remote image. Here's the source code:

--Boundary_(ID_Xv5rL9uiDYGo/ao0JFMFRw)
Content-type: text/html; charset=iso-8859-1
Content-transfer-encoding: 8BIT


<html>
[...]

But this seems to be OT for this bug?
<img src="http://www.pacificnano.com/images/article/nanop1.jpg" width="185"
height="186" vspace="5" hspace="5">

    
    

    
  
in the junk mail controls, do you have it set to sanitize html messages marked
by junk? I believe that's what prevents the display of remote images in the junk
mail.

    
    

    
  
I don't see such an option in my Junk Mail controls. Would be nice to have it
though. The only option to disable downloads from remote images is in the
Preferences -> Privacy -> Images; but that's for all emails, not only for junk
mails.

    
    

    
  
ah, that could be a tbird only ui right now.

    
    

    
  
(In reply to comment #48)
> The problem is that in those cases you have to click on it and move it
> manually, thus you have to open the message

You can avoid this by
1) turning on View|Message Body As|Simple (or Plain), OR
2) closing the message preview window (F8 hotkey), OR
3) right-clicking and using the Move submenu.


(In reply to comment #54)
> ah, that could be a tbird only ui right now.

Yes, it is -- bug 179568 has a patch for the trunk, but it hasn't been checked 
in.

    
    

    
  
reply/additional remark to comment #49:

I  have a self-created folder on the IMAP server where I have emails selected
via the filter rules. When I open this folder, the messages get downloaded and
are then sent through the junk filter. (Actually, I would prefer if the junk
filter would be applied before the regular, self-defined filter rules.) There
were three emails left that were not marked at Junk and therefore left unread in
this folder. I manually marked them one after the other as junk, and 2 of them
got automatically moved into the junk filter. One of them, however, is left in
the orginal folder, though marked as junk. I have the suspician that the reason
might be that I marked it as junk right at the time when another message was
moved into the junk folder.
Product: MailNews → Core

    
    

    
  
It's not immediately clear where this bug hass got to, as various other problems
have been mingled in with it, I believe.

FWIW though, I'm seeing the problem on T'bird 0.9 on Windows XP. Some relevant
settings:

1) POP3 server
2) Not entirely sure about folder compaction - I know I've seen the setting
somewhere, but can't find it now
3) Junk mail uses adaptive filter and white lists, and moves incoming junk mail
into the Junk folder
4) When I manually mark messages as junk, move them to the hunk folder
5) When displaying HTML messages marked as junk, sanitize the HTML

Using Tools | Run Junk Mail Controls On Folder works fine when it's got into a
dodgy state, but the next time I get some mail, the mail filter still leaves new
junk in my inbox.

    
    

    
  
(In reply to comment #34)
Alan Hart, I think your case(IMAP,connection already lost when tried to move) is
different/specific/unique problem, and you'd better to open new
bug(Product=Core,Component=Mailnews: Networking or Filterr), with sumaary of
both "IMAP" and "connection loss".



    
    

    
  
I'm seeing the same problem in Thunderbird 1.0. When I first start the program,
incoming junk mail is automatically moved to the Junk folder (as per my
settings). However after some time (say half a day), incoming junk mail is no
longer automatically moved and I must move it manually. If I close the program
and re-open the problem is gone.


    
    

    
  
The original reporter of this bug, and also comment 2's reporter, stated that 
the problem went away with 1.4, which is unsurprising; a lot of work occurred 
between 1.3 and 1.4.

However, this bug has several reports of the same problem in later builds.

Possible causes raised in this bug:
 - If the Junk folder is being compacted (as can happen automatically at 
startup), or possibly if junk mail is being auto-expired (deleted after N days), 
then the junk filters may occasionally not be able to move messages to the Junk 
folder; that's basically the same problem as bug 168648.  Note: the compaction 
issue is limited to junk folders for POP or Local Folder, but an IMAP account 
can specify a junk folder in local storage as well.

 - If the Junk Filter Log grows too large, then the messages will not be moved; 
this is bug 200594.

 - comment 30 (Alan Hart) raised an issue about IMAP problems, which was 
initially suspected to be a problem with too many connections attempted to the 
server; but the problem persisted after the connection limit was reduced, and a 
new protocol log provided for that (attachment 147995 [details]); no analysis of that log 
has been mentioned here.  David Bienvenu, could you take a look at that?


The other comments reporting this problem are:

 - comment 12 (Ere Maijala): the problem occurred for IMAP on an unspecified 
build, with unspecified frequency.

 - comment 20 (Luis Almeida): the problem occurs consistently in Moz 1.5 for 
POP3 -- could be the filter-log-too-large problem.

 - comment 48 & 56 (mg8@gmx.de): the problem is occasional on IMAP with 
Moz 1.8a4 (is your junk folder remote or local?)

 - comment 57 (Jon Skeet): the problem is consistent with TB 0.9, POP3.

 - comment 59 (Chris January): TB 1.0 will stop moving junk after it's been up 
for a while, but restarting fixes the problem.  This same symptom is mentioned 
at bug 270039 comment 4 & 5, and bug 270466 -- this appears to be a new symptom 
previously unreported in this bug.

If any of the above commenters are no longer experiencing the problem, please 
say so.


Potential dupes of this bug: bug 231845 (IMAP), bug 270039 (non-default Junk 
Folder), bug 272414 ("empty mails" not getting moved).
Whiteboard: [see comment 60]

    
    

    
  
reply to comment 60: my junk folder is remote. The bug still occurs to me
frequently, on Thunderbird 0.9 as well as on Mozilla, on Win98 and on WinXP.
(Also --- this only happens to me since I have an IMAP server. No problems
before on a POP3!)

    
    

    
  
I reported this bug in version 1.5. I'm now using version 1.8b and the bug
doesn't show any more.

    
    

    
  
In my case the bug *appears* to have been caused by running out of IMAP
connections - 4 of my accounts are on the same server so I need to limit the
cached connection count to 1 for each of them (Courier-IMAP default settings
limit connections from a single IP address to 4).

bug 163951 is caused by the same problem for me, it seems.

I will wait and see if the problem is now 100% cured by reducing the cached
connection count.

Note: duplicates of this bug appear to include: bug 270537, bug 272414, bug 231845.

Alan

    
    

    
  
> Note: duplicates of this bug appear to include: bug 270537, bug 272414, bug
231845.

Hmm. Should have read more carefully. 272414 occurs with certain specific
messages (reliably) whereas (to the best of my knowledge) this bug is unreliable.

    
    

    
  
This problem was largely cured for me by reducing the cached connection count,
though I believe there may be the odd occasion where one or two messages don't
get moved.

The problem tends to occur when you have multiple accounts on the same server
and a server that limits the number of incoming connections from the same IP
address.

Is anyone still seeing this problem after reducing the number of cached connections?

Alan

    
    

    
  
Problem went away for me by me deleting my existing IMAP junk folders and letting Thunderbird recreate them.  It might be that parts of the logic in Thunderbird is case sensitive (my original folders were named 'junk', and Thunderbird recreated them as 'Junk').  This also made them be recognized as junk folders in the UI, they are now assigned the special junk folder icons whereas they were just folders before.

    
    

    
  
(In reply to comment #66)
> Problem went away for me by me deleting my existing IMAP junk folders and
> letting Thunderbird recreate them.

That does not fix the problem for me.  TB doesn't create a Junk folder by itself after it's restarted, and recreating the Junk folder doesn't help either.


    
    

    
  
*** Bug 270039 has been marked as a duplicate of this bug. ***

    
    

    
  
Using the released Thunderbird version 1.5 (20051201) on Windows XP (SP2) I experience the same issue as orginally described in this bug.

I have three POP accounts set to download to separate local folders.  All three filter junk mail (using the adaptive filter) to the single Junk folder in "Local Folders".  For at least the first two accounts, there is an occasional mail, marked (correctly) as junk, left in the account folder Inbox after filtering.

I can observe no pattern to the mails that are not moved, although I suspect it does not happen for one of the three POP accounts.  To give an idea of volume, there are about 40 mails junked each day, and I would not see this problem more frequently than once every couple of days.

Hope this helps!
Alastair


    
    

    
  
See bug 352225 -- TB 1.5/Seamonkey 1.0, and later, no longer automatically create a Junk folder when an account is created and "Move junk to [folder]" is turned on, so of course no junk messages are going to be moved.

    
    

    
  
*** Bug 318143 has been marked as a duplicate of this bug. ***

    
    

    
  
I experience this bug for some time now, but have the distinct impression it has gotten worse after the recent upgrade to TB 1.5.0.7

Running TB (currently 1.5.0.7 but same bug happened in earlier versions) under Windoze Xp. It will happen a number of times every day (I get some 150 junkmails a day unfortuately).

    
    

    
  
Hi

I'm using versión 1.5.0.8 (20061025) in Windows 2000 SP4 and I'm having the same problem.

Every time I check mail, all spam is filtered and moved to Junk folder, except one or two that remains in Inbox. I think it only happens in my IMAP account. POP3 accounts works fine.

    
    

    
  
I recently started having this problem with TB1.5.0.8, WinXP, IMAP server.  

Junk marking and filtering has been working fine for years across the versions, but now no matter what version I try to use, some mail gets marked as junk, but stays in the inbox.  I don't see any pattern.  It appears random.

    
    

    
  
(In reply to comment #74)
Same here with WinXP, TB version 2 beta 2 (20070218).

    
    

    
  
Latest trunk builds have the same problem. Mail marked as junk isn't moved to junk and even manually marking a message junk doesn't move it, it just changes from unread to read.

    
    

    
  
Ere, have you checked to make sure the Junk folder is specified correctly for each account?

    
    

    
  
Yes, and it works sometimes, but usually not. The mailbox is IMAP and junk should be moved to Junk in Local Folders.

    
    

    
  
No. Don't waste time & bandwidth downloading Junk. It should be moved to an IMAP Junk folder, if the user is connected via IMAP.

    
    

    
  
Well, thanks for the suggestion, but the server in question has a limitation in the number of folders so can't do that. And that's completely missing the point in that it should work whether the move is done on the same server or between servers/local folders.

    
    

    
  
Ere, can you send me your prefs.js?

Is the Junk.msf file up to date, i.e., if you click on the local Junk folder, does it rebuild the summary file?

    
    

    
  
The suggestion about IMAP in comment #80 also ignores the fact that this also happens to SMTP clients.  That may or may not be a useful workaround for IMAPpers, but it doesn't address the bug.  My junk folder is up-to-date.  I'm on a Mac.

    
    

    
  
#80 is not a workaround. It's a suggestion on how to implement the Junk feature. IMAP users should not waste bandwidth downloading spam. For POP clients this is not an option, but IMAP should (at least optionally) file efficiently, on the server.

FWIW, Eudora does this in IMAP mode. When you define an IMAP account ("personality"), it asks for (server-side) Trash & Junk folders, and uses those when IMAP messages are trashed or junked. Saves a lot of bandwidth vs. downloading all that stuff you don't want.

    
    

    
  
Chris, that's the default but there are cases where the defaults are no good. 

David, I'll send you prefs.js and check Junk.msf when I get to the machine.

    
    

    
  
prefs.js mailed. Junk.msf is up to date.

    
    

    
  


  
There are currently at least two problems with summary files in nsMsgLocalMailFolder and nsMailboxService. They might be causing trouble in some cases (I got a locked junk folder because of this, it seems). This patch is to fix these issues:

- m_parsingFolder was left true in nsMsgLocalMailFolder if nsMailboxService failed to run the parsing url (return value was not checked)
- parsing would always fail because the mailbox url was not properly parsed (mailbox://C:\Documents... was parsed as scheme=mailbox, host=C).

David, what do you think?

        Attachment #261276 -
        Flags: review?(bienvenu)

    
    

    
  
Ere, has this gotten worse on the trunk? I'm trying to figure out if the file spec exorcism has created the need for this patch...

    
    

    
  
Could be. At least I think I managed to rebuild the index once a month ago. Or then I had to resort to deleting summary file to make it rebuild, can't remember right now. But now it wasn't possible at all without the patch.

    
    

    
  
Comment on attachment 261276 [details] [diff] [review]
Possible fix - checked in

thx, Ere - this fixes other issues as well.

        Attachment #261276 -
        Flags: review?(bienvenu) → review+

    
  

        Attachment #261276 -
        Flags: superreview?(mscott)

    
  
Assignee: sspitzer → emaijala

    
    

    
  
Once this "cooks" on trunk, is there any shared code and therefore chance it could come to the branch as well?

    
  

        Attachment #261276 -
        Flags: superreview?(mscott) → superreview+

    
    

    
  
Ere, I've landed your patch on the trunk, since it fixed a serious regression in trunk builds. The checking for an error running the mailbox url part might help on the branch.

    
  
No longer blocks: 377239

    
    

    
  
The problem is still present in TB 2.0.0.0. I would estimate that it happens in about 20 to 40 percent of the cases when I download new mails. It is always either *all* or *none* of the correctly tagged junk mails that are moved to the junk folder.

    
    

    
  
It has been on trunk for 2 weeks now. The junk mail feature doesn't really work at all on 2.0.0.0 branch. Think this could come over now for testing? Thanks.

    
    

    
  
I can reproduce the problem reliably. Seamonkey 1.1.1 moves *no* mails, Seamonkey 1.0.8 moves *all* of them.

    
    

    
  
[Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9a5pre) Gecko/20070529 SeaMonkey/1.5a] (nightly) (W2Ksp4)

While all had been working fine for quite a long time,
I've been experiencing this again, for a few days on my Imap Inbox folder:
most messages stay there, only 1-2 messages are actually moved daily.

Yet, they are marked as "moved" in <junklog.html> (275 073 bytes),
that's 13 messages like
[
Detected junk message from &quot;Edi howden&quot; &lt;Edi.howden@sdk-web.de&gt; - When overridden in a descendant class, gets or sets the content length of data being received. at 26/05/2007 23:52:47
moved message id = 000b01c79fe0$3283d620$128a1552@mitch to mailbox://zyx@z.y.z/Junk

...

Detected junk message from &quot;Profiles Farm&quot; &lt;wcvfjaym@eilsen.com&gt; - FRANKF MARKT ALLE! at 01/06/2007 05:56:13
moved message id = 095E9FC75A9D1F5.A948ACD790@eilsen.com to mailbox://zyx@z.y.z/Junk
]
(NB: FWIW, if log is correct, affected messages are on 26th/28th/29th/31th/01st, some before this 29th nightly.)

In case it could help, I haven't closed MailNews nor "touched" <junklog.html> yet...
(I only hope that this build(s) is not affected by a temporary regression due to the string cleanups happening in MailNews or some other checkin...)

    
    

    
  
I'm seeing this again too. When I clicked the Junk folder Thunderbird complained it was locked and didn't display the junk messages, but then after moving back to inbox the messages were suddenly moved and Junk folder freed.

    
    

    
  
[Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9a5pre) Gecko/20070529 SeaMonkey/1.5a] (nightly) (W2Ksp4)

Yes, it seems to be a likely cause (as was said in the past):
I now clicked on my (local/POP3) Junk folder, which was indicated as empty,
it updated itself to show 1 unread message.
I'll see if this "solved" the issue for now or not.

Yet, from my previous comment, the log file (behaviour) seems to be too optimistic... (should be fixed "too").

    
    

    
  
In my case I do not have a Junk folder, I am moving the junk mails directly into the trash folder. This works fine with SeaMonkey 1.0.8, but SeaMonkey 1.1.1 exhibits this bug reproducibly.

    
    

    
  
(In reply to comment #98)
> [Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9a5pre) Gecko/20070529
> SeaMonkey/1.5a] (nightly) (W2Ksp4)
> 
> Yes, it seems to be a likely cause (as was said in the past):
> I now clicked on my (local/POP3) Junk folder, which was indicated as empty,
> it updated itself to show 1 unread message.
> I'll see if this "solved" the issue for now or not.

Ii cured it for sometime, then started again.
I had not restarted the application;
FWIW, both times I got 1 "Unable to copy to Sent folder" (should be "Draft") while composing a new message.

    
    

    
  
Well, after days and (now) some restarts, this does not work for long and I soon get this bug back.
Reinstalling SM/20070515 to see if it makes a difference.

    
    

    
  
(In reply to comment #96)
> Yet, they are marked as "moved" in <junklog.html> (275 073 bytes),

I filed:
Bug 383202 – <junklog.html> entries read "moved" even when messages are not actually moved

    
    

    
  
It appears to be fixed in Seamonkey 1.1.2

    
    

    
  
I can confirm this (again) now with Seamonkey 1.1.5 latest version stable.

This bug now exists over 4!!!!! years. Who is responsible for managing this bug and when is progress to be expected?!

    
    

    
  
I solved this same problem very easily (once I tried everything else ;>).  I too have several e mail accounts, and checked the "Move junk messages to:" settings on every one of them with no success. Junk would not move out of the Inbox automatically. Tried several of the the things suggested above, like compacting folders, etc. Nothing worked. 

Then it occurred to me: when I set up all of my e mail accounts, I configured all of them to deliver mail to the "Global Inbox" (store mail in Local Folders). ZAP! Brain fog clears... I discovered I also needed to set the Junk mail settings for one more account, the "Global Inbox" or Local Folders account! It was at the bottom of all the other accounts and it just never occurred to me to look there since all my "real" e mail accounts were correctly configured.

Go to > Tools > Account Settings, and click on Local Folders / Junk Settings.  Check the box: "Move new junk messages to: "Junk folder on: Local Folders"

Worked for me. Problem solved.

    
    

    
  
I don't believe Alan Thompson is describing the same problem, which is an intermittent failure to move *some* of the marked messages into the correct folder.


    
    

    
  
(In reply to comment #106)
> I don't believe Alan Thompson is describing the same problem, which is an
> intermittent failure to move *some* of the marked messages into the correct
> folder.
> 

Same here.  I just wanted to mention that it feels like a race condition (in that it appears this process is not happening at the right time always when multiple accounts are involved) to me.  I'm only really checking two email accounts, both of which get a lot of spam.

If:

1. I try to delete, move, or read a message while my mail is being downloaded.

2. My second mail server responds either too quickly. too slowly, (perhaps?  it only happens one of three or four times) or in some other unexpected fashion.

3. I stupidly start a compaction process before mail is completely downloaded (perhaps while my second mail server is responding too slowly) - but this can be automatically triggered these days.

Then some (or all) of the marked messages will not be correctly moved.  Depending on the circumstance, it may be all, all up to a certain one, or only those from one account.

Silly question: is it completely impossible to not have the messages appear in the inbox in the first place?  Can't the filters by applied upon receipt and not as a post-run?  I realize this is a different bug (does such a bug exist?) but it would likely address this issue as well.

Also, Is there anyone having this problem with only a single account?  If so, I would even suggest it's probably not the same issue.

-[Unknown]

    
    

    
  
In response to the last comment, I am using several IMAP accounts.

Alan

    
    

    
  
I too am having the problem, and have several IMAP accounts and explicitly do not use a global inbox for anything.  While the problem was occasional with Thunderbird 1.x and usually solved by restarting it, it has become annoyingly frequent and *not* fixed by restarting with 2.x.  Tbird has become incredibly unstable with 2.x as well, crashing or locking up frequently, but that's another bug report.
Flags: blocking-thunderbird3?

    
  
QA Contact: laurel → filters
Product: Core → MailNews Core
Flags: blocking-thunderbird3? → blocking-thunderbird3-

    
    

    
  
Comment on attachment 261276 [details] [diff] [review]
Possible fix - checked in

this is checked in, but I believe there are other remaining causes of this problem.

        Attachment #261276 -
        Attachment description: Possible fix → Possible fix - checked in

    
  
Flags: wanted-thunderbird3?

    
    

    
  
(In reply to comment #91)
> Once this "cooks" on trunk, is there any shared code and therefore chance it
> could come to the branch as well?

(In reply to comment #92)
> The checking for an error running the mailbox url part might help on the branch.

how do we assess whether attachment 261276 [details] [diff] [review] is worthy of being put on TB2?

    
    

    
  
In bug 497622, I have a fix for two issues in moves and copies. The symptoms of bug 497622 are failed moves, which are also the symptoms of this current bug. I think it is likely that the issues fixed in 497622 are some of, if not the, "other remaining causes of this problem" mentioned in comment 110.

Marking the dependency.
Blocks: 497622

    
    

    
  
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.10pre) Gecko/20100413 SeaMonkey/2.0.5pre - Build ID: 20100413000559

FWIW, I'm occasionally seeing this (mail correctly labeled as Junk, but sitting in Inbox). All my mail servers are POP3 and use the Global Inbox.
OS: Windows 2000 → All
Hardware: x86 → All
Whiteboard: [see comment 60] → [see comment 60] [also mozilla-1.9.1]

    
    

    
  
Replying to an off-line question from Wayne:

No, I did not have this problem for a long time. In other words WFM.

    
    

    
  
I did not encounter it for a long time as well. But the amount of spam has decreased significantly to maybe 1-2 messages per week, so I don't really have a statistically significant data basis.

    
    

    
  
Same here, but I still get a reasonable amount of spam through certain accounts.  I used to get it quite frequently.

That said, it's possible it's only less likely not, but not really sure.

-[Unknown]

    
    

    
  
bienvenu, Ere, what do you think about closing this?  

despite serge's last comment about still seeing this and comments 96 - 101 ... because of Ere's patch of comment 9, the long time this has been open during which time many contributing issues may have been fixed (mainly side causes not directly related to junk such as folder busy) I'm thinking this bug should be closed FIXED.  And new bugs filed for follow up of specific reports where they can receive better focus.


(Jacek, Joerg, Brackets, thanks for the update)

    
    

    
  
I haven't seen this anymore for a long while, I'd vote for closing.

    
    

    
  
consensus = WFM
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → WORKSFORME
Flags: wanted-thunderbird3?

    
    

    
  
The bug is still very present for me (and extremely annoying): TB 12.0.1 on WinXP with 2 IMAP accounts. How can I reactivate this? Or should I re-report it?

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: