Closed Bug 359339 Opened 18 years ago Closed 15 years ago

messages marked as junk and moved to trash folder lose junk status (with IMAP server who doesn't support user-defined keywords)

Categories

(MailNews Core :: Networking: IMAP, defect)

x86
macOS
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
Thunderbird 3.0b2

People

(Reporter: asa, Assigned: Bienvenu)

References

Details

(Keywords: regression)

Attachments

(3 files, 1 obsolete file)

I am using the latest branch builds and the Zimbra IMAP server. When a message is marked junk and moved to the trash folder, the junk indicator is lost. This makes finding and correcting junk false positives nearly impossible for me.
Do you have hardware or OS wrong? Because Mac OS running on a PC sounds weird.

I've also got the problem that Mail marked as Junk loses the indicator, that's happening when it's being moved to a different folder via a regular filter, per junk filter settings, or if manually marked as spam with junk options set to either delete (move to trash) or move to a different folder. It's also happening if deleting (moving to trash) an already marked email. This is in conjunction with IMAP. Using AOL's free email service (but not as ISP).

http://www.aol.de/Hilfeportal-Fragen-email-Web/Steht-mir-IMAP-Verfuegung-830300017-0.html

Also see bug 360463 (possible duplicate).
I just switched from POP3 to IMAP for one of my accounts and I see this behavior in Thunderbird 2.0.0.9 on Windows XP.

Mail is received in Incoming-Folder, marked as junk and filtered to Junk-Folder. When I delete a message to trash, this message is losing its junk status.

Could be related to Bug 360463.

The IMAP-Server is Dovecot.
Assignee: mscott → nobody
This is still 100% reproducible on Courier IMAP, the following version:

* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT
THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready.
Copyright 1998-2004 Double Precision, Inc.  See COPYING for distribution
information.

This is with Thunderbird version 2.0.0.17 (20080914).

David Bienvenu, if you're still following this, I gave you a couple of IMAP
accounts to play with on this server a while back (something@jynxy.net). Those
will manifest the problem if you want to look at this. Happy to re-create if
you no longer have details.

I originally posted this on bug 360463 and then reproduced it here after noticing that that bug is now closed.

Cheers

Alan
I think this is definitely wanted for TB 3 - if you make a mistake manually marking a message as junk, you don't really want to go into your junk and have to mark it as junk to un-mark it as junk, especially as we're fixing bug 470011.
Flags: wanted-thunderbird3+
Flags: blocking-thunderbird3?
(In reply to comment #5)
> I think this is definitely wanted for TB 3 - if you make a mistake manually
> marking a message as junk, you don't really want to go into your junk and have
> to mark it as junk to un-mark it as junk, especially as we're fixing bug
> 470011.

I meant bug 208197.
Attached file Protocol Log
Protocol Log of what is happening, I did:

1) Start Thunderbird (it checked my mail server for new mail by default).
2) Select a message in the inbox
3) Mark it as junk (message was moved to junk folder).
4) Select Junk Folder (message in junk folder but not indicated as junk).
OK, this server doesn't support user-defined keywords - I'm not sure why the pending attribute code isn't working in this case. Perhaps we're not going through the code path that sets the pending attributes in the junk code case.
ok, for 3.0 at least, this is a regression from the imap delete in the background fix - imap moves never get to the code that sets pending attributes anymore, because the offline move playback does not call nsImapMailFolder::CopyMessages at all. This actually probably caused several regressions, so I need to figure out a way to fix it.
Attached patch proposed fix (obsolete) — Splinter Review
The fix is to make offline imap/move copy set attributes on pending headers just like online move/copy used to. To do that, I had to make a helper routine to do that (easy) and then get headers to pass to that helper (hard). Because the original headers have been deleted from the source db when we play back the operation, I needed to use the "fake" offline headers in the target folder, which are clones of the original source headers. In order to get the fake offline headers, I had to resurrect the src message key property of offline imap operations.

There's also a bit of whitespace cleanup in the patch.

This should fix this bug, and the round trip problem you were seeing with zimbra and clearing the junk flag (I think).
Assignee: nobody → bienvenu
Attachment #355867 - Flags: superreview?(bugzilla)
Attachment #355867 - Flags: review?(bugzilla)
This is certainly an improvement, one issue though:

1) Select a message in inbox, mark it as junk
2) Go to the junk folder, message (usually) isn't displayed, if it is it hasn't got the junk marking.
3) Go to a different folder then back to junk again - message now displayed.

I'm getting an assertion when I click on the junk folder the first time:

###!!! ASSERTION: Overwriting an existing document channel!: '(loadFlags & nsIChannel::LOAD_REPLACE) || !(mDocumentRequest.get())', file /Users/moztest/comm/main/src/mozilla/uriloader/base/nsDocLoader.cpp, line 514
After step 3, does the message have the junk marking?

Does it help if you wait a little while between step 1 and 2? Not that I can imagine why it would help...

A protocol log would be helpful.

Re the assertion, that's what you see if you're loading one message and then you load an other message before the first message load has finished, iirc.
(In reply to comment #12)
> After step 3, does the message have the junk marking?

Yes.

> Does it help if you wait a little while between step 1 and 2? Not that I can
> imagine why it would help...

No, in fact in the case I did it, I think autosync kicked in and the message is now gone from my system (not in junk either) - and I've just tried this a second time and the message got lost again.

> A protocol log would be helpful.

Will attach in a moment.

> Re the assertion, that's what you see if you're loading one message and then
> you load an other message before the first message load has finished, iirc.

I may have got where the assertion happens wrong - it happens when you mark as junk - so the junk status is being changed, and it gets moved, and we load the next message. This sounds like a different bug.
Courier IMAP does not aggressively sync the changes between multiple connections. E.g., if with connection 1 on the inbox, I move a message to Junk, and then click on the junk folder, I may not see that moved message right away, if a second connection was already open on junk. Eventually the state is synced between the connections, however.

Do you have a log of where a message was lost? Or was one lost in the session for the log you attached?
Is this bug related to IMAP only? 
From time to time some of the junk messages are losing their junk status residing in Junk folder on my POP-accounts (Local Folders, actualy). And in my case 'losing the junk status' does not seems to relate to compacting the folders, because I have autocompacting turned off. 

(Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b3pre) Gecko/20090116 Shredder/3.0b2pre)
ping re #15 - was the message lost in the session that you attached a log for? I think the basic approach is something I might build on to fix bug 439108...
Version: 2.0 → Trunk
marking plus, this is a regression due to the better faster imap stuff.
Flags: blocking-thunderbird3? → blocking-thunderbird3+
Keywords: regression
Whiteboard: [has patch, waiting for info]
(In reply to comment #16)
You are seeing bug 471682.
no, that's for local messages, not imap messages, which is what this bug is about.
(In reply to comment #20)
Comment 16 WAS about local messages. All I'm saying is that Valery is seeing a different bug than this one.
Adding "IMAP" and "doesn't support user-defined keywords" in bug summary, to reduce confusion like Comment #16.
Summary: messages marked as junk and moved to trash folder lose junk status → messages marked as junk and moved to trash folder lose junk status (with IMAP server who doesn't support user-defined keywords)
Component: General → Networking: IMAP
Product: Thunderbird → MailNews Core
QA Contact: general → networking.imap
Update: with the patch in bug 469231 applied as well as this one it fixes comment 11:

(In reply to comment #11)
> 1) Select a message in inbox, mark it as junk
> 2) Go to the junk folder, message (usually) isn't displayed, if it is it hasn't
> got the junk marking.
> 3) Go to a different folder then back to junk again - message now displayed.

the junk message is correctly displayed at step 2.

However, it doesn't fix the fact that if autosync kicks in before you go to the junk folder then the message is lost.
Whiteboard: [has patch, waiting for info] → [needs new patch?]
Comment on attachment 356923 [details]
Protocol Log after patch.

this log shows that the message is *not* deleted from the junk folder; in fact, it shows us downloading the headers for it.  Which again makes me suspect that the problem doesn't have to do with this patch but rather the problems we had with thinking messages were killed (bug 469231 and perhaps bug 414723)
I'm going to try this on the mac as well and see if I can reproduce it there with the test account...
Attachment #360576 - Flags: superreview?(bugzilla)
Attachment #360576 - Flags: review?(bugzilla)
Comment on attachment 360576 [details] [diff] [review]
non-bitrotted patch, also contains fix for 208197

asking for r/sr on cumulative patch since they need to go in together...
Blocks: 208197
Status: NEW → ASSIGNED
Comment on attachment 355867 [details] [diff] [review]
proposed fix

(old patch)
Attachment #355867 - Attachment is obsolete: true
Attachment #355867 - Flags: superreview?(bugzilla)
Attachment #355867 - Flags: review?(bugzilla)
Attachment #360576 - Flags: superreview?(bugzilla)
Attachment #360576 - Flags: superreview+
Attachment #360576 - Flags: review?(bugzilla)
Attachment #360576 - Flags: review+
Comment on attachment 360576 [details] [diff] [review]
non-bitrotted patch, also contains fix for 208197

With this patch and a mailnews clobber, I can't seem to reproduce my earlier problems now.

couple of nits:

+  PRBool moveMessages,changeReadState;
+  nsCOMPtr <nsIMsgFolder> targetFolder;

Space after the comma, and no space before < please.

Otherwise lets get this in for some more testing.
oops, forgot to address nits - I'll do that right now...but this is now fixed.
Status: ASSIGNED → RESOLVED
Closed: 15 years ago
Resolution: --- → FIXED
Whiteboard: [needs new patch?]
Target Milestone: --- → Thunderbird 3.0b2
You need to log in before you can comment on or make changes to this bug.