361182 - IMAP mail moved by filter incorrectly marked as read

Status: RESOLVED INVALID

Description

[Richard Reich]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
Build Identifier: Thunderbird version 2 beta 1 (20061118) (and previous 2.0 versions)

An automatic filter on an IMAP account INBOX moves mail to folder "To me" if the To: or Cc: includes my email address.

Expected: Mail moved by the filter will be unread.

Actual: Mail is marked read.

Reproducible: Always

Steps to Reproduce:
1. Create a filter to move mail from INBOX to a folder.
2. Set it to run automatically.
3. Note the "read" status of unread mail that is moved by the filter's operation.

Actual Results:  
Newly arrived mail moved by the filter is marked read.

Expected Results:  
Newly arrived mail moved by the filter is marked unread.

Although this is similar to other reports I've found, they seem to involve more complex interactions with junk filtration.  (Moving marked junk mail to "Junk" is enabled, but this report considers non-junk messages that have not been marked as such either automatically or manually.)

Comment 1

[Mike Cowperthwaite]

Is the destination folder also IMAP, or a Local Folder?

FWIW, I tried filtering incoming IMAP to both types of folders and the message remains Unread in both cases.  TB 2b1-1111 / fastmail.fm IMAP.

Comment 2

[Richard Reich]

(In reply to comment #1)

Destination folder is IMAP on the same server.  The folder structure is INBOX with all other folders appearing as subfolders.  (I dislike this intensely, but it seems to be sweeping the world of IMAP host providers as they adopt Courier.)

I am also running Avast (free edition). I'm currently trying to make sure that the "allow anti-virus clients..." setting is irrelevant to this bug.

Comment 3

[David :Bienvenu]

the quarantine setting is specific to pop3, so not relevant here.

Are you talking about the message getting marked read in the source folder, not the destination? If so, that's on purpose - there might be a pref for turning that off. But we automatically mark messages read in the source folder when you move them.

You can set your online directory to "INBOX." to get sub-folders of the inbox to appear at the same level as the inbox.

Comment 4

[Richard Reich]

(In reply to comment #3)
> the quarantine setting is specific to pop3, so not relevant here.

Okey dokey.

> Are you talking about the message getting marked read in the source folder, not
> the destination? If so, that's on purpose - there might be a pref for turning
> that off. But we automatically mark messages read in the source folder when you
> move them.

I've read the previous para a half-dozen times but I still do not believe I understand it.  When the message is moved, it appears in the destination folder and disappears from the source folder.  If you mean you mark messages as read in the source folder, then move them to the destination folder, that is exactly the bug I am reporting.  If that is intentional, it is a bad intention.

> You can set your online directory to "INBOX." to get sub-folders of the inbox
> to appear at the same level as the inbox.

That is its current value.  It does not get subfolders to appear at the same level.  Refreshing (in the subscription dialog) doesn't help.  "Allow server to override..." has no effect one way or the other.  (I recall the setting is the server's default.)

Should I submit this as a separate bug?

Comment 5

[Richard Reich]

(In reply to comment #4)

> > You can set your online directory to "INBOX." to get sub-folders of the inbox
> > to appear at the same level as the inbox.
> 
> That is its current value.  It does not get subfolders to appear at the same
> level.  Refreshing (in the subscription dialog) doesn't help.  "Allow server to
> override..." has no effect one way or the other.  (I recall the setting is the
> server's default.)
> 
> Should I submit this as a separate bug?
> 

WAIT!  I am completely mistaken.  I thought you meant the "namspace" setting.  I just tried INBOX. in the "server directory" field and it works!  Thanks!

I am talking about the folder hierarchy display problem I had, not the reported bug.

Comment 6

[David :Bienvenu]

If you thought that last paragraph was confusing, this one might be worse :-)

an IMAP move is really a copy followed by a delete. A delete is just storing a flag on the message saying it's Deleted, but it's still in the folder until the folder is expunged (we hide this from you unless you're using a particular delete setting known as the imap delete model). All I was saying was that when we store the /DELETE flag on the message in the source folder, *after* the copy, we also store the /SEEN flag on the message in the source folder. 

Since we mark the message read in the source folder *after* we've copied it to the destination folder, it should have no effect on the copy of the message in the destination folder. But broken IMAP servers have been known to mess this up...an imap protocol log of a session where this happens, generated by following these instructions, might help me tell if that's what's going on: http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap


glad the "INBOX." setting is working for you.

Comment 7

[Richard Reich]

(In reply to comment #6)
> If you thought that last paragraph was confusing, this one might be worse :-)
> 
> an IMAP move is really a copy followed by a delete. A delete is just storing a
> flag on the message saying it's Deleted, but it's still in the folder until the
> folder is expunged (we hide this from you unless you're using a particular
> delete setting known as the imap delete model). All I was saying was that when
> we store the /DELETE flag on the message in the source folder, *after* the
> copy, we also store the /SEEN flag on the message in the source folder. 
> 
> Since we mark the message read in the source folder *after* we've copied it to
> the destination folder, it should have no effect on the copy of the message in
> the destination folder. But broken IMAP servers have been known to mess this
> up...an imap protocol log of a session where this happens, generated by
> following these instructions, might help me tell if that's what's going on:
> http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap

Actually, I understand that perfectly.  Thanks!

I made a log, which I will attach momentarily.  It contains two interesting events (and given how much spam I get, maybe some irrelevant spam moves).

(1) I sent myself a test note from another account (at Yahoo).  I did nothing after sending it.  Didn't even give Thunderbird focus.  Saw the message arrive and get moved to "To me" folder (that is, saw file name bolding).  Interestingly, the destination folder stayed bold until a "new mail" sound and alert popped up.  Then it lost bold!  I checked the destination folder at that point and there was the mail.  So I tried...

(2) Turn off option for sound and alert popup on new mail.  Repeat (1).  I waited a while to see if the bolding would be lost on the destination folder.  It was not.  Looked in the folder and there was the mail, marked unread.

> 
> glad the "INBOX." setting is working for you.

Thanks!!  I really hate that subfolder stuff.

Comment 8

[Richard Reich]

Created attachment 246182 [details]
IMAP log file

Comment 9

[Mike Cowperthwaite]

I was noticing some odd notification behavior when testing this filtering.  I'll have to retest, but: I didn't see a popup alert for any filtered message; filtering to a Local Folder, I got the sound but no tray icon; filtering to an IMAP folder, I got the icon but, one time anyway, no sound.  This using the Mark As Deleted deletion model, if that makes a difference.

Comment 10

[David :Bienvenu]

Looks like a server problem to me - here's the relevant snippet. We "peek" at the content-type header and the first 2k of message body - peeking is IMAP-speak for give me part of the message, but don't mark it as seen. Unfortunately, the server is marking it as SEEN

3292[2eaec88]: 3f34860:xxx.xxx.com:S-INBOX.To me:SendData: 14 UID fetch 1793 (UID BODY.PEEK[HEADER.FIELDS (Content-Type)] BODY.PEEK[TEXT]<0.2048>)


3292[2eaec88]: ReadNextLine [stream=283c610 nb=65 needmore=0]

3292[2eaec88]: 3f34860:xxxx.xxx.com:S-INBOX.To me:CreateNewLineFromSocket: * 187 FETCH (UID 1793 BODY[HEADER.FIELDS ("Content-Type")] {43}


3292[2eaec88]: 3f34860:xxxx.xxxx.com:S-INBOX.To me:STREAM:OPEN Size: 0: Begin Message Download Stream

3292[2eaec88]: ReadNextLine [stream=283c610 nb=41 needmore=0]

3292[2eaec88]: 3f34860:xxx.xx.com:S-INBOX.To me:CreateNewLineFromSocket: Content-Type: text/plain; charset=ascii


3292[2eaec88]: ReadNextLine [stream=283c610 nb=2 needmore=0]

3292[2eaec88]: 3f34860:xxx.xx.com:S-INBOX.To me:CreateNewLineFromSocket: 


3292[2eaec88]: ReadNextLine [stream=283c610 nb=22 needmore=0]

3292[2eaec88]: 3f34860:xx.xxx.com:S-INBOX.To me:CreateNewLineFromSocket:  BODY[TEXT]<0> {183}


3292[2eaec88]: ReadNextLine [stream=283c610 nb=78 needmore=0]

3292[2eaec88]: 3f34860:xxx.xxx.com:S-INBOX.To me:CreateNewLineFromSocket: test=0A=0A=0A=0A__________________________________________________=0ADo You=


3292[2eaec88]: ReadNextLine [stream=283c610 nb=78 needmore=0]

3292[2eaec88]: 3f34860:xxx.xxx.com:S-INBOX.To me:CreateNewLineFromSocket:  Yahoo!?=0ATired of spam?  Yahoo! Mail has the best spam protection around =


3292[2eaec88]: ReadNextLine [stream=283c610 nb=27 needmore=0]

3292[2eaec88]: 3f34860:xxx.xxx:S-INBOX.To me:CreateNewLineFromSocket: =0Ahttp://mail.yahoo.com 


3292[2eaec88]: ReadNextLine [stream=283c610 nb=3 needmore=0]

3292[2eaec88]: 3f34860:xxx.xxx.com:S-INBOX.To me:CreateNewLineFromSocket: )


3292[2eaec88]: ReadNextLine [stream=283c610 nb=31 needmore=0]

3292[2eaec88]: 3f34860:xxx.xxx.com:S-INBOX.To me:CreateNewLineFromSocket: * 187 FETCH (FLAGS (NonJunk))


3292[2eaec88]: ReadNextLine [stream=283c610 nb=24 needmore=0]

3292[2eaec88]: 3f34860:xxx.xxxx.com:S-INBOX.To me:CreateNewLineFromSocket: 14 OK FETCH completed.


3292[2eaec88]: 3f34860:xxx.xxx.com:S-INBOX.To me:SendData: 15 IDLE


3292[2eaec88]: ReadNextLine [stream=283c610 nb=37 needmore=0]

here's where it tells us the message is /SEEN.

3292[2eaec88]: 3f34860:xxx.xxx.com:S-INBOX.To me:CreateNewLineFromSocket: * 187 FETCH (FLAGS (\Seen NonJunk))

If you have any way of getting hold of the people responsible for the server, you might forward this to them...

Comment 11

[David :Bienvenu]

this looks like a server problem - here's the server greeting, for future reference - it looks like some Courier variant.



 * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2004 Double Precision, Inc.