automatic message filter looses some data from the header after move from POP3 to IMAP account



11 years ago
7 years ago


(Reporter: mars, Unassigned)


Firefox Tracking Flags

(Not tracked)




(2 attachments)



11 years ago
User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv: Gecko/20070309 Firefox/
Build Identifier: (20080213)

this is quite annoying behavior in the Thunderbird that's sitting there for some time. Version at this moment.
So in principle I have defined e-mail accounts in that order:
0) ... some IMAP/POP3 accounts ...
1) one IMAP account with folders "Inbox", "Sent", "Trash", let's call it "my new account"
2) one POP3 account with a folder "Inbox", let's call it "my old account"
3) Local Folders (not used by these two)

There's a filter on the POP3 account ("my old account"), that moves ALL messages to the IMAP/Inbox ("my new account"). Both accounts are setup to check for new messages at startup and every 10 minutes. These are two different servers, two different e-mail addresses.
Often it appears, that new messages that were moved from POP3 account appear in the IMAP/Inbox without some header data: Subject, Sender etc. The message source is correct, only the Thunderbird's database is broken. Messages have correct header data on the server of "my new account" (IMAP), because I can see them through Web-mail correctly. Solution for this situation is to delete "my new account" (IMAP) settings from Thunderbird, add it again, and re-download headers of messages. Other solution is to move messages without subject to other folder (f.e. Trash) and move it back to Inbox. Also, it seems that this behavior appears only during downloading messages at startup, later on (after 10 minutes) messages are downloaded and moved correctly. 

Reproducible: Sometimes

Steps to Reproduce:
1. Create POP3 and IMAP accounts (different servers), setup to check messages at startup
2. Create a message filter on POP3/Inbox account that moves all messages to IMAP/Inbox 
3. Shutdown Thunderbird.
4. Send some messages to POP3
5. Start Thunderbird and wait for downloads/move filter to finish.
Actual Results:  
Messages in the IMAP/Inbox account have empty Subject and Sender fields.

Expected Results:  
Messages in the IMAP/Inbox account shout have correct Subject and Sender fields.

A screenshot:

Comment 1

11 years ago
Created attachment 309916 [details]
> Also, it seems that this behavior appears only during downloading messages at startup,
> later on (after 10 minutes) messages are downloaded and moved correctly. 

This indicates that cause is contention between followings.
  A. move of POP3 mails to IMAP's Inbox(APPEND) by filter at start up
  B. FETCH to IMAP's Inbox by 'check for new messages at startup'
And phenomenon looks to be similar to other problems when move/copy failure.
  Bug 216442 : Move failure due to incorrect(longer) RFC822.SIZE
  Bug 368112 : Interrupted move (verified1.8.1.4  i.e. fixed by Tb 
  Bug 405440 : Interrupted move (verified1.8.1.12 i,e. fixed by Tb  
  Bug 398498 : Filter action of "copy then move" fails

> Actual Results:  
> Messages in the IMAP/Inbox account have empty Subject and Sender fields.

Sounds to be garbage after move failure. (phantom mail seen when above bugs)
What date is displayed in Date column?

Followings are possibly easier/simpler recovery procedures.
 1. Properties of mail folder/General Information tab/Rebuild Index
 2. Delete ".msf" file and restart Tb

If contention of Inbox folder of IMAP is cause of move failure or fetch failure (then .msf corruption), I think following is effective workaround if next option is set appropriately, 
  Account Settings/Server Settings/Advanced
    Maximum number of server connection to cache
although it won't resolve problem perfectly.
Workaround :
  1. Create IMAP mail folder(s) to move(upload) POP3 mails to IMAP server
  2. Change move target of POP3 message filter to the folder created by step 1.
If problem can be re-produced, get IMAP protocol log, and check real protocol level flow to see what is happening. See Bug 402793 Comment #1 for getting log. I think follwing parameter will be helpful in your case. 
> setenv NSPR_LOG_MODULES imap:5,pop3:5
Oh sorry, you attached screens shot to comment #1.
Date column looks to be time-stamp of FETCH, not 1970/1/1(or 1969/12/31).
It seems to be caused by FETCH failure due to interfere by APPEND of POP3 mail, instead of move(APPEND to IMAP folder by POP3 filter) failure due to interfere by FETCH of IMAP mail.

Comment 5

11 years ago
I have just selected two messages from POP3/Inbox, right-click, selected to move to IMAP/Inbox and... one (only one) of messages was indexed without Subject and Sender. This is always reproducible. I can move it back to POP3/Inbox (then sender/subject appears ok again), and another move gives exactly the same result. Attached logs:

selected 2 messages, moved to IMAP/Inbox. message2 does not have subject and sender. clicked to view body of message2. 

some spam :-) worked ok

forwarded spam: didn't work, subject/sender was not indexed.

Comment 6

11 years ago
Created attachment 310437 [details]
(In reply to comment #6)
> Logs

IDLE is seen in you log, so your IMSP server supports IDLE and you don't disable use of IDLE.
From point of IMAP protocol level view, there is no incorrect command/sequence is seen in your log. APPEND's by mail move are successful. This is reason why re-fetch of all mails works well.
Problem looks to be ".msf"(cached header data etc.) corruption when contention.
 (1) APPEND of first mail by mail move
 (2) New mail notification for appended mail, because IDLE is used
 (3) APPEND of second mail by mail move while (2) is in progress
 (4) New mail notification for second mail while (2) is in progress

Because NSPR log doesn't have timestap, it's very difficult to analyze timing related issue. When MS Win, "NSPR logging with DebugView" can be used to add timestamp (See Bug  86396 Comment #7). Do you have MS Win PC? If yes, can you re-produce problem on it? If no, I'll try to re-produce problem with my Gmail IMAP account(I use MS Win-XP SP2). 

Comment 8

11 years ago
> Do you have MS Win PC? 

Not with Thunderbird, sorry. I'm surprised that Thunderbird does not log timestamp by default.
Seems to be same problem as Bug 379039. xref.
Depends on: 379039


9 years ago
Component: General → Filters
Product: Thunderbird → MailNews Core
QA Contact: general → filters

Comment 10

9 years ago
Marcin, do you agree this is the same as  Bug 379039 -  Some random messages not showing in Inbox IMAP folder ?

Comment 11

7 years ago
Marcin seems to be gone
Last Resolved: 7 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.