Copy to folder by message filter locks dest folder - copy of second or later mail by filter won't complete might see JS Error of "folderSummaryInfoEl.parseFolder is not a function" in newmailalert.js

VERIFIED FIXED

Status

--
major
VERIFIED FIXED
12 years ago
11 years ago

People

(Reporter: ianribas, Assigned: Bienvenu)

Tracking

({verified1.8.1.5})

Trunk
x86
Windows XP
verified1.8.1.5
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

12 years ago
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4
Build Identifier: versão 2.0.0.4 (20070604)

I create a filter with the following conditions:

- From is in address book Personal
- Subject doesn't contain ...

And the following actions:

Mark as read
Copy to ....

The filter is to be applied to the Inbox folder of the "Local Folders" account. It is a kind of "archive" filter.

I hav two email accounts, both saving new messages on the Inbox folder of the "Local Folders" account.

The problem is that, when receiving messages, only the first message received that matches the filter is copied to the chosen folder, the others are marked as read but not copied. Stranger still, I cannot copy or move messages to the folder even by hand, the folder seems somehow "blocked". If I restart thunderbird I can then copy the messages to the folder.

In the filter log, all actions are logged ok, even those that have not really happened! I see the all messages have been trated by the filter because they are marked as read, but the copy action does not work, even if it is logged as if it did.

I'm using thunderbird 2.0.0.4, brazillian portuguese version, on windows XP english.

Below a copy of the msgFilterRules.dat file:

version="8"
logging="yes"
name="Arquivamento"
enabled="yes"
type="1"
action="Mark read"
action="Copy to folder"
actionValue="mailbox://nobody@Local%20Folders/Antigos-2007"
condition="AND (from,is in ab,moz-abmdbdirectory://abook.mab) AND (subject,doesn't contain,[Echecs])"


Reproducible: Always

Steps to Reproduce:
1. Create filter with a "From is in address book" condition and a "copy to folder" action on local folders/Inbox
2. Make sure you are in your own address book
3. Send two or more test messages to yourself
Actual Results:  
Only first message is copied to folder and folder is "blocked".

Expected Results:  
All messages should have been copied.
(In reply to comment #0)
> Actual Results:  
> Only first message is copied to folder and folder is "blocked".

It sounds that filter is waiting for lock or copy failed due to dead lock or something wrong, although culprit of dead lock or something wrong can be "copy by filter" himself.
Processes I can imagine which possibly access to folder of Antigos-2007 are;
 - Message filter for other accounts or "Local Folders"
 - Junk filter of other account (if mail is moved to Antigos-2007 by filter)
 - Internal re-build index if something wrong occurred during filtering
 - Automatic compact folder
 - Check for Retention Policy
 
Questions to clarify.
(Q1) POP3? IMAP? If POP3, Global Inbox? (sounds POP3 and Global Inbox)
(Q2) When POP3 and Global Inbox, both message filter for account and message
     filter of account for Global Inbox(usually Local Folders) can be defined.
     Do you have filter for both? (2 accounts, then 3 filters can exist)
(Q3) Do you have other filter rule which moves(not copy) mail to Antigos-2007?
     Please check all three message filters, if multiple message filters.
(Q4) Do you enable "allow antivirus clients to quarantine individual messages"?
(Q5) Do you enable "Check for new messages at startup"?
     If yes, will problem occur when options are changed?
        - Disable "Check for new messages at startup"
        - Disable "Check new message every NN minutes"
      (Case-1) Get Msgs
      (Case-2) Get All New Message  
(Q6) Do you install extension(add-on)?
     If yes, will problem occur even when safe-mode of Thunderbird?
(Reporter)

Comment 2

12 years ago
Answers:

Q1: It is effectively POP3 and Global Inbox, sorry for not mentioning it earlier.

Q2: I tried the filter for the Global Inbox only, so only one filter exists for the Global Inbox folder.

Q3: No other filters exist.

Q4: "allow antivirus clients to quarantine individual messages" is not enabled.

Q5: The two "Check ..." options were set. Tried taking them off and nothing changed. Tried case-1 and case-2 also to no avail.

Q6: I have QuickFile installled. Tried safe mode and the provblem continues.

Other than that:
- "automatic compact folder" seems to be off.
- There was a "retention policy" for the Inbox folder as "Use server defaults" (or something that would translate to that), changed to "Never delete" and problem persists. The terget folder has the same "Never delete" policy.
- Is there a way to check if index rebuild is going on?

Anyway, thanks for the prompt answer! Even if mine was not as swift.
(Q7) Do you use anti-virus software? Are file of Antigos-2007 and Antigos-2007
     excluded from scan target of anti-virus software?
(Q8) If not excluded, what will happen when "Local Folders" directory is excluded
     from scan target of anti-virus software.
Problem is re-created with Thunderbird version 3.0a1pre (20070612) (Win-XP SP2).
 Two mails on POP3 server to be filtered. Get Msgs => only first one is copied.
 (Tb's mail directory is not excluded from scan target of avast! 4)
Next error in Error Console.
> Error: folderSummaryInfoEl.parseFolder is not a function
> Source File: chrome://messenger/content/newmailalert.js
> Line: 88
After above problem,
 1. Move of a mail(drag&drop) to the copy target folder of filter does nothing.
 2. Compact folder, Re-build index says Done Compaction or Done.
 3. Move mail(D&D) from the folder to other healthy folder worked fine.
 4. Same action as step 1 still does nothing.
Above JS error seems to have locked the folder when copy of first mail by filter. Note: I didn't do test case of no scan by avast! yet.

Confirming.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Ian Ribas(bug opener), is there above error message in Error Console when the problem occur on Tb 2.0? 
Changing to Core because error in messenger/content/newmailalert.js.
Component: Mail Window Front End → MailNews: Backend
Product: Thunderbird → Core
QA Contact: front-end → backend
Version: unspecified → Trunk
CC-ing to David Bienvenu.
Bug 307540, Bug 331391, Bug 368383, Bug 376235 look to be reports of some phenomena in this bug and/or reports of problem after this bug.
Bug 289683 is possibly one of oldest report of problem, I'm not sure though.
Above JS Error occurr just after copy of first filtered mail, i.e. problem is recreated with only one mail and your message filter.
I tested after stop standard shield(on-access scan)) of avast. Above JS error was issued, and problem occurred even when no scan by avast.
Ian Ribas, please ignore (Q7)/(Q8) in Comment #3.
Changing summary, because cause is JS error just after first copy by filter, and because subsequent problem is not only "copy of second or later mail by filter won't complete".
Ian Ribas(bug opener):
Please modify/correct bug summary if you feel it's inappropriate.
Summary: "Copy to folder" filter copying only first message → JS Error of "folderSummaryInfoEl.parseFolder is not a function" in newmailalert.js just after "Copy to folder by message filter", and the folder is locked, then subsequent problem occurs(e.g. copy of second or later mail by filter won't complete)
(Assignee)

Comment 11

12 years ago
I believe that js error in newmailalert.js (which is TB only) is just a symptom of the underlying problem, and not the cause. The underlying problem seems to be that the db is marked invalid - I haven't reproduced this bug to verify it, however. I will try that today.
Assignee: nobody → bienvenu
(In reply to comment #11)
Oh, David has come!
Possibly problem when "copy to a folder under Local Folders" in POP3 account's message filter. I tested with filter of "If (condition), Add a Tag, Copy to a folder under Local Folders", according to bug opener's comment #0.
Filter rule I used in problem re-creation test.
> action="AddTag"
> actionValue="tag-01"
> action="Copy to folder"
> actionValue="mailbox://nobody@Local%20Folders/filter-copy-test"
> condition="OR (from,contains,aaa@xxx.yyy.zzz)"
(my real mail address is replaced by aaa@xxx.yyy.zzz before paste)
(Assignee)

Comment 14

12 years ago
OK, I've reproduced this problem locally in 2.0.0.4 and the trunk - the issue is that for some reason the first copy request with that folder as a target never gets cleared in the copy service, and all the other copy requests get in line behind it. I'm not sure why it doesn't get cleared, since the copy does seem to be getting executed.
Status: NEW → ASSIGNED
(Assignee)

Comment 15

12 years ago
Posted patch proposed fixSplinter Review
When using the global inbox, you want to copy from m_downloadFolder, which is the global inbox, not m_rootFolder, which is the incoming server's inbox. I'm not sure why the copy worked for the first message at all with the old code.
Attachment #268866 - Flags: superreview?(mscott)
(In reply to comment #15)
> Created an attachment (id=268866) [details]
> proposed fix

> I'm not sure why the copy worked for the first message at all with the old code.

Oh, sorry. I forgot to mention very important thing.
Reporter's case is Global Inbox case. But when I tested, I tested with account of no Global Inbox which I usually use. So probably m_downloadFolder==m_rootFolder in my case, and mail is copied to "Local Folders" by filter.
Can this explain why?
(Reporter)

Comment 17

12 years ago
Just to say that I don't get anything in the Error Console ... And to thank
again for the prompt answers. Waiting eagerly for the fix to be released ;-)
(Assignee)

Comment 18

12 years ago
I'm not sure what's going on here - m_downloadFolder and m_rootFolder are the same folder, it turns out, if I'm reading the code correctly. That explains why the first copy works at all. Where we're failing is in the copy service code that compares pointer values for the srcFolder nsISupports pointer for the copy. Somehow the QI's are coming out different, which doesn't make any sense. I guess I'll have to debug it more to figure out why my change seemed to work, and if there's some other way to fix it.

Updated

12 years ago
Attachment #268866 - Flags: superreview?(mscott) → superreview+
(Assignee)

Comment 19

12 years ago
adjusting summary since it's the copy by filter that's the interesting part. I take back what I said earlier - m_downloadFolder and m_rootFolder aren't the same. The reason the copy works at all is because we use the msgHdr->GetFolder() result as the source folder for the copy url, but we check for the src folder passed into the copy operation when trying to clean up the request. This causes confusion. So the fix is good, but I might want to try to cleanup the msgcopyservice code so that we can't get into this situation again...
Summary: JS Error of "folderSummaryInfoEl.parseFolder is not a function" in newmailalert.js just after "Copy to folder by message filter", and the folder is locked, then subsequent problem occurs(e.g. copy of second or later mail by filter won't complete) → Copy to folder by message filter locks dest folder - copy of second or later mail by filter won't complete might see JS Error of "folderSummaryInfoEl.parseFolder is not a function" in newmailalert.js
(Assignee)

Comment 20

12 years ago
fixed on trunk - only fixes global inbox case.
Status: ASSIGNED → RESOLVED
Last Resolved: 12 years ago
Resolution: --- → FIXED
(Assignee)

Comment 21

12 years ago
Comment on attachment 268866 [details] [diff] [review]
proposed fix

request 2.0.0.x approval - simple fix that only affects pop3 copy filter actions.
Attachment #268866 - Flags: approval1.8.1.5?
(Assignee)

Comment 22

12 years ago
What's still broken is when you have a pop3 filter that does a copy to a local folder followed by a move to a local folder. The copy happens asynchronously, but the move is synchronous, and clears the bits out from under the copy operation. That's Bug 376235, which is really unrelated to this issue.
(Assignee)

Updated

12 years ago
No longer blocks: 376235
(In reply to comment #20)
> fixed on trunk - only fixes global inbox case.
Is my non Global Inbox case covered by other bug?
(Assignee)

Comment 24

12 years ago
WADA, I can't really say. Can you try a build from tomorrow or later and see if it works? Do you have any filter that does a move of the message that your test filter is doing a copy of? It doesn't have to be the same filter...
(In reply to comment #24)
> Do you have any filter that does a move of the message (snip)
No. I tested with "Add Tag" and "Copy to folder" only.
I'll test build your patch is landed on, and feed back to you.
To David Bienvenu:
Question before test - can "quarantine" option have relation to problem?
Should I test both quarantine=Off case and quarantine=On case?
(Assignee)

Comment 27

12 years ago
quarantining is definitely one variable in the equation, so testing both cases would be good.

Comment 28

12 years ago
Comment on attachment 268866 [details] [diff] [review]
proposed fix

a=mscott for mailnews only change to 1.8.1.5.
Attachment #268866 - Flags: approval1.8.1.5? → approval1.8.1.5+
(Assignee)

Comment 29

12 years ago
fixed for 1.8.1.5
Keywords: fixed1.8.1.5
Tested with Tb version 3.0a1pre (2007070804) (Win-XP SP2).
(Case-1 : quarantine option =OFF)
   Two mails were copied by filter successfully. No lock of folders was observed.
   => VERIFIED
(Case-2 : quarantine option =ON)
   Two mails were NOT copied by filter. No message filter log.
   => Re-open
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Additional result.
"Run Filter on Folder" works well even after problem of Case-2.
Duplicate of this bug: 368383
(Assignee)

Comment 33

12 years ago
I think the quarantine case is a different issue/bug. Did it work before this change?
(Assignee)

Comment 34

12 years ago
I'm going to re-resolve this as fixed, and open a different bug for the quarantine case.
Status: REOPENED → RESOLVED
Last Resolved: 12 years ago12 years ago
Resolution: --- → FIXED
(Assignee)

Comment 35

12 years ago
see Bug 387361  for the quarantine case.
Changing to VERIFIED, according to result of quarantine=OFF case in comment #30.
Status: RESOLVED → VERIFIED
(Assignee)

Updated

12 years ago
Duplicate of this bug: 368383
verified fixex 1.8.1.5 using Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.5) Gecko/20070716 Thunderbird/2.0.0.5 ID:2007071611 (Thunderbird 2.0.0.5 RC1) and the steps to reproduce from this bug. 
Keywords: fixed1.8.1.5 → verified1.8.1.5

Comment 39

12 years ago
I seem to have this problem but i will state it in my own words:
i cannot delete this folder i cannot copy to it i cannot move it
 
the mail sort filter gives an error:
ALERT!  The messages could not be moved or copied to the folder(foldername)because writing to folder failed.To gain diskspace....
I tried the remedy but disk space is not the issue.

I now have this issue with thunderbird 2 as well
I can rename the folder but the renamed folder cannot be moved or deleted or repositioned either!
It is listed as fixed but it still cannot be gotten rid of.
the file is effectively locked.
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.