Last Comment Bug 751562 - Filters which move emails to folders broke after 12.0.1
: Filters which move emails to folders broke after 12.0.1
Status: VERIFIED FIXED
: regression
Product: MailNews Core
Classification: Components
Component: Filters (show other bugs)
: 12
: x86_64 Windows 7
: -- normal with 2 votes (vote)
: Thunderbird 16.0
Assigned To: David :Bienvenu
:
Mentors:
: 752866 757331 776826 (view as bug list)
Depends on: 714364 1070646
Blocks:
  Show dependency treegraph
 
Reported: 2012-05-03 07:16 PDT by bugzilla
Modified: 2015-05-02 13:18 PDT (History)
15 users (show)
vseerror: in‑testsuite?
vseerror: in‑litmus?
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
fixed
fixed


Attachments
Thunderbird error console values taken after the filter error message is shown (66.27 KB, image/jpeg)
2012-05-03 07:16 PDT, bugzilla
no flags Details
proposed fix (3.12 KB, patch)
2012-06-22 14:31 PDT, David :Bienvenu
neil: review+
Details | Diff | Review
fix addressing Neil's comments, for trunk. (3.13 KB, patch)
2012-06-29 07:31 PDT, David :Bienvenu
no flags Details | Diff | Review
patch for aurora (probably applies to beta as well) (3.97 KB, patch)
2012-07-02 10:13 PDT, David :Bienvenu
standard8: approval‑comm‑aurora+
standard8: approval‑comm‑beta+
Details | Diff | Review

Description bugzilla 2012-05-03 07:16:59 PDT
Created attachment 620691 [details]
Thunderbird error console values taken after the filter error message is shown

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0
Build ID: 20120420145725

Steps to reproduce:

Accepted the upgrade to 12.0.1 for Thunderbird on 04/30, after which filters for my POP3 account which move incoming emails to different subfolders are triggered for "organizing" my incoming emails on each restart of the Thunderbird program.


Actual results:

The filters which move incoming POP3 emails to different subfolders in Thunderbird are now having problems locating those subfolders since my upgrade to 12.0.1.

The error message reads:

"The folder 'Folder1' could not be found, so filter(s) associated with this folder will be disabled. Verify that the folder exists, and that filters point to a valid destination file."

After receiving the above message when a filter for "Folder1" meets the criteria to move an incoming email to it, I can Edit that filter (which is now disabled), see that the previously saved "Folder1" selection for the "Move to" operation has been blanked out, then reselect "Folder1" and save the filter again. After re-enabling the filter via the filter list checkbox, I can run it manually and it works fine during that Thunderbird session.

However, if I close Thunderbird and reopen it, if that re-enabled filter is triggered for a new incoming email, I will receive the same error message shown above and its folder selection field value will once again be blanked out.

Also noticed a potentially more general issue with filters and folder values within them on 12.0.1:

If I open "Tools" -> "Message Filters" after restarting Thunderbird, any filter which moves files to a folder shows a blank folder value when first I "Edit" that filter. But, closing that filter edit dialog and then re-hitting the "Edit" button from the filter list shows the appropriate folder name in the "Move to" field for that same filter.


Expected results:

The filters which move incoming POP3 emails to different subfolders in Thunderbird should actually move the new emails to folders when a respective filter's criteria gets triggered (e.g., match a From address value, etc.).

I regressed my Thunderbird install to 11.0.1 + Lightning (my only extension) to 1.3 and the filter behavior was corrected: filters properly moved incoming emails to their respective folders.

I don't know enough about the internals of Thunderbird to be sure, but is my problem reported here at all related to https://bugzilla.mozilla.org/show_bug.cgi?id=748090 ?
Comment 1 :aceman 2012-05-03 08:26:44 PDT
That bug is fixed in 12.0.1 so it should be different. However, your problem may is similar to bug 714364. That bug just does not mention the folder targets gettting corrupted (blank). And the fix there is just to enable the filters again (not fix the targets).

The error console seems OK. Only the gloda error is interesting, why would it fail at move/copy. Can you try right-click on a folder (that has the problem) then Properties -> Repair folder?
Comment 2 bugzilla 2012-05-03 10:43:38 PDT
Bug 714364 does seem similar in the blanking of their Folders field value, but it mentions that incoming emails are correctly routed before the field blanking seems to occur.

My error condition prevents incoming emails from being moved correctly.  Also, editing and re-enabling the affected filter in my error situation provides a temporary fix: while Thunderbird remains running, the corrected filters operate as expected and their Folder field values look as expected, but after stopping and restarting Thunderbird the error condition occurs all over again and their folder values are blanked.

I attempted to "repair" one of the folders related to a filter that has given me this error message and subsequent tests still exhibited the same error I have reported.

Further exploration:

Testing just now confirmed that some other filters still work; I can't see any differences between the filter types and logical conditions that would make one filter work vs another in my setup.  They each search for different values in order to sort emails into different folders.

I decided to Delete one of the problem filters and recreate it exactly as it was before.  Then, I enabled it and restarted Thunderbird.  The result was something a bit new in this case: no error message popped up on Thunderbird restart and the filter remained enabled, but the email landed in the Inbox because the filter's "Move Message to" value was blanked (as before).  I have since reselected the proper folder and saved that filter for a few restarts of Thunderbird and it gets blanked out at each startup (as before).  Older filters with this error condition still cause a message popup to occur, though.
Comment 3 :aceman 2012-05-03 11:29:46 PDT
Can you paste a sample filter definition (that misbehaves) from your msgFilterRules.dat file (it is stored along the files that contain your mail folders)?
It should look like this:
name="test name"
enabled="yes"
type="17"
action="Move to folder"
actionValue="mailbox://aceman@mail.sk/Inbox/Folder/test"
condition="AND (subject,contains,xyz)"
Comment 4 bugzilla 2012-05-03 11:47:07 PDT
Here is what one of the problem filters looks like after it has been correctly enabled and Thunderbird shut down:

name="Route American Express"
enabled="yes"
type="17"
action="Move to folder"
actionValue="mailbox://mymail@mymailserver.com/Use%20and%20Buy/Products-Services/AMEX"
condition="OR (from,contains,americanexpress.com) OR (from,contains,aexp.com)"


Here is how it appears after Thunderbird is restarted and the error has occurred:

name="Route American Express"
enabled="no"
type="17"
action="Move to folder"
actionValue="mailbox://mymail@mymailserver.com/Use%20and%20Buy/Products-Services/AMEX"
condition="OR (from,contains,americanexpress.com) OR (from,contains,aexp.com)"
Comment 5 :aceman 2012-05-03 12:03:27 PDT
And after you view this filter in the editor the folder picker (target) is blank?
Strange, it seems identical.

Can you try disabling the Global search and indexer in Preferences -> Advanced -> General ?
Comment 6 bugzilla 2012-05-03 12:26:43 PDT
Disabling that option causes no apparent difference in the error condition being raised, from what I can see.

Of note: the Folder field value is only turned off in msgFilterRules.dat after an email triggers the filter, it seems - just starting and stopping Thunderbird does not affect the filter's state.
Comment 7 David :Bienvenu 2012-05-03 12:27:13 PDT
If we don't think "Use%20and%20Buy/Products-Services/AMEX" exists when we try to run the filter, we'll disable the filter. Does the folder exist?
Comment 8 bugzilla 2012-05-03 16:24:27 PDT
I can confirm that the folder exists in Thunderbird and filter work fine with that value if I reselect the folder within the associated filter, then run that filter in the same Thundbird session.

This problem only occurs when filters run on incoming mail during a Thunderbird restart, it seems.

Overlaying my current install back to Thunderbird 11.0.1 with Lightning 1.3 works fine for the same filters (as have all past versions since I instantiated these filters+folders a couple years back).
Comment 9 WADA 2012-05-06 17:21:33 PDT
bugzilla@ooofest.com(bug opener), do you see error like following in Error Console?
> Discovering folders for account failed with exception: [Exception...
> "Component returned failure code: 0x80550013 [nsIMsgFolder.subFolders]"
> nsresult: "0x80550013 (<unknown>)"
> location: "JS frame :: resource:///modules/MailUtils.js :: MailUtils_discoverFolders :: line 77"  data: no]

If Tb profile was created by Tb 12.0.1 en-US and is used by en-US version only, above error was not shown. However, once the Tb profile was used by localized Tb(Tb 12.0.1 ja in my case) and folders were shown in localized folder name, above error was shown for each account except Local Folders upon restart of Tb by both Tb en-US and Tb ja. If panacea.dat is deleted and all .msf of all Tb's special folders(Inbox.msf, Trash.msf, ...) are deleted and Tb en-US is restarted, all special folders were shown in original English name again and above error disappeared.

Above error doesn't seem to affect ordinal filter execution, but it may affect on filter execution in some circumstances.
Do you use localized Tb?
Did you use your Tb profile by Beta/Aurora/Trunk daily builds?

Do you see your problem with Tb's -safe-mode?
(thunderbird.exe -safe-mode. no Lightning) 

Do you see your problem with newly created profile by Tb 12 and a problematic POP3 account only?
Please don't forget to set "Leave Messages on Server" and to disable any auomatic mail deletion in your test.
Comment 10 WADA 2012-05-06 17:39:49 PDT
(In reply to bugzilla from comment #0)
> However, if I close Thunderbird and reopen it, if that re-enabled filter is
> triggered for a new incoming email, I will receive the same error message
> shown above and its folder selection field value will once again be blanked
> out.

If auto-compact is enabled and is executed without prompt(mail.purge.ask=false), big different between "just after restart of Tb" and "after using Tb for a while" is "silent execution of auto-compact upon first mail download after restart and filter move". Once auto-compact is invoked, Tb won't schedule auto-compact again for an hour by "1 hour timer". This timer is reset to ZERO by restart of Tb. 
If auto-compact is invoked by filter move, and if auto-compact & filer move for other mail is executed cocurrently, filter move action may be interfered by auto-compact. And it may cause internal filter execution error and it may produce faked "folder not found" condition.  

Do you enable auto-compact? (Tools/Options/Advanced/Network&Disk Space, Disk Space)
If yes, is mail.purge.ask=true used? (via Tools/Options/Advanced/General, Config Editor)
Comment 11 bugzilla 2012-05-07 06:27:28 PDT
(In reply to WADA from comment #9)
> bugzilla@ooofest.com(bug opener), do you see error like following in Error
> Console?
> > Discovering folders for account failed with exception: [Exception...
> > "Component returned failure code: 0x80550013 [nsIMsgFolder.subFolders]"

I do not see code 0x80550013 in the error log or dialogs that pop up when this error condition occurs.

> If Tb profile was created by Tb 12.0.1 en-US and is used by en-US version
> only, above error was not shown. However, once the Tb profile was used by
> localized Tb(Tb 12.0.1 ja in my case) and folders were shown in localized
> folder name, above error was shown for each account except Local Folders
> upon restart of Tb by both Tb en-US and Tb ja. If panacea.dat is deleted and
> all .msf of all Tb's special folders(Inbox.msf, Trash.msf, ...) are deleted
> and Tb en-US is restarted, all special folders were shown in original
> English name again and above error disappeared.

I am temporarily running an US system with Japan Locale settings (to help display proper fonts for another program), so deleted panacea.dat and all .msf files, then restart Thunderbird - same errors occurred.  I tried deleting and recreating one of the problem filters that cause this error condition and reset all conditions, but again the same error occurred.
 
> Do you use localized Tb?

I use an English Thunderbird, but it was upgraded to 12.0.1 on this Locale = Japan setting.  Thunderbird has been upgraded many times on this same temporary Locale setting (i.e., I switch back and forth).

> Did you use your Tb profile by Beta/Aurora/Trunk daily builds?

No, only the major releases that auto-update provide.

> Do you see your problem with Tb's -safe-mode?

Yes.

> Do you see your problem with newly created profile by Tb 12 and a
> problematic POP3 account only?

My profile was created before Thunderbird 12.0.1 and I only use POP3 with this account.

> Please don't forget to set "Leave Messages on Server" and to disable any
> auomatic mail deletion in your test.

OK - I have been recreating some of the trigger conditions by sending emails with specific subject values to help testing move along, but can leave messages on the server for now.
Comment 12 bugzilla 2012-05-07 06:32:53 PDT
(In reply to WADA from comment #10)
> Do you enable auto-compact? (Tools/Options/Advanced/Network&Disk Space, Disk
> Space)

Yes, for when it detects that at least 20MB can be saved.

> If yes, is mail.purge.ask=true used? (via Tools/Options/Advanced/General,
> Config Editor)

This value was "false", so I set it to "true" and retried the test, but the same error condition occurred.
Comment 13 bugzilla 2012-05-09 06:43:07 PDT
For what it's worth, without changing anything else in my system, I reverted back to Thunderbird 11.0.1 and Lightning 1.3 without making any changes, re-indexing, etc. and all looks + works as expected when it comes to the filters moving new mail into specified folders.
Comment 14 :aceman 2012-05-09 08:10:57 PDT
(In reply to David :Bienvenu from comment #7)
> If we don't think "Use%20and%20Buy/Products-Services/AMEX" exists when we
> try to run the filter, we'll disable the filter. Does the folder exist?

Could this be something like exiting on SetSpec() failure? Something what bug 186724 changed, but that only landed in TB13 so there would be an unexplained discrepancy.
Comment 15 :aceman 2012-05-28 01:56:08 PDT
What is the language of the user interface of your Thunderbird? Do you use a version other than English?
Comment 16 bugzilla 2012-05-29 22:40:53 PDT
(In reply to :aceman from comment #15)
> What is the language of the user interface of your Thunderbird? Do you use a
> version other than English?

The language is English on Windows 7 Home Edition, though I've used a different Locale setting at various times to better support some specific applications (i.e., not Thunderbird).
Comment 17 :aceman 2012-05-29 22:47:26 PDT
So you downloaded the en-US version of Thunderbird?
Comment 18 bugzilla 2012-05-29 23:07:30 PDT
(In reply to :aceman from comment #17)
> So you downloaded the en-US version of Thunderbird?

Yes.
Comment 19 alt4moz 2012-06-19 14:06:44 PDT
Same happens for me on Win7 32-bit and on Linux 64-bit (Kubuntu 12.04 with TB 13.01, mozilla distribution).
On Win7 I reverted to 11.0.1 and then the problem is not there.
On Kubuntu I installed TB 13.0.1 and then copied my saved Win7 64-bit TB profile.

This profile (the one now used on Kubuntu), is a different one than the one used on Win7 32-bit. It was in an earlier incarnation copied from WinXP 32-bit to Win7 64-bit. On that Win7 64-bit system the same problem with the filters also occurred, and was no longer there when reverting back to TB 11.0.1. On Kubutu, with TB 13.0.1 the problem is there again.
Comment 20 alt4moz 2012-06-20 05:45:42 PDT
Additional (to the previous comment) observation on the Linux 64-bit system: After opening TB, and attempting to move a message using the move button, sub-folders are not accessible in that menu. The sub-folders only become accessible once a message somewhere lower down in the tree has been opened.

Perhaps the issue for which this is a comment is related to that phenomenon. It looks as if TB is lazy in opening and listing available folders. While that is good in some situations, perhaps it is slightly too lazy.
Comment 21 :aceman 2012-06-20 06:04:43 PDT
I have observed something similar while changing code in the Extra Folder Columns extension. I loop over the subfolders of a folder using .subFolders and then .getNext() until .hasMoreElements() is false. This works for the 1st level of folders. However sometimes when getting into lower levels (2nd and more) the .getNext() call returns "undefined" so I can't get the properties of the folder I want. It happens on folders that are not yet visible in the folder pane, because their parent is collapsed. After uncollapsing the parrent these bad folders start to work fine and also after they are collapsed again.

Also, the folders are instantaneously fixed if in some account I open the Copy to menu on a msg and expand the list to see the problematic folders. It seems TB now fetches their data. After that they work fine in the loop I mentioned.

So it really looks like TB is not fetching folder properties even when they are requested in some cases. This is on TB 16a1.
Comment 22 :aceman 2012-06-20 06:09:25 PDT
To those who see the problem with filters:
Do you have all the target folders specified in the misbehaving filters visible/expanded in the left folder tree pane? Or are they collapsed (their parents have the + signed grippy)?
Comment 23 bugzilla 2012-06-20 06:59:11 PDT
My folders are typically collapsed during each Thunderbird restart, when the filters run on new, incoming email.
Comment 24 :aceman 2012-06-20 07:23:02 PDT
Can you try leaving all the folders expanded (showing the "-" sign grippy) before TB restart?
Comment 25 bugzilla 2012-06-20 08:08:27 PDT
. . . and, that was key: testing on v13.0.1 does reveal that the filters cannot find my subfolder when Thunderbird starts with all folders collapsed, which causes the error condition I reported at the beginning of this bug report.

However, if first I expand the folder tree to target folders for my filters, then restart Thunderbird, newly incoming emails on TB startup route correctly to those folders via the filters and the filters remain enabled.

Good find.
Comment 26 David :Bienvenu 2012-06-20 08:17:58 PDT
this implies that DiscoverSubFolders with deep = true isn't doing a true deep discovery for the berkeley mailbox store. I'll look into it.
Comment 27 alt4moz 2012-06-20 23:50:42 PDT
Confirming that with all folders expanded in the tree before restarting TB, both issues are gone. The filters work as they should, and so does moving messages.
I also typically restart TB with all folders collapsed.

This is in the Linux environment. Haven't tried the Windows environment, yet.
Comment 28 alt4moz 2012-06-21 01:16:41 PDT
Upgraded TB from 11.0.1 to 13.0.1 on Win7 32-bit. Same result: when starting TB with the trees expanded filters work alright, when starting TB with the trees collapsed the problem appears.
Comment 29 David :Bienvenu 2012-06-22 08:29:22 PDT
I've verified that DiscoverSubFolders with deep = true isn't doing a deep discovery. I couldn't reproduce the bug with a test profile because GetChildWithURI does a deep discovery as a side effect, which discovers all folders. In any case, this should be easy enough to fix by adding ".sbd" when trying to find child folders.
Comment 30 David :Bienvenu 2012-06-22 14:31:44 PDT
Created attachment 635919 [details] [diff] [review]
proposed fix

this fixes it for me. I just have to check that it doesn't break any of our existing tests, and see if I can add a unit test for it.
Comment 31 :aceman 2012-06-22 14:47:19 PDT
For some reason I can't reproduce the problem with the extension on my Linux test install of TB16. I could try it out later if there is a trybuild for Windows where I see the problem consistently.
Comment 32 :aceman 2012-06-24 13:45:53 PDT
OK, now I somehow got the problem on linux. And after applying the patch it gets even worse so it does not fix it.
I can't say if it fixes the original reporter's problem.
Comment 33 David :Bienvenu 2012-06-24 15:38:02 PDT
(In reply to :aceman from comment #32)
> OK, now I somehow got the problem on linux. And after applying the patch it
> gets even worse so it does not fix it.
> I can't say if it fixes the original reporter's problem.

What does worse mean? Filters that weren't disabled before are now disabled?
Comment 34 bugzilla 2012-06-24 16:41:48 PDT
Unfortunately, I don't have a local build environment for testing the proposed patch under Windows - is there a means for me to obtain this pre-built or should I look into establishing a development environment?
Comment 35 :aceman 2012-06-24 23:22:32 PDT
(In reply to bugzilla from comment #34)
> Unfortunately, I don't have a local build environment for testing the
> proposed patch under Windows - is there a means for me to obtain this
> pre-built or should I look into establishing a development environment?

David could produce you a trybuild for Windows.

(In reply to David :Bienvenu from comment #33)
> What does worse mean? Filters that weren't disabled before are now disabled?

My problem from comment 21 (which I suppose is caused by the same underlying issue) got worse. Some folders that were resolved fine before now fail (i.e. getNext() on subfolder iterator returns "undefined").
Comment 36 David :Bienvenu 2012-06-25 07:16:27 PDT
(In reply to :aceman from comment #21)
> I have observed something similar while changing code in the Extra Folder
> Columns extension. I loop over the subfolders of a folder using .subFolders
> and then .getNext() until .hasMoreElements() is false. 
Are you checking if a folder has subfolders before calling subFolders and getNext and hasMoreElements?
Comment 37 :aceman 2012-06-25 07:21:43 PDT
Yes, this is the code:
if (aFolder.hasSubFolders) {
  let subFolders = aFolder.subFolders;
  while (subFolders.hasMoreElements()) {
    let subFolder = subFolders.getNext();
    ... do stuff on subFolder and it throws as subFolder is sometimes "undefined" ...
  }
}
Comment 38 David :Bienvenu 2012-06-25 07:36:28 PDT
(In reply to :aceman from comment #37)
> Yes, this is the code:
> if (aFolder.hasSubFolders) {
>   let subFolders = aFolder.subFolders;
>   while (subFolders.hasMoreElements()) {
>     let subFolder = subFolders.getNext();
>     ... do stuff on subFolder and it throws as subFolder is sometimes
> "undefined" ...
>   }
> }

I'm really hard-pressed to see how that issue is related to this issue. Are you calling QI(Ci.nsIMsgFolder) on the subFolder before doing anything with it? And you don't have to check if a folder has sub folders, as long as it's while (hasMoreElements) instead of do { } while(hasMoreElements) - it was unclear from your previous comment.

Is it the first time through the loop that you're getting undefined, or are there sometimes subfolders and then the last one is undefined? dumping the aFolder.URI and then subFolder.URI might be helpful.
Comment 39 :aceman 2012-06-25 07:50:37 PDT
I do no QueryInterface(Ci.nsIMsgFolder) on subfolder, where should I add it?

The relation to this issue is this:
The undefined entries go away as soon as I expand some parent of the problematic folder.

The dump is like this:
Whole account is collapsed.

Timestamp: 25.6.2012 16:46:07
Error: mailbox://nobody@Local%20Folders/test6
Source File: chrome://extra-cols/content/main.js
Line: 69

Timestamp: 25.6.2012 16:46:07
Error: undefined
Source File: chrome://extra-cols/content/main.js
Line: 74

Timestamp: 25.6.2012 16:46:07
Error: undefined
Source File: chrome://extra-cols/content/main.js
Line: 69

(Line 69 is aFolder.URI, line 74 is subfolder.URI)
There is "test61" folder in "test6", and also "test611" in "test61". The "test611" is not even attempted as test61 fails. The loop runs fine as soon as I expand Local folders account.
Comment 40 David :Bienvenu 2012-06-25 07:52:17 PDT
let subFolder = subFolders.getNext().QueryInterface(Components.interfaces.nsIMsgFolder);
Comment 41 :aceman 2012-06-25 08:03:50 PDT
Well, that does help!
Why is it needed only starting at folders 2 levels deep?
Comment 42 David :Bienvenu 2012-06-25 08:12:58 PDT
(In reply to :aceman from comment #41)
> Well, that does help!
> Why is it needed only starting at folders 2 levels deep?

Maybe some xpcom wrapper caching? I really don't know.
Comment 43 :aceman 2012-06-25 08:21:38 PDT
Ok, thanks. So just ignore my comments on this bug starting at comment 21:)

I think I also see the problem reported here (after start of TB16, many of my filters have blank folder targets shown in the filter editor), but I have not yet looked into it if it is this bug or some of my other development.

Can you make a try-build for the guy wanting to test the patch on Windows?
Comment 44 David :Bienvenu 2012-06-25 08:35:13 PDT
(In reply to :aceman from comment #43)
> Can you make a try-build for the guy wanting to test the patch on Windows?

While I was trying to figure out your issue, I was working on a unit test for the bug and found some other odd things about folder discovery. I'm going to try to reproduce the issue inside TB before doing a try server build.
Comment 45 David :Bienvenu 2012-06-25 11:24:54 PDT
OK, I've finally reproduced this locally. It's a bit tricky because the following conditions need to be met:

The top level parent has to start with a name alphabetically after INBOX, or else looking for the INBOX causes us to discover all folders, including deeply nested children, before the INBOX.

All special folders in the account must exist (e.g., Sent, Drafts, Templates, Junk) if there are prefs that say to use those special folders, because if they don't exist, we conduct a rather exhaustive search looking for them, which causes us to discover all folders.
Comment 46 David :Bienvenu 2012-06-25 15:02:01 PDT
windows try server build with my patch should show up here: http://ftp.mozilla.org/pub/mozilla.org/thunderbird/try-builds/bienvenu@nventure.com-47091d294f1a
Comment 47 bugzilla 2012-06-25 19:48:44 PDT
David, I am sorry for being unfamiliar with the build deliver system here, but I tried

thunderbird-16.0a1.en-US.win32.installer.exe	25-Jun-2012 16:15

at

http://ftp.mozilla.org/pub/mozilla.org/thunderbird/try-builds/bienvenu@nventure.com-47091d294f1a/try-comm-central-win32/

and was able to recreate the same error condition which led off this bug report.

Should I be using a different daily build, perhaps?
Comment 48 David :Bienvenu 2012-06-25 21:10:56 PDT
it doesn't look like my patch got pushed to that try build. I'll try again tomorrow.
Comment 49 David :Bienvenu 2012-06-26 06:55:09 PDT
Sorry for the confusion; I think this build will have my fix:

http://ftp.mozilla.org/pub/mozilla.org/thunderbird/try-builds/bienvenu@nventure.com-b660ac57bd00
Comment 50 bugzilla 2012-06-26 07:59:09 PDT
Thanks David - unfortunately, that directory does not yet seem available, so I'll check back periodically.
Comment 51 David :Bienvenu 2012-06-26 08:02:11 PDT
(In reply to bugzilla from comment #50)
> Thanks David - unfortunately, that directory does not yet seem available, so
> I'll check back periodically.

sorry, I should have said it should be there in an hour or two.
Comment 52 David :Bienvenu 2012-06-26 11:48:00 PDT
(In reply to David :Bienvenu from comment #49)
third time's the charm? http://ftp.mozilla.org/pub/mozilla.org/thunderbird/try-builds/bienvenu@nventure.com-8e206f657a87 - should be up in an hour or two.
Comment 53 :aceman 2012-06-26 12:12:51 PDT
(In reply to :aceman from comment #43)
> I think I also see the problem reported here (after start of TB16, many of
> my filters have blank folder targets shown in the filter editor), but I have
> not yet looked into it if it is this bug or some of my other development.

This manifests itself like this:
After start of TB I go into filter editor and Edit a filter that has a target in a collapsed folder. The folderpicker at the Copy action is blank. I close the dialog and Edit the filter again. Now the target appears correctly.

David, your patch seems to fix this problem for me (compiled myself on trunk on Linux). After the patch the target folder shows up at first open of filter edit dialog. Let's see what the reporter sees.
Comment 54 :aceman 2012-06-26 12:13:27 PDT
Comment on attachment 620691 [details]
Thunderbird error console values taken after the filter error message is shown

This attachment does not seem to contain anything helpful.
Comment 56 bugzilla 2012-06-26 15:31:04 PDT
David, the build at

http://ftp.mozilla.org/pub/mozilla.org/thunderbird/try-builds/bienvenu@nventure.com-8e206f657a87/try-comm-central-win32/

worked normally in my testing and I could not reproduce the error condition which led to this bug report.

Looks good, thank you.
Comment 57 neil@parkwaycc.co.uk 2012-06-28 00:27:32 PDT
Comment on attachment 635919 [details] [diff] [review]
proposed fix

>-  nsresult AddSubFolders(nsIMsgFolder *parent, nsIFile *path, bool deep);
>+  nsresult AddSubFolders(nsIMsgFolder *parent, nsCOMPtr<nsIFile> &path, bool deep);
Since it wasn't clear from the quick skim of the patch that was all that I have time for right now, why this change?
Comment 58 David :Bienvenu 2012-06-28 06:41:45 PDT
(In reply to neil@parkwaycc.co.uk from comment #57)
> Comment on attachment 635919 [details] [diff] [review]
> proposed fix
> 
> >-  nsresult AddSubFolders(nsIMsgFolder *parent, nsIFile *path, bool deep);
> >+  nsresult AddSubFolders(nsIMsgFolder *parent, nsCOMPtr<nsIFile> &path, bool deep);
> Since it wasn't clear from the quick skim of the patch that was all that I
> have time for right now, why this change?

because I wanted to do

+    path = do_QueryInterface(dirFile);

and I already had a comptr handy...
Comment 59 neil@parkwaycc.co.uk 2012-06-28 12:13:55 PDT
(In reply to David Bienvenu from comment #58)
>  I wanted to do
> 
> +    path = do_QueryInterface(dirFile);

You hadn't noticed that we'd got rid of nsILocalFile?
Comment 60 neil@parkwaycc.co.uk 2012-06-28 12:39:36 PDT
Oh, I guess this needs to go on branches too...
Comment 61 neil@parkwaycc.co.uk 2012-06-28 12:40:59 PDT
Comment on attachment 635919 [details] [diff] [review]
proposed fix

r=me for branches which still use nsILocalFile, but for trunk, you could try using something like this:

nsresult rv;
nsCOMPtr<nsIFile> tmp; // at top level so we can safely assign to path
bool isDirectory;
path->IsDirectory(&isDirectory);
if (!isDirectory)
{
  rv = path->Clone(getter_AddRefs(tmp));
  NS_ENSURE_SUCCESS(rv, rb);
  path = tmp;
  nsAutoString leafName;
  path->GetLeafName(leafName);
etc.

But please remove the do_QueryInterface on trunk either way!
Comment 62 David :Bienvenu 2012-06-29 07:31:20 PDT
Created attachment 637903 [details] [diff] [review]
fix addressing Neil's comments, for trunk.
Comment 63 David :Bienvenu 2012-06-29 14:43:48 PDT
http://hg.mozilla.org/comm-central/rev/304b06e5a6d4
Comment 64 David :Bienvenu 2012-07-02 10:13:28 PDT
Created attachment 638402 [details] [diff] [review]
patch for aurora (probably applies to beta as well)
Comment 65 Mark Banner (:standard8) 2012-07-03 04:27:43 PDT
Comment on attachment 638402 [details] [diff] [review]
patch for aurora (probably applies to beta as well)

[Triage Comment]
a=me for Aurora & beta, as I think we should fix this regression for the next release.
Comment 68 bugzilla 2012-07-19 06:03:23 PDT
For what it's worth, I can confirm that this fix worked after the v14.0 update automatically installed for my Thunderbird client today.

Thanks again!
Comment 69 :aceman 2012-07-19 06:12:15 PDT
Thanks for the confirmation.
Comment 70 :aceman 2012-07-24 07:05:24 PDT
*** Bug 776826 has been marked as a duplicate of this bug. ***
Comment 71 :aceman 2012-07-24 22:35:40 PDT
*** Bug 752866 has been marked as a duplicate of this bug. ***
Comment 72 :aceman 2012-07-24 22:47:23 PDT
*** Bug 757331 has been marked as a duplicate of this bug. ***
Comment 73 Wayne Mery (:wsmwk, NI for questions) 2013-02-01 06:46:58 PST
*** Bug 755317 has been marked as a duplicate of this bug. ***
Comment 74 John Major 2014-02-06 15:47:12 PST
There seems to have been some sort of reversion. 
I am having this problem with 24.1.0 and have been for some time.
I am sorry, I don't know exactly when it started.
I am using IMAP over SSL and the folder names meet the conditions outlined by David in comment 45.
System is Fedora 19 x64.

If I re-enable the filter by ticking the check box, without expanding the offending target folder, and re-run the filters, the filter is disabled again. If I expand and collapse the target folders parent, All is ok again. TB runs 24/7 on this desktop and the problem reappears periodically. Not sure what period yet.

Is there an option to say "Leave my filter settings alone. I know better than you what folders I have." ? This feature must be very annoying for road warriors who occasionally don't have access to some target folders, but would like the filters to just work next time they do.
Comment 75 :aceman 2014-02-07 00:35:53 PST
It seems that the unit test from comment 44 was never attached so we do not detect regressions. David, would you be able to find it?
Comment 76 Wayne Mery (:wsmwk, NI for questions) 2014-09-01 05:25:12 PDT
(In reply to :aceman from comment #75)
> It seems that the unit test from comment 44 was never attached so we do not
> detect regressions. David, would you be able to find it?

Bienvenu isn't active.  But this seems pretty important. Can you sort it out with someone else's help?
Comment 77 :aceman 2014-09-20 13:10:12 PDT
No, I have no idea what Bienvenu had in mind.
Comment 78 David :Bienvenu 2014-09-21 10:43:06 PDT
From my reading of this bug, it was about local folders, and the most recent reporter is using IMAP. Folder discovery is fairly different for IMAP. My first question would be whether jcmajor is using IMAP subscription or not. My guess is not, which opens up a whole raft of issues because we don't build up the whole folder tree with an LSUB of all folders.
Comment 79 John Major 2014-09-21 18:06:11 PDT
In case it helps, here are some things that may or may not be playing a role here.

- I am using IMAP over SSL against dovecot + maildir. 
- The doveot is on ubuntu.
- I have changed distro from fedora to ubuntu 14 and the problem occurs less often but persists.
- For virtualisation, I have my eth port bridged (ie outside the control of NetworkManager) and this causes NetworkManager to misrepresent the connected status.
- I have not explicitly subscribed to any folders, however a subscription file does appear in my dovecot mail store. The entries in this file reference all folders and subfolders in the mail store, including the ones that vanish. The entries do not appear to be sorted at all except possibly in accidental order of folder creation.

I am following this thread, so feel free to request any further info that helps or to test something and I'll do my best.

At the very least, if it could stop changing my config for me, that would significantly reduce the irritation.

Regards
John
Comment 80 :aceman 2014-09-21 23:42:32 PDT
(In reply to David :Bienvenu from comment #78)
> From my reading of this bug, it was about local folders, and the most recent
> reporter is using IMAP. Folder discovery is fairly different for IMAP. My
> first question would be whether jcmajor is using IMAP subscription or not.
> My guess is not, which opens up a whole raft of issues because we don't
> build up the whole folder tree with an LSUB of all folders.

Thanks, but I was more interested in your comment 44, where you mention a unit test for the fix you have done here. Just whether you have the test lying around, because it seems you never attached it.
Comment 81 David :Bienvenu 2014-09-23 12:48:36 PDT
I started work on a unit test, but I don't have it anymore - My hard drive was wiped when I left and I wasn't able to save all my different trees.

And again, that unit test was for local mail folders, not imap folders, which require a completely different setup.

Note You need to log in before you can comment on or make changes to this bug.