Closed Bug 1367438 Opened 9 years ago Closed 7 years ago

Message Filters no longer run automatically in 52.1.1

Categories

(Thunderbird :: Filters, defect)

Product:
Thunderbird
Component:
Filters
Version:
52 Branch
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(thunderbird_esr52? affected)

Status:
RESOLVED INCOMPLETE
Tracking Flags:
Tracking Status
thunderbird_esr52 ? affected

People

(Reporter: dragons4, Unassigned)

References

(
URL
)

Details

(Keywords: regression, regressionwindow-wanted, Whiteboard: [regression:TB52.1.1])

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/7.1.2 Safari/537.85.11 Steps to reproduce: 1) Upgraded Thunderbird from 45.8.0 (message filters working as expected) to Thunderbird 52.1.1 Actual results: Message filters still work, but message filters no longer being run autumatically (I did not change anything other than upgrading Thunderbird, so no reason for action of Message Filters to change) I receive new mail, and it sits in the mail folder at charter.net, my internet provider. I move mail manually from Inbox on charter.net to Local Folders Inbox. Or, I have to open the Inbox on dragons4@charter.net (click triangle to expand dragons4@charter.net, select Inbox, click on Tools and then select Run Filters on Folder). Messages in either case are moved from charter.net Inbox to Local Folders Inbox. All messages remain in my Inbox and do not go to the subfolders they should go to. If I select Tools then Run Filters on Folder, the filters work as they have up until this release and move messages to appropriate subfolders Expected results: I have a Message Filter that automatically moves email from the Inbox at Charter.net to my Local Folders Inbox. When I open a Thunderbird window & it checks for messages (or anytime that Thunderbird checks for messages), messages show up in the Charter.net inbox, then are moved in a blink of an eye to my Local Folders Inbox I have a set of Message Filters that move, for example, messages from [MyOlympus] a Yahoo group to a MyOlympus sub-folder. Previous to 52.1.1, this happened automatically. Now I have to select Tools and Run Filters on Folder for the filters to work. I have changed nothing else on my system except for updating Thunderbird. I have not made any recent changes to the filters I had set up.
(In reply to dragons4 from comment #0) > messages show up in the Charter.net inbox, then are moved in a blink of an > eye to my Local Folders Inbox Given that description, I think you're running the filter in question *after* the junk classification, right? Just as an aside, configuring the filters "before junk" makes the system more efficient, since the messages don't pass through the first inbox any more and go straight to the correct folder. I'm not aware of any changes in that area between TB 45 and TB 52, so perhaps there is an incompatible add-on. Can you please run TB with all add-ons disabled, see Help menu. Of course you need to try that before changing the filter definition from "after" to "before" if you decide to do so.
Same issue here, no add-ons at all. In the "Message Filters" dialogue, when I choose a folder from the "Choose Folder..." drop-down, whatever I choose is not remembered for next time I open the dialogue. v52.1.1 on Windows 7 (64bit).
(In reply to deKay from comment #2) > Same issue here, no add-ons at all. > > In the "Message Filters" dialogue, when I choose a folder from the "Choose > Folder..." drop-down, whatever I choose is not remembered for next time I > open the dialogue. > > v52.1.1 on Windows 7 (64bit). And to confirm, going back to 52.0.0 restores functionality.
Hmm, there is no "Choose Folder..." on the "Message Filters" dialogue, so I don't know what you're referring to. Can you attach a screenshot. Also, I'm not aware of any changes to message filters between 52.0 and 52.1.1.
(In reply to Jorg K (GMT+2) from comment #4) > Hmm, there is no "Choose Folder..." on the "Message Filters" dialogue, so I > don't know what you're referring to. Can you attach a screenshot. Also, I'm > not aware of any changes to message filters between 52.0 and 52.1.1. Not sure how to attach a screenshot, but here's a link: https://ibb.co/fAUCQv
You can attach a file above with "Attach File". Never mind, thanks for the screenshot, I see you meant the: Run selected filter(s) in folder: Choose Folder... And that folder gets forgotten in 52.1.1, you're saying. Let me check that.
Well, when you select "Tools > Message Filters" the "Run selected filter(s) in folder" should be pre-populated with the folder that was current when you selected the command. I can't even imagine circumstances where you'd even see "Choose Folder...". I can see this working in TB 52.1.1, en-US version. As I said before, we didn't change anything in this area from 52.0 to 52.1.1. Anyway, we should move this issue to another bug since this bug here is about filters not being run automatically as they should be.
Looking at if after what you've said, I think the Choose Folder thing isn't a bug - I was accessing Message Filters from the page that comes up when I select the root of the account in the left tree, so no folder would pre-populate. I mentioned it as I'd not realised you could do it via Tools, and my logic was that perhaps the filter wasn't running because for some reason it didn't know which folder to run it on (as the setting wasn't there). I see that's probably not the case now. All that said, I can still confirm the filters don't automatically apply on 52.1.1, but do in 52.0.
(In reply to deKay from comment #8) > All that said, I can still confirm the filters don't automatically apply on > 52.1.1, but do in 52.0. OK, now we need to know more details about the filters that don't apply. Are they defined for an IMAP account or a POP account, are they "before junk" or "after junk", see my comment #1. Also, to narrow it down a bit, how about the intermediate versions: 52.0.1 and 52.1.0? When did the problem start exactly? You can get those versions from: http://ftp.mozilla.org/pub/thunderbird/releases/52.0.1/ http://ftp.mozilla.org/pub/thunderbird/releases/52.1.0/ Maybe you can also paste the definition of the filter from msgFilterRules.dat which you can find in your profile in the folder for your account. Use "Help > Troubleshooting Information, Profile Folder" to open it.
OK, to answer your questions: It's an IMAP account, and the filters are applied before junk. v52.0.1 works v52.1.0 works v52.1.1 does not (all en-GB, if that matters) The problem started when I upgraded to v52.1.1, although I can't remember which version I upgraded from. Not v52.x, that much I'm sure of - something older. I've tried deleting and recreating the filter in case something broke in upgrade, but that didn't work. msgFilterRules.dat reads: version="9" logging="no" name="Match All Messages" enabled="yes" type="17" action="Copy to folder" actionValue="mailbox://nobody@Local%20Folders/Local" condition="ALL"
Thanks, I'll look into it. There were very few changes between 52.1.0 and 52.1.1. We basically only uplifted bug 1355350 which may have affected IMAP processing. So that rule basically moves all messages to a local folder, right?
(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #12) > (speculatively setting flags) Yes, very speculatively since I can't reproduce this. Neither with trunk nor with TB 52.1.1. I have this rule: name="All messages" enabled="yes" type="17" action="Move to folder" actionValue="mailbox://nobody@Local%20Folders/Mai%202" condition="ALL" and it works. I noticed that deKay has "Copy to folder", so I'll try this now ... works too. So can someone confirm this?
OK, it seems that if I completely wipe my Thunderbird profile, and set it up again from scratch, it works. It would appear that it's only upgrading to v52.1.1, and perhaps only upgrading from a certain version or versions that already has such a filter rule set, that causes it to break.
Thanks for the feedback. Yes, "fresh profile" is usually the ultimate test. However, there seems to be a gremlin in the box somewhere, so other people upgrading my have a problem. As I said, I created the filter according to your specification and didn't see a malfunction. Can you check whether your IMAP inbox where the message first arrives before it gets copied is selected for offline use (right-click on folder, Properties, Synchronisation). And if you don't select it for offline use, does it make a difference. It doesn't for me, but the change from 52.1.0 to 52.1.1 were in the processing of folders *not* selected for offline use.
It *is* selected for offline use. Setting it as *not* selected makes no difference - now, at least. But I've wiped my profile, as I said.
Oh, and as far as I'm aware, it was always set as selected for offline use.
Same problem here. Upgraded from 52.something to 52.1.1 and filters stopped applying automatically. Filters basically scanned mail in (IMAP) inbox (pre-spam) and moved messages some to local folders, based on title and/or sender criteria. They worked as expected, except that they stopped executing automatically when Thunderbird was updated to 52.1.1. They work if executed manually, though. No other changes were applied to any configuration as well. No add-ons either. Also, Thunderbird is configured to work ONline. I have detected this issue both at home and at work. There are a lot of installations affected in the company I work for currently, and this issue has created a little chaos here, since we rely heavily on filters to classify all the mail we get. And there's a lot of junk and alert-type emails in there. I can confirm that the issue has not been resolved as of 52.2.1. Wiping the profile is fine at home, but there are too many installations of Thunderbird where I work, so we have discarded that sollution. If you need more information to fix this please contact with me and I will try to gather it. I have access to plenty of affected installations as I said.
It's worth noting that configuring filters to be executed *after* spam filtering seems to fix the problem: they execute automatically again after configuring them that way.
OK, looks like we have enough complaints here to give this bug some attention. So the claim is that the filters don't run in TB 52.1.1 but they do run in TB 52.1.0 as per comment #10. Jose, can you confirm that? So 52.1.1 doesn't work, 52.1.0 does? You can download those version from here http://ftp.mozilla.org/pub/thunderbird/releases/52.1.0/ http://ftp.mozilla.org/pub/thunderbird/releases/52.1.1/ There are also http://ftp.mozilla.org/pub/thunderbird/releases/52.0/ and http://ftp.mozilla.org/pub/thunderbird/releases/52.0.1/ Before we go searching for the bug, it would be good to know which exact versions work or don't work. If the problem really occurred between 52.1.0 and 52.1.1 we'd be looking at these changes: https://hg.mozilla.org/releases/comm-esr52/pushloghtml?fromchange=755364c7305e&tochange=fa95e667c1ed There were some changes in IMAP related to message caching, but nothing related to filters.
These are the tests I've performed. I have done two series, with a new profile and with my existing one. I first installed 45.8.0 which seemed to work for everyone and then installed manually to the next versions one at a time. Then I created a new profile and made a new series of tests. Discovering that 52.0 fails was a little surprise, so I tested again with another version to check if I were doing things wrong, which didn't seem to be the case. Tests with an existing profile (created circa Oct 2016) 45.7.1 - OK <-- tested with this *after* testing 45.8.0 and 52.0, because 52.0 not working seemed inconsistent with previous feedback on this issue 45.8.0 - OK 52.0 - fails 52.0.1 - fails 52.1.0 - fails 52.1.1 - fails 52.2.0 - fails 52.2.1 - fails Tests with a new profile created today 45.8.0 - OK <-- profile created here 52.0 - OK 52.0.1 - OK 52.1.0 - OK 52.1.1 - OK 52.2.0 - OK 52.2.1 - OK Hope this is useful in any way. Please request any more info you may need.
Thanks for testing. The surprising result is that all versions work as they should on a new profile. The answer of the Firefox support team would be: Reset your browser, but for e-mail we don't have such an option and creating new profiles is hard since you need to manually move e-mail storage/folders/archives. 45.8.0 working on the old profile and 52.0 not working is really bad news since there are hundreds of changes between the two versions and it's quite impossible to track down the particular change that caused this in the hope of finding the setting in the profile which could be changed to "repair" existing profiles. All I can suggest is to take a look at the old profile in the config editor and see which preferences have been changed manually. Or diff the prefs.js files in the old and new profiles, that will be tedious since I'd expect many changes. Other thing you can do is bisect the problem with Daily versions of TB, so you grab the last TB 45 Daily and see when the problem was introduced. "Moz Regression", http://mozilla.github.io/mozregression/quickstart.html, can help, but it will be very time consuming and tedious. Since our software appears to work, perhaps with a small bug that under some circumstances the upgrade from TB 45 to TB 52 doesn't work as desired, I don't think we can dedicate a whole lot of resources to this. Have you checked https://support.mozilla.org/en-US/products/thunderbird for similar reports and possible solutions? I can see this: https://support.mozilla.org/en-US/questions/1161258 which doesn't appear to be helpful.
Thanks for your input on this. I understand that not a lot can be done here from your side, and I'm not high on time to go through all that extensive testing right now. After considering what you said, I will be wiping my profile at home and starting with a new one, since that's not a lot of hassle to me. At the company I'm working for currently, we have decided to do that as well for the people who need the filters but don't have any local content that we need to migrate manually. For those who have local content, we will be configuring filters to run after spam checks for now.
I'm pretty convinced that this won't help, but it's worth a shot: Set mail.imap.filter_on_new to false and see whether it makes a difference. That's the only thing filter-related that changed between 45 and 52 I can think of.
If mail.imap.filter_on_new proves inconclusive, and *before* you blow away the testcase profile... It really would be helpful if you can find the regression range with the mozregression tool, because if you are seeing a problem, then other users are also likely seeing a problem. It is truly easy to run this tool. If you are willing I can assist, and give your a starting range. > Since our software appears to work, perhaps with a small bug that under some circumstances the upgrade from TB 45 to TB 52 doesn't work as desired, I don't think we can dedicate a whole lot of resources to this. Well, it works under controlled conditions, but that's obviously not where all our users live. Plus, it fails for the user on *multiple machines. You really paint too much a pessimistic view Jorg. If we get a regression range from the user then we very likely get a potential solution, and no developer needs to dedicate much resources. But even so, you are also not the only resource here. So it's really too soon to give up on this when we have a testcase, plus an intelligent reporter, which as you know both are difficult to come by.
Component: Untriaged → Filters
Flags: needinfo?(dragons4)
(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #25) > You really paint too much a pessimistic view Jorg. Not really, I made some suggestions in comment #22: Compare preferences and find the regression. I could even take a look at the preferences if I were supplied a working and a non-working prefs.js via a private message. Rename the files to .txt before sending since some mail systems block .js attachments.
I experienced this problem yesterday and turned on the message filter log. The first line in the log is "Applied Filter" the second line is // Filter Action Failed: "Move failed" with error code=0x8055000f while attempting: Applied filter "S..." //. I saved the log to a text file. I have not tried any of the suggestions listed by others. Thought I would start over again and not install any add-ins initially. I wasn't sure if maybe the 'no longer supported' MozBackup was causing this. If I manually Run Filters on Folder (Inbox) the items get moved from the Inbox to the appropriate folder and marked read. As soon as I click on the folder the new items turn to bold characters as unread. I am using TB 54.0.1 (32-bit)on 64-bit, Windows 10 "Clean Install" performed yesterday. I have not had this problem previously and had performed a Windows 10 "Reset" that is *supposed* to keep personal files but removed all my Mozilla and other non-ms apps. This was during July 4 and the new TB install at that time did not have this problem. It was also migrated using MozBackup-1.5.1-EN.
Looks like 0x8055000f is NS_MSG_ERROR_WRITING_MAIL_FOLDER https://dxr.mozilla.org/comm-central/rev/18217986ac6a0d930b6dc7d693045c2eb7045b50/mailnews/base/public/msgCore.h#120 So as the log suggested, the move to this folder failed. It's only surprising that it works when running the filter manually, but that's a different code path.
See Also: → 1383101
(In reply to Jorg K (GMT+2) from comment #28) > Looks like 0x8055000f is NS_MSG_ERROR_WRITING_MAIL_FOLDER > https://dxr.mozilla.org/comm-central/rev/ > 18217986ac6a0d930b6dc7d693045c2eb7045b50/mailnews/base/public/msgCore.h#120 > > So as the log suggested, the move to this folder failed. It's only > surprising that it works when running the filter manually, but that's a > different code path. see bug 931303
see comment 19, comment 29
Severity: normal → major
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?(acelists)
Whiteboard: [regression:TB52.1.1]
No, this is not my territory :)
We're not going to have much fun with this here and there is little chance of finding anything. Look at comment #14 and #21 to see that this appears to be a profile problem. People experiencing the bug get going again with a new profile. So how do you expect to find the regression? A person who can reproduce the problem would have to bisect sending and receiving a message to be filtered upon receipt for every iteration.
sure, a bisect would be perfect. heck, might not even be successful bisecting nightlies. not all bugs are fun. bug 1410392 is another aceman for some reason your NI didn't clear
Flags: needinfo?(acelists)
Hello again, To add to (or confuse) things... this happened again. I'm now running 52.4.0, and for the last few days this same filter hasn't been triggering. Like last time, I deleted the profile and created a new one and it's working again.
Hello, I'm using 52.9.1 (Mac) and I have this problem. I'm not sure exactly when it started (version-wise), but it was definitely a few updates back. Filters don't work automatically. They work fine manually. Is there a solution to this problem yet (that doesn’t involve deleting/recreating my profile)? Thanks
Hello, I have had a similar problem with email filters not working correctly. To resolve the issue, I had to go to where my profile was stored, then look for the folders that won't accept emails. I found that the two folders that weren't accepting filtered emails were improperly formatted. The default subfolder looks to be untyped in my version, yet the two bugged ones were created as windows folders, instead. I deleted the old folders, restarted Thunderbird, and recreated the folders by right clicking the main folder, then selecting create subfolder. Then I changed the target of the filter to match the new folder name(just a double check). Now my emails filter correctly when I pull them in, and I can see them as unread in the right menu. Hope this helps. (In reply to dragons4 from comment #0) > User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) > AppleWebKit/600.2.5 (KHTML, like Gecko) Version/7.1.2 Safari/537.85.11 > > Steps to reproduce: > > 1) Upgraded Thunderbird from 45.8.0 (message filters working as expected) to > Thunderbird 52.1.1 > > > Actual results: > > Message filters still work, but message filters no longer being run > autumatically (I did not change anything other than upgrading Thunderbird, > so no reason for action of Message Filters to change) > > I receive new mail, and it sits in the mail folder at charter.net, my > internet provider. > > I move mail manually from Inbox on charter.net to Local Folders Inbox. Or, I > have to open the Inbox on dragons4@charter.net (click triangle to expand > dragons4@charter.net, select Inbox, click on Tools and then select Run > Filters on Folder). Messages in either case are moved from charter.net Inbox > to Local Folders Inbox. > > All messages remain in my Inbox and do not go to the subfolders they should > go to. If I select Tools then Run Filters on Folder, the filters work as > they have up until this release and move messages to appropriate subfolders > > > Expected results: > > I have a Message Filter that automatically moves email from the Inbox at > Charter.net to my Local Folders Inbox. When I open a Thunderbird window & it > checks for messages (or anytime that Thunderbird checks for messages), > messages show up in the Charter.net inbox, then are moved in a blink of an > eye to my Local Folders Inbox > > I have a set of Message Filters that move, for example, messages from > [MyOlympus] a Yahoo group to a MyOlympus sub-folder. Previous to 52.1.1, > this happened automatically. Now I have to select Tools and Run Filters on > Folder for the filters to work. > > I have changed nothing else on my system except for updating Thunderbird. I > have not made any recent changes to the filters I had set up.
(In reply to Ian Sullivan from comment #36) > Hello, I have had a similar problem with email filters not working > correctly. To resolve the issue, I had to go to where my profile was > stored, then look for the folders that won't accept emails. I found that > the two folders that weren't accepting filtered emails were improperly > formatted. The default subfolder looks to be untyped in my version, yet the > two bugged ones were created as windows folders, instead. I deleted the old > folders, restarted Thunderbird, and recreated the folders by right clicking > the main folder, then selecting create subfolder. Then I changed the target > of the filter to match the new folder name(just a double check). Now my > emails filter correctly when I pull them in, and I can see them as unread in > the right menu. > Ian, Were these local folders or IMAP folders? If the latter, when deleting the folders and restarting, didn't that recreate the folders as untyped? Or did they still look like Windows folders?
See Also: → 1394285
(In reply to sherman from comment #37) > (In reply to Ian Sullivan from comment #36) > > Hello, I have had a similar problem with email filters not working > > correctly. To resolve the issue, I had to go to where my profile was > > stored, then look for the folders that won't accept emails. I found that > > the two folders that weren't accepting filtered emails were improperly > > formatted. The default subfolder looks to be untyped in my version, yet the > > two bugged ones were created as windows folders, instead. I deleted the old > > folders, restarted Thunderbird, and recreated the folders by right clicking > > the main folder, then selecting create subfolder. Then I changed the target > > of the filter to match the new folder name(just a double check). Now my > > emails filter correctly when I pull them in, and I can see them as unread in > > the right menu. > > > > Ian, > > Were these local folders or IMAP folders? > > If the latter, when deleting the folders and restarting, didn't that > recreate the folders as untyped? Or did they still look like Windows folders? Sorry for the delay. Here is a broken down version of events that may clarify my previous post. 1. Closed Thunderbird 2. Went to the Local Folder for Thunderbird 3. Found two folders in Local Folder that were not working properly, Windows was reading the folder as a Windows folder, not an IMAP folder. 4. Deleted the two folders mentioned in #3. 5. Restarted Thunderbird. 6. Inside Thunderbird, I selected the main folder for my email account, and right clicked it. 7. Selected create subfolder from the drop-down menu. 8. Completed the subfolder creation process. 9. Set up a filter to dump particular emails into the new subfolder. 10.Verified filter was working appropriately. 11.Verified improperly typed folders didn't regenerate. 12.Works to this day. Have a happy holidays, Folks.
(In reply to deKay from comment #34) > Hello again, > > To add to (or confuse) things... this happened again. deKay, Do you still have the failing profile? Can you send someone the filters?
Flags: needinfo?(andyk)

(In reply to Wayne Mery (:wsmwk) from comment #40)

(In reply to deKay from comment #34)

Hello again,

To add to (or confuse) things... this happened again.

deKay,
Do you still have the failing profile?
Can you send someone the filters?

It's worked for over a year (since recreating the profile), so not any more, sorry.

Flags: needinfo?(andyk)

We don't have feedback from the original author, and this report has a wide variety of issues, so closing as incomplete. Also, IIRC In version 68 we will have better filter logging. So I suggest that further work on this be based on testing of version 68 which will soon be released as beta.

I am however curious about Ian's statements about "Windows was reading the folder as a Windows folder, not an IMAP folder." Please describe exactly what this means, i.e. for example what you see in the folder properties.

Status: NEW → RESOLVED
Closed: 7 years ago
Flags: needinfo?(dragons4) → needinfo?(ijsullivanbusiness)
Resolution: --- → INCOMPLETE
Flags: needinfo?(ijsullivanbusiness)
You need to log in before you can comment on or make changes to this bug.