1457667 - filters do not work on broken mails with 1/1/1970 date

Status: RESOLVED INVALID

(Thunderbird :: Filters, defect)

Description

[User Dderss]

Attachment: Thunderbolt filters of broken mail FAIL.png

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:59.0) Gecko/20100101 Firefox/59.0
Build ID: 20180323154952

Steps to reproduce:

To deal with the never-resolved #Bug 209501, which is about broken mail letters dated 01/01/1970 appearing, I created filters on all folders and mail accounts I have that would delete such junk automatically, without me doing it manually.


Actual results:

Instead of working as intended, nothing happens at all, and thus I have to still manually delete the broken mails.


Expected results:

Ideally, the #Bug 209501 should be fixed, but until then the filters should work. Otherwise, Thunderbolt becomes UNBEARABLE.

Comment 1

[User Dderss]

Attachment: Thunderbolt filter settings.png

The filter does not work whether it runs automatically or in cases when I ran it manually.

As it can be seen from the previous (the first) attached screenshot, the log for this filter on this mail account is empty no matter what, so it never triggers.

Comment 2

[Jorg K (CEST = GMT+2)]

Those phantom messages indicate a corrupt mailbox. It is not a surprise that filters don't work in that case.

I can only suggest to repair the folder (right-click, Properties, Repair Folder).

We would fix bug 209501 if we had a reproducible case. Personally, I've seen phantom messages in the context of "undo delete" (bug 209189) and in bug 1106225, and both bugs were fixed.

Comment 3

[User Dderss]

I have set up repair folder automation on every possible occasion, and it still does not help.

Also, Thunderbolt filters work locally, meaning that once a letter is fetched from a certain account, the application runs the filters. Yes, such letters are corrupt, but they are there -- the users can go to these letters, look at them, the letters have a concrete date assigned, so the filters should work, but they do not.

My case of the 1/1/1970 corrupt shadow/phantom mails is _kind of_ reproducible as I see them almost every day.

Is there a tool I could run on Thunderbolt mail database at the moment when it has a corrupt shadow/phantom letter (before I manually delete it)? 

BUT even with no such thing, it should not be hard for the Thunderbolt programmers at all to then detect such incomplete, corrupt letters and just never add them to the database to not create this junk. The easiest way would be to just insert a check for a date of the letter -- if the date is resolved into 1/1/1970, then just do not add such letter to the Incoming folder. 

Can you please do this? It is just a few lines of code (with an option in "about:config"), and it will make the lives of many people much easier in this aspect.

Comment 4

[Jorg K (CEST = GMT+2)]

Maybe we can start by naming the software correctly, it's called Thunder*bird*.

So the corrupt folders are local folders or IMAP folders? If they are small, you could provide a sample folder, that is raw message file together with MSF file, for inspection.

Are you using any add-ons?

Comment 5

[User Dderss]

1. Yes, I always confuse the name. ;)))) Thanks for the correction.

2. The folders are IMAP (but they are, of course, also local as everybody can see by turning on the off-line/autonomous mode).

3. I do not currently have a broken letter, so I can not check, but once it will happen again I will try to extract such corrupt letter if it is possible and reply to this thread. The folder, alas, might be just too big, but MSF should hopefully already expose the way how the letter is corrupted.

4. The list of my extensions is such:

-- Auto Select Latest Message (Automatically select and display the latest message on startup or when switching folder for the first time)

-- Color Folders (Color Folder Treeview Icons)

-- Enigmail (OpenPGP message encryption and authentication)

-- ImportExportTools (Adds tools to export/import messages and folders)

-- Lightning (Make Thunderbird or Seamonkey even more powerful with the fully integrated Lightning Calendar)

-- Manually sort folders

-- Xpunge (Xpunge is an extension that combines the functionality of emptying certain folders (trash and junk) and also compacting user-specified folders in a single action and for multiple accounts)

Comment 6

[User Dderss]

Attachment: View source of a broken 1-1-1970 letter.png

This is how an attempt to "View source" on a broken letter with 1/1/1970 date looks like.

Comment 7

[User Dderss]

BugZilla does not allow me to to attach a zero-length file, but this is exactly what I have got when I selected "Save" in the menu for "View Source" window -- the file got a crooked corrupt name, though: "INBOX)5546".

Comment 8

[Wayne Mery (:wsmwk)]

comment 0 suggest you a times see this problem frequently, but there have been no updates since you reported the bug.  You haven't seen this again since reporting the bug?

Comment 9

[User Dderss]

I am seeing this problem all the time, every day or almost every day.

However, the root cause for this is #bug 209501 which was confirmed 16 (!) years ago, and still, in all these years,
 no one was assigned to fix it even to this day.

Comment 10

[Matt]

Have you tried the deletion of the relevant MSF file as suggested in bug 209501 ?

Comment 11

[User Dderss]

I have recently set-up the whole thing anew again, and this issue is still returning.

In my case, it happens only when I delete a mail, though not always, only occasionally. It successfully gets transferred to the Trash folder with no errors of any kind at any point, but at the same time leaves a shadow clone dated 1/1/1970. And it is annoyingly persistent at times as deleting the shadow mail does not help initially and needs to be repeated, though eventually, it does go away.

Comment 12

[Matt]

Clearly, as the displayed date is a zero in C++ and the result of missing data, a filter is not ever going to find the relevant entry by specifying a date value.

This bug has been here for years, and never have the basic support steps been gone through. Long experience has shown that most issues with data relate to file contention, but let's try this in a methodical manner.

Does this issue occur when you are running Thunderbird in safe mode?

Additionally;

• Is there an anti-virus scanning the folder while Thunderbird is running? We do not recommend real-time scanning of mail folders while Thunderbird is open. An exclusion is generally required in the anti-virus product.

• Is there any form of streaming backup or cloud synchronization being used on the folder where Thunderbird stores its data?

• Is the profile folder physically stored on a locally attached drive? or is it stored in a NAS or other networking device or USB drive?

• What version of Thunderbird are you using now? Given your past use of Enigmail this may be very pertinent.

Comment 13

[Matt]

I actually think this bug is invalid. There is a phantom entry in the MSF, perhaps with a corrupt mbox. But the fact a filter will not act against a message without a header or contents is not a bug. (zero length EML file)

Comment 14

[Wayne Mery (:wsmwk)]

per comment 13