Open Bug 633093 Opened 13 years ago Updated 5 years ago

Message spilt when using mail filters for tagging and moving

Categories

(MailNews Core :: Filters, defect)

Product:
MailNews Core
Component:
Filters
Platform:
x86
All
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

(Not tracked)

People

(Reporter: stefanehaas, Unassigned)

Details

(Keywords: dataloss, testcase, testcase-wanted)

Attachments

(1 file)

Inbox containing \0
448.39 KB, application/zip
Details 
User-Agent:       Mozilla/5.0 (Windows NT 5.1; rv:2.0b12pre) Gecko/20110208 Firefox/4.0b12pre SeaMonkey/2.1b3pre
Build Identifier: Mozilla/5.0 (Windows NT 5.1; rv:2.0b12pre) Gecko/20110208 Firefox/4.0b12pre SeaMonkey/2.1b3pre

Since 2.1b3 several mails are split in pieces after being touched by mail filter for retreival, tagging and moving

Reproducible: Sometimes

Steps to Reproduce:
1. Receive (imap) mail
2. Mail filter auto applies and tags message
3. Tagged split files remain in Inbox
Actual Results:  
Message split apart (tagged but not moved)

Expected Results:  
Message tagged and moved safely

Worked in 21b2 form > 100.000 mails but fails frequently in 21b3

Manually moved mails cloak target mail folders (incl. no message display at all)

Appears on plain text and html mails
When I disable (all) mail filters the messages remain safely where they are (not useful if you receive > 5.000 mails/day)
Component: MailNews: Backend → Filters
Product: SeaMonkey → MailNews Core
QA Contact: mailnews-backend → filters
Version: unspecified → Trunk
(In reply to comment #0)
> Actual Results:  
> Message split apart (tagged but not moved)

Are action of "add tag" and "move to folder" defined in single filter rule? Or defined in different filter rules?
Is "filter apply of the rule(s)" logged in Filter log as expected?
Move to which folder? Folder of same IMAP account? Folder of differnt IMAP account? Or local mail folder(POP3 account or Local Folders)?
If IMAP folder, offline-use=On? Or Off? (Inbox and move target, Folder Properties/Synchronization)
Is move target folder manually opened before the new mail arrival and filtering?

If "add tag TTT then move to FFF", next is executed.
  (XXX is UID of mail. UID is shown at "Order Received" column if IMAP folder)
  at a cached connection for Inbox
  1. uid store XXX +Flag (TTT)
  2. uid copy  XXX "FFF"
  3. uid store XXX +Flag (\Deleted)
  4. After that, fetch of copied mails is executed at folder FFF.
"tagged (and copied), but not moved" is usually step 1. and 2. is executed but step 3. is not executed or fails. 
Can you get IMAP log and check IMAP level flow around "add tag then move" for a mail which was "tagged but not moved"?
> https://wiki.mozilla.org/MailNews:Logging
> Win example : SET NSPR_LOG_MODULES=timestamp,imap:5
If log analysis by developers is needed, attach log file to this bug(never paste), after remove irrelevant data(mail data is not needed except Message-ID:, Subject: etc. to know which mail) and replace sensitive data, please.
Are action of "add tag" and "move to folder" defined in single filter rule? Or
defined in different filter rules?
- Same rule

Is "filter apply of the rule(s)" logged in Filter log as expected?
Move to which folder?
- Did log rules so far (size), no switched on but currently I diabled the rules as this corrupts the whole mail folder file

- Separate local folder for every rule
Folder of same IMAP account?
- No

Folder of differnt IMAP account?
- No

Or local mail folder(POP3 account or Local Folders)?
- Yes

If IMAP folder, offline-use=On? Or Off? (Inbox and move target, Folder
Properties/Synchronization)

Is move target folder manually opened before the new mail arrival and
filtering?
- May happen as I check the mails there

Seems to happen on IMAP and POP3 accounts (I receive & filter mails form both) - counld not yet verify a pattern excpet using filters
Seems to happen when mail is read from POP3 account - I switched to IMAP now (and keep watching)
Attached file Inbox containing \0 — 
Contains \0 chars in line 750, 12085 ...
(In reply to comment #0)
> Firefox/4.0b12pre SeaMonkey/2.1b3pre
(In reply to comment #4)
> Seems to happen when mail is read from POP3 account - I switched to IMAP now
> (and keep watching)
(Bug summary) filters for tagging and moving  

Do you enable qurantine option? (mailnews.downloadToTempFile=true)
If yes, same problem as bug 582918?
Stefan, what is the setting for mailnews.downloadToTempFile in your about:config?
The mailnews.downloadToTempFile is true (I can't remember setting it explicitely by t states "chaned by user")

I don't think it's the same as 582918 as it insertes \0 chars which confuses the mail display. When I remove the \0 the mail "folder file" seems to react normally again.
(In reply to comment #8)
> I don't think it's the same as 582918 as it insertes \0 chars which confuses
> the mail display. When I remove the \0 the mail "folder file" seems to react
> normally again.

To demonstrate phenomenon of data corruption in Inbox by bug 582918, three mails case is presented as "Steps To Reproduce" in that bug.
"Single mail" case produces next simpler phenomenon;
  - At move target folder of filter move.
    mail is shown at thread pane of move target folder of filter rule,
    but nothing is shown as mail data because nothing is writte to file,
    Compact Folder may not remove garbage at thread pane,
    by Repair Folder, filtered mail at thread pane disappears.
  - At inbox of POP3 account where mail is downloaded.
    filtered mail appears by Repair Folder,
    because mail data is written to file for Inbox by filter move
    and the written mail is not flagged as Deleted in file for Inbox.
Stefan Haas(bug opener), what is difference of your problem from bug 582918?
I con confirm the above behavior from bug 582918, but in my bug case the mail target (moved to) folder file contains a section with lots of \0 which confuses all further processing of the folder. As this does not happen in bug 582918 I assume this is yet another bug.

In this case I have to use an editor to remove all \0 from the file, delete its msf and restart SM. Then the mail folder contents is displayed correctly.

It seems this only occurs on POP3 mail folders. I used POP3 mainly before and now switched almost all mail to IMAP and it works fine for those mail accounts.

I have not yet established a repeatable pattern as in the other bug it is easier to get rid of all my POP3 accounts and read them as IMAPs - as the error seems to be related with the POP3 account reader/filter.
(In reply to comment #10)
> As this does not happen in bug 582918 I assume this is yet another bug.

In that bug, once "mailnews.downloadToTempFile=true && POP3 account is mandatory condition of that bug" was found, everyone does do next when he knows that he is experiencing that bug;
 1. Recovery from that bug, by Folder Properties/Repair Folder etc.
 2. Change to mailnews.downloadToTempFile=false, because everyone knows true of
    the setting is a cause. by reading thru of that bug.
 3. Then he won't experience that bug again.
 4. If he is requested to check "whether bug still occurs or not",
    he starts from "no garbage by that bug" status with setting change,
    and when he see that bug, report it and go to step 1 again.
So, no one doesn't check about phenomena by that bug after problems by that bug ater problems by that bug after problems by that bug after ....
Everyone, who tries to reproduce that bug, starts with no garbage by that bug, and when he sees problem of that bug, he stops to see problem by that bug.

Problem which you are seeing is phenomena by that bug after problem by that bug ater problem by that bug after problem by that bug after ....?
Or problem which you are seeing is absolutely different problem from that bug?
Sorry for my typo. Please read like next, please.
> So, no one checks about phenomena by ...
> So, anyone doesn't check about phenomena by ...
FYI.
As seen in duplication test result of that bug, garbled mail data can be written in Inbox, in addition to phantom mail at move target folder(shown at thread pane only. no mail data is written in file for move target folder, then blank mail is displayed).
If you see problem which looks produced by that bug again, do next;
(1) At move target folder : Folder Properties/Repair Folder
(2) At Inbox : Folder Properties/Repair Folder
In any case, show "Order Received" column. Value shown the column is offset in file for mail folder if local mail folder. It'll help your analysis of funny pkenomenon you see.
WADA, is more information needed to decide *what* needs to get fixed?

rkent feels this is definitely filter bug, so confirming
Status: UNCONFIRMED → NEW
Ever confirmed: true
Keywords: dataloss, qawanted
Can someone confirm that this is still in issue in the current trunk of Thunderbird?
Keywords: qawanted → testcase, testcase-wanted
OS: Windows XP → All
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: