917769 - Gloda.index_msg ERROR Problem encountered during message move/copy: undefined, if "filter move with after Junk Classification" happens when outdated msf condition exists

Status: NEW

(MailNews Core :: Search, defect)

Description

[Chris W]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Firefox/24.0 (Beta/Release)
Build ID: 20130910160258

Steps to reproduce:

Message Filters stopped working after update to 24.0


Actual results:

Message Filters stopped working after update to 24.0


Expected results:

Mails coming into inbox should be sorted to corresponding folders based on Message Filters rules that were set up.  Stopped working after update to 24.0

Comment 1

[:aceman]

After the filters fail to apply to an incoming message, is there anything in Tools->Error console? Also, you can go into the filter editor and see if everything looks right there. Then you can paste here a sample filter that is not working.

Comment 2

[Chris W]

Attachment: sample filter

A sample filter.  It didn't sort automatically in 24.0 when new mail arrives.  It then sorted to the designated folder if I ran the filter manually within the filter editor.  Error message in the next attachment.

Comment 3

[Chris W]

Attachment: error message for the sample filter in the last attachment

error message for the sample filter in the last attachment

Comment 4

[Chris W]

Attachment: popup alert

I also got a popup alert.  I did have enough disk space and write privileges set up correctly.  When I ran the filter manually inside the filter editor, it worked properly and sorted the message to its designated folder.

Comment 5

[:aceman]

Thanks, interesting. Can you try backing up the target folder "Folder" (the folder file is located in the Local Directory path set in your account configuration) and then right click the folder -> properties-> repair ?

Comment 6

[Chris W]

It's fine now.  The folder property must be corrupted after the update.  Thanks for showing me.

Comment 8

[Chris W]

Problem seemed to persists in some cases, even after repairing folder properties.  Some working filters (that were fixed) became dysfunctional again.  Like before, when I ran the filters manually after they failed to sort, they work.  They just didn't sort automatically when new mail arrived.

Comment 9

[bgauthier]

I observed the same situation this morning. Yesterday morning, I repaired all relevant folders and all filters appeared to work fine automatically. This morning, the filters worked for about 200 messages but did not for about 10; however, these 10 were treated correctly once I issued the command to apply the filters from the menu.

Comment 10

[WADA:World Anti-bad-Duping Agency]

(In reply to Chris W from comment #3)
> Created attachment 807005 [details]
> error message for the sample filter in the last attachment

This is improvement in Tb 24.
  When outdated-msf condition exists in filter move target folder,
  after writing filter log of "mail is moved to move target folder",
  Tb 17 : "filter move action" silently fails. Messae filter does do nothing.
          (see bug 495760)
  Tb 24 : reports error in "filter move action", and "filter move" fails.

As written in bug 905576, "outdated-msf condition" or "loss of .msf file" is logged by MSDB:5 of NSPR logging(log of "error opening db 80550005", or "error opening db 80550006"). And "internal rebuild-index("internal Repair Folder) is known by log for "temporary .msf file" such as "nsMsgDatabase::Open(C:\DOCUME~1\ ... \Temp\MozillaMailnews\<FolderName>.msf".

AFAIK, if internal reuild-index is running, "filter move" can not move mails to filter move target folder. And AFAIK, log by MsgCopyService:5 is written upon "filter move".
Get NSPR log with "SET NSPR_LOG_MODULES=timestamp,MSGDB:5,MsgCopyService:5"(Win example) and check log file content by Text Editor.

Comment 11

[bgauthier]

Attachment: Error message (folder not writable)

Comment 12

[bgauthier]

This morning, some filters did not work because a folder (cron.daily) was found to be not writable (see attachment https://bugzilla.mozilla.org/attachment.cgi?id=808496 in French). This is a folder that I "repaired" a couple of days ago. Manual firing of the filters moved the message to their intended destination (before repairing the folder).

Comment 13

[WADA:World Anti-bad-Duping Agency]

(In reply to bgauthier from comment #12)
> some filters did not work because a folder (cron.daily) was found to be not writable

Sorry for misleading explanation. The error message is not "a folder is not writable" or "filter move failed". The error message is shown by Indexer of "Global Search and Indexer". i.e. Something wrong happened during filter move(then mail can't be moved by filter), then error is detected by Indexer. 

What is logged in message filter log(filterlog.html) for the non-moved mails?

Bug 762318 and Bug 750630 are known. "Checking Mail (After Classification)" is a known workaround of these problems, although opossite problem is reported to bug 900142(bug opener says "if After Classification, problem occurs").
Will "Checking Mail (After Classification)" reduce frequency of your problem?

Comment 14

[Wayne Mery (:wsmwk)]

(In reply to WADA from comment #13)
> (In reply to bgauthier from comment #12)
> > some filters did not work because a folder (cron.daily) was found to be not writable
> 
> Sorry for misleading explanation. The error message is not "a folder is not
> writable" or "filter move failed". The error message is shown by Indexer of
> "Global Search and Indexer". i.e. Something wrong happened during filter
> move(then mail can't be moved by filter), then error is detected by Indexer. 
> 
> What is logged in message filter log(filterlog.html) for the non-moved mails?
> 
> Bug 762318 and Bug 750630 are known. "Checking Mail (After Classification)"
> is a known workaround of these problems, although opossite problem is
> reported to bug 900142(bug opener says "if After Classification, problem
> occurs").
> Will "Checking Mail (After Classification)" reduce frequency of your problem?

Comment 15

[bgauthier]

The problem persists in TB 24.0.1. Respectfully, I don't see how the referenced bugs help with my problem.

Comment 16

[WADA:World Anti-bad-Duping Agency]

(In reply to bgauthier from comment #15)
> I don't see how the　referenced bugs help with my problem.

Because your problem occurs even after "Repair Folder", and because "filter move by manual filer run" always works well in your case, probability of outdated-msf condition or deleted .msf or broken .msf is perhaps low.
  If problem like bug 905576, mail is moved to move target folder.
  It's simply that unread mail count is not updated at folder pane after filter move.
  When the "move target folder" is clicked, "Repair Folder" is automatically invoked,
  then unread mail count is updated and moved mails are shown at thread pane.

A possible cause of such phenomenon is that "write mode file open by filter move upon mail download" is interfered by someone else. Candidates of "someone else" is Tb's other tasks, program other than Tb such as backup software, virus scaner.
Purpose of "After Classification" is;
  After Tb download a mail to Inbox, before Tb invokes message filter on the mail,
  force Tb to call Junk filter for the mail.
  This postpones Tb' message filter application on the mail,
  then "open of move target file by filter move" is postponed.
This may reduce frequency of interfere of "file open by filter move" by someone else.

Comment 17

[bgauthier]

Thank you for your message. After reading it, I changed the file permissions of the mail folders and files to make them wider. And, there was no filtering problem this morning. I will keep an eye open and report back if the conclusion that my problem was file permission related does not stand. Thanks to all for considering this problem.

Comment 18

[WADA:World Anti-bad-Duping Agency]

As seen in following comment #4, "adding 'me too' to this bug" usually implies and should imply that "enough disk space and write privileges" is quranteed, because error message indicate such issue...
> I also got a popup alert.  I did have enough disk space and write privileges set up correctly.
Sigh...

Comment 19

[bgauthier]

Fair enough but why is it that the automatic filter process requires different file permissions than the manual filtering process?

Comment 20

[WADA:World Anti-bad-Duping Agency]

(In reply to bgauthier from comment #19)
From what permission to what permission did you change?

What did you do for "files to make them wider"?
Enlarged setting like "directory size limiation" or "total disk space for user"?

Comment 21

[bgauthier]

No, I changed the FILE PERMISSIONS of the mail folders and files to make them wider: I changed them all to 0777 from, generally, 0644, sometimes 0700 -- all files and folders are owned by user nobody and group nogroup.

Comment 22

[bgauthier]

I am sorry to report that making the account folder and file permissions 0777 did NOT fix the problem. I still have some messages that do not get sorted automatically (while others do) but that behave reliably upon manually initiating the sorting rules. I have not been shown a pop-up window stating that the sorter could not write in a given folder in a while, however.

Benoît

Comment 23

[Andrew White]

I have this exact bug per the Popup Alert above and I would like to try :aceman's proposal:

"Can you try backing up the target folder "Folder" (the folder file is located in the Local Directory path set in your account configuration) and then right click the folder -> properties-> repair ?"

I'm not sure where to right click. Is this in Windows explorer? There is no "repair" option there ... where do I right click?

Comment 24

[:aceman]

Rightclick the folder inside running Thunderbird.

Comment 25

[Andrew White]

I'm sorry, it is clear to you what you mean but it is not clear to me. I have right-clicked everywhere I can think of and I cannot find anything that produces a "repair" option. Please could you give me a detailed Tools > Account Settings > etc > etc ...(or whatever?) Many thanks!

What I see is that the Local Directory path contains a small number of .sbd files which correspond to valid filters. It appears that the problem is deleted .sbd files. It that useful information at all?

Comment 26

[S. Christian Collins]

Is anybody still experiencing this bug with Thunderbird 24.1.0? I don't seem to be experiencing this bug anymore, and I'm wondering if this was solved in the Thunderbird 24.1.0 update. I don't exactly remember when Thunderbird got updated to 24.1.0 (it must have came with some Kubuntu updates), but I haven't had a single error in weeks.

Comment 27

[bgauthier]

Using TB 24.1.0, I encounter this error every day. This is not fixed.

Comment 28

[bgauthier]

I updated to TB 24.1.1 yesterday. This morning, no pop-up error message window saying that such-and-such a folder could not be written into, but 18 messages did not get dispatched correctly nonetheless.

Benoît

Comment 29

[bgauthier]

CORRECTION: I did get the usual "folder could not be written into" -- just later than usually. So, no change in the existence or behaviour of this bug in TB version 24.1.1.

Benoît

Comment 30

[JoWestla]

I am eperiencing the same bug since a felt eternity

My best guess always was that this bug results due to a collision between compression/indexing tasks of mail folders and simultane email-download/filtering processes. Some of my mail folders are quite large (around 20 of my folders are >1 GB) and additionally all folders are located on a mounted network drive.

Comment 31

[bgauthier]

My folders are on a mounted drive as well but they are pretty lean: right now, the largest is 113 megs apart from the Trash that is 149 megs. The inbox is 96 megs; other folders are less than 20 megs.

Benoît

Comment 32

[Wayne Mery (:wsmwk)]

(In reply to JoWestla from comment #30)
> I am eperiencing the same bug since a felt eternity
> 
> My best guess always was that this bug results due to a collision between
> compression/indexing tasks of mail folders and simultane
> email-download/filtering processes. Some of my mail folders are quite large
> (around 20 of my folders are >1 GB) and additionally all folders are located
> on a mounted network drive.

in which case .... please test your theory. disable automatic compact

Comment 33

[bgauthier]

Note that automatic compacting is off in my case.

Benoît

Comment 34

[JoWestla]

(In reply to Wayne Mery (:wsmwk) from comment #32)
> (In reply to JoWestla from comment #30)
> > I am eperiencing the same bug since a felt eternity
> > 
> > My best guess always was that this bug results due to a collision between
> > compression/indexing tasks of mail folders and simultane
> > email-download/filtering processes. Some of my mail folders are quite large
> > (around 20 of my folders are >1 GB) and additionally all folders are located
> > on a mounted network drive.
> 
> in which case .... please test your theory. disable automatic compact

Hmm - I disabled automatic compacting, and I also disabled Lightning (just to be sure), the behavior  (sadly) persists.

Comment 35

[Wayne Mery (:wsmwk)]

one way to get a grip on this is for any of you reporting this problem to determine the regression range.   A 1-2 day regression
range would be awesome.  The process is to use binary search
http://garykwong.wordpress.com/2009/02/24/howto-find-regression-windows-through-manual-binary-search/

builds at https://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/2013/ and https://ftp.mozilla.org/pub/mozilla.org/thunderbird/releases

Comment 36

[yanlu]

I have same issue since october (and now on ubuntu 13.10) and so have more people as can be seen on french forum http://forums.mozfr.org/viewtopic.php?f=4&t=115283).

It does not happen to random folders, allways to the same folders.

To overcome the issue, i move the folder's content into another new folder, thats seem to be enough.

Comment 37

[bgauthier]

This solution would not work with the Trash folder, would it, it being sort of a system folder? Trash is one of the folders that is giving me grief.

Benoît

Comment 38

[yanlu]

sure it wouldnt.

what you can do is create another mail account, or use another one where the trash folder is OK, and use its trash folder as a general trash folder.

Comment 39

[:aceman]

You can try to delete the Trash folder while TB is NOT running. It should be automatically recreated empty as it is a special and required folder.

Comment 40

[JoWestla]

(In reply to Wayne Mery (:wsmwk) from comment #35)
> one way to get a grip on this is for any of you reporting this problem to
> determine the regression range.   A 1-2 day regression
> range would be awesome.  The process is to use binary search
> http://garykwong.wordpress.com/2009/02/24/howto-find-regression-windows-
> through-manual-binary-search/
> 
> builds at https://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/2013/
> and https://ftp.mozilla.org/pub/mozilla.org/thunderbird/releases

Will try! Is there maybe also a different option (like e.g. a debugging logfile one could generate), because finding the correct regression might take quite a while.

Comment 41

[Ray Lutz]

I have had similar problems and have been pulling my hair out for a while. Again, this started recently, in the last update or two. I have been using Thunderbird for many years and am a sophisticated user.

There are a number of apparently interrelated problems.
1. all junk mail operations, message moving, etc. becomes nonfunctional after while. Restarting Thunderbird allows these functions to resume. This is apparently tied to getting messages, i.e. with automatic checking for messages every X minutes, then after that check, it locks up in this fashion. Also, after lock-up, the little icon in the tray of an envelope to indicate new messages will be present. Sometimes, multiple icons are shown, say five or six. Hovering over them and they will disappear.
2. When I turn off automatic message checking, then the lock up does not occur until I start checking messages. When I manually check messages, then I get the "Can't filter messages to mailbox XXX" notice. It only occurs during filtering when getting mail. Manual execution of filters does not do this.
3. Also, when the system "locks up" it is unable to save messages to the sent folder. I have turned off saving of sent messages and I just send myself a BCC. There is no way to save the message once it "locks up" as saving to drafts does not work either.

In terms of attempts to resolve this, I have done just about everything I know how to do, including deleting .msf files, deleting junk, moving messages to new folders, etc. etc. My latest test is turning off automatic checking for messages, and this delays the entry into the lockup mode. The only way out of the mode is to shut down Thunderbird and restart it.

I have a combination of POP accounts and IMAP accounts. The "Can't write to folder" happens when I check mail for a folder that has mail filters enabled during when mail is received. Thus, I can avoid the "Can't write to folder xxx" error if I turn on automatic mailbox checking but then it locks up after a while, and I have to restart Thunderbird. I'm not sure if it eventually locks up with manual mailbox checking.

Comment 42

[WADA:World Anti-bad-Duping Agency]

(In reply to Ray Lutz from comment #41)
> The "Can't write to folder" happens when I check mail for a folder
> that has mail filters enabled during when mail is received.

"Can't write to folder" case is already processed in bug 931303.
This bug was initially opened for attachment 807005 [details] : error message for the sample filter in the last attachment. "Can't write to folder" case is added by other than bug opener after 5 days from bug open.

As mentioned in that bug and bugs pointed in that bug, "Repair Folder"(Rebuld Index) usually resolves the "Can't write to folder" error upon filter move when new mail download.
Have you tried "Repair Folder" of filter move target folder?

As mentionedd in bugs pointed in that bug, "MsgDatabase Open error" may happen when the error message is shown. And, if Rebuild-Index happened, a specific log is written by NSPR logging with MSGDB:5.
Get NSPR log with MSGDB:5 and check log file content using Text Editor by yourself.
Is "error opein db ..." like log seen? Is log like "nsMsgDatabase::Open( ...\Temp\MozillaMailnews\<FolderName>.msf, ..." seen in log?

Comment 43

[Ray Lutz]

I've already been through repairing indexes, deleting all .msf files, etc. The reason I don't think it has anything to do with the folder indexes is because filtering works fine when I do it manually. If I set all filters for a given inbox to not filter when messages are received, then after messages are received, I can manually "Run filters on folder" and it works fine. Thus it seems that the folder index cannot be the reason this fails.

I only suspect that the two issues are related because they both started at about the same time and seem to be contingent on the same set of circumstances. I acknowledge that bug 93103 is discussing the can't move messages issue. That is unresolved, but I found that by not automatically getting messages, I can avoid that failure at random times (i.e. when messages are retrieved) and instead the failure would occur when I manually invoke message checking. However, then I get this "Can't write to folder" error, which can be avoided by disabling automatic filters.

Thus workaround for both problems is: Disable automatic message checking and disable automatic filtering when messages are retrieved. Instead, manually check messages on server (avoids folder write error) and manually invoke filters on folders (avoids "Can't write to folder" error).

Comment 44

[WADA:World Anti-bad-Duping Agency]

(In reply to Ray Lutz from comment #43)
> I've already been through repairing indexes, deleting all .msf files, etc.

"Delete .msf file" is "manual/intentional corruption of .msf file"(MagDatabase).
"Rebuild Index"(Same as Repair Folder) will be executed only when;
  "explicit folder open" is executed(folder click at folder pane) is executed 
  "Repair Folder" is requested.
  "Folder Properties" is manually shown. (may be, only when "outdated msf condition exists)
"Filter move to the folder, for which .msf file is deleted by you" won't invoke internal rebuild-index(see bug 495760).

"Delete .msf file followed by explicit folder open by folder click" is final recovery method when "Repair Folder" can't work well.
Please don't mix "manual/intentional MagDatabase corruption by manual/intentional deletion of .msf file" case.

Comment 45

[WADA:World Anti-bad-Duping Agency]

(In reply to bgauthier from comment #11)
> Created attachment 808496 [details]
> Error message (folder not writable)

Problem in following message you cited is already processed in bug 931303.
> filterFolderWriteFailed=The messages could not be filtered to folder '%S'
> because writing to folder failed.
> Verify that you have enough disk space, and that you have write privileges to the file system, then try again.
To avoid confusion, and to focus on original problem of comment #0 only, problem around error message by Gloda, I hide your attachment.
If it's wrong, please remove "obsolete" flag of your attachmeny.

Comment 46

[WADA:World Anti-bad-Duping Agency]

(In reply to Chris W from comment #4)
> Created attachment 807015 [details]
> popup alert
> 
> I also got a popup alert.  I did have enough disk space and write privileges
> set up correctly.  When I ran the filter manually inside the filter editor,
> it worked properly and sorted the message to its designated folder.

Problem in the "filter move failure" message is already processed in bug 931303.
And, "pretty misleading error message" is already processed by separated bug 957495.
To focus on problem around error message by Gloda only, I hide your attachment.
If it's wrong, please remove "obsolete" flag of your attachment.

Comment 47

[WADA:World Anti-bad-Duping Agency]

(In reply to Chris W from comment #3)
> Created attachment 807005 [details]
> error message for the sample filter in the last attachment

The error message is shown by Gloda.
> Error: 2013-09-19 09:40:52  gloda.index_msg  ERROR  Problem encountered during  9/19/13 9:40:52 AM
>        message move/copy: undefined
> Source File: resource:///modules/gloda/log4moz.js
Following message attached to comment #4(processed by bug 931303) is observed at same time.
> The messages could not be filtered to folder 'Folder'
So, setting dependency to that bug for ease of problem analysis and tracking,
and confirming per screen shot.

Comment 48

[WADA:World Anti-bad-Duping Agency]

Error message of "Problem encountered during message move/copy:" is shown at;
> http://mxr.mozilla.org/comm-central/source/mailnews/db/gloda/modules/index_msg.js#2480
> 2480       } catch (ex) {
> 2481         this.indexer._log.error("Problem encountered during message move/copy:",
> 2482                                 ex.stack);
> 2483       }
"undefined" in the error message is ex.stack value.

try block in msgsMoveCopyCompleted: is very big... (entire logic of the function is in try)
Even if "Move" case is guessed, "if (aMove) {" block is also big...
Why String(ex), property of ex(exception event object), is not logged?
Excepton type, source file, source line is pretty easily dumped by code like next...
   var list=new Array(); for(var key in ex){ list[list.length]=key+"="+ex[key]; }
   var msgtext = String(ex) + " " + list.join(", ") ;

It's perhaps phenomenn after following.
  MsgDatabase is broken due to problem like bug 905576,
  or, after it, "outdated msf condition" is generated
  => Phenomenon of bug 931303 occurs in "filter move" of Tb 24.
However, because Gloda is enabled by default, if this bug always occurs after bug 931303, error of this bug should be reported by many problem reporters in bug 931303.

Chris W(bug opener), Inbox is IMAP folder or "filter move target folder" is IMAP folder?
Move target folder name in "filter move failure" message is "Folder" in your case.
Do you actually use folder name of "Folder"?

Comment 49

[WADA:World Anti-bad-Duping Agency]

Sorry, I missed filter rule screen shot. Filter move target was actually "Folder".

Comment 50

[:aceman]

Can the problem be solved by disabling Global search and indexer in Preferences? Can you try that?

Comment 51

[WADA:World Anti-bad-Duping Agency]

I could see following error message ｂｙ chace, by intentional "outdated msf condition" test.
> Error: 2014-01-24 13:57:39	gloda.index_msg	ERROR	Problem encountered during message move/copy: undefined
> Source File: resource:///modules/gloda/log4moz.js Line: 688

[Steps to reproduce]
(0) Gloda is enabled by chance in test.
(1) A local mail folder, call "FolderX", is not opened yet.
    => msgDatabse for FolderX is not opened( yetFolderX.msf is not read yet).
(2) Change FolderX content by Text Editor.
      Add other mail data(from "From - ..." line) at end of file.
      => File size and last modified tiestamp of file named FolderX is changed.
    This is emulation of "outdated msf condtion" by following.
    - Mail data is normally appended to file named FolderX, by download, copy, move.
      Before data of msgDatabase is writte back to FolderX.msf, power failure of PC occurs.
(3) Run filter on other folder, call FolderA,
    and move some mails(Unread state) from FolderA to FolderX by message filter.
    - Message filter appends all mail data to file named FolderX normally.
    - Mails are normally deleted from FolderA
      => MsgDBHdrDeleted event is notified to event listener.
    - No error message is shown by filter.
    - However,
      (a) Message count(unread message count) of FolderX at folder pane is not updated.
      (b) MsgDBHdrAdded is not notified to event litsener.
      (c) Above message is shown at Error Console by Gloda.
(4) Access msgFolder.msgDatabase.xxx property(for exmple, xxx=lastUseTime) by JavaScript
    => Followin exception occurs
      [Exception... "Component returned failure code: 0x80550005
      [nsIMsgFolder.msgDatabase]"  nsresult: "0x80550005 (<unknown>)"
     0x80550005 is "outdated msf condition".
(5) Explicitly open FolderX by folder click of FolderX
    => Because physical open of FolderX.msf is invoked,
       and because "outdated msf condition" is detected by folder open request,
       and because explicitl folder open invokes internal rebuild-index,
       "Repair Folder" of FolderX is executed automatically,
       then manually added mail, mail moved by filter, are shown correctly.

Filter code fault? or Gloda code fault?

Comment 52

[Ray Lutz]

Update on my testing:
1. I re-enabled automatic message checking and downloading. This is okay. Thus, I only have to disable automatic filtering upon message reception.
2. Before I took these steps, disabling Global search and indexer in Preferences did not fix the problem.
3. I have re-enabled saving sent mail to sent folder. This now works.
4. With this configuration, TB is stable and does not "lock up" like it was.

My hunch is a file-locking race condition. I don't know the guts of TB to be much help in finding it, but I would suggest reviewing file locking/unlocking code carefully.

Comment 53

[WADA:World Anti-bad-Duping Agency]

Second "Steps to reproduce" is found.

[Steps to reproduce #2]
(0) Gloda is enabled.
    Filter rule: if subject contains xxx, move to FolderX(move target folder)
    FolderA(move source folder) has some Unread mails with xxx in Subject:.
(1) When FolderX is not opened(msgDatabase for FolderX is not cached yet),
    open FolderX.msf file with "write exclusive "mode by other program such as Text Editor
    (don't use notepad.exe. notepad.exe doesn't lock file).
    This is emulation of "open FolderX.msf by backup software etc."
(3) Run filter on FolderA.
    => Even though "Open MsgDatabase FolderX" fails because "open of FolderX.msf" fails,
       message filter silently moves mail data to FolderX.
       Mail data is appended to file named FolderX, without correct update of FolderX.msf.
       Mails are deleted from FolderA(move source folder).
    => Following error is reported by Gloda.
       gloda.index_msg ERROR Problem encountered during message move/copy: undefined
       Source File: resource:///modules/gloda/log4moz.js Line: 688
(4) If FolderX is clicked at folder pane while FolderX.msf file is opened by Text Editor,
    following exception is shown at Error Console.
      Error: NS_ERROR_FILE_TARGET_DOES_NOT_EXIST: Component returned failure code:
      0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST) [nsIMsgFolder.updateFolder]
      Source File: resource://gre/modules/dbViewWrapper.js Line: 93
    0x80520006  is "msf file is not found" error.
(5) Close FolderX.msf at other software such as Text Editor other than notepad.
    This is emulation of "end of backup software".
    When folder is clicked at folder pane, rebuild-index is internally executed,
    and Unread mail count is updated at folder pane,
    and "mails moved by message filter" is shown at thread pane.

In both STR #1 and STR #2, problem is "pretty-unclear/useless-for-user message shown by Gloda". "Open msgDatabase error in Gloda" should be notified to user as "Open msgDatabase error in Gloda".

Reason why Gloda tries to do "message move/copy" when msgDatabase for "filter move target  folder" is ;
  filter code executes filter move even though "msgDatabase open of filter move target
  folder" fails and "mails are moved event" is notified to Gloda.
So,　mail culprit of this bug can be called "filter move by message filter".

Comment 54

[WADA:World Anti-bad-Duping Agency]

FYI.
If "open file named FolderX.msf" is successful but "open file named FolderX" fails(emulation of interfere of FolderX access by other software such as auto-backup), filter code writes filter log of "move to FolderX is executed", but filter code does do nothing(mails are kept in move source folder).
Because "mave mails from move source folder to move target folder" is not executed by filter code, notification of "mails are moved by filter" won't be sent to Gloda. So, this bug won't happen.

Comment 55

[Wayne Mery (:wsmwk)]

this and bug 931303 *really* need an accurate regression range.
Can someone at least determine what beta version 18-23 this problem first appears?
https://ftp.mozilla.org/pub/mozilla.org/thunderbird/releases/

Comment 56

[WADA:World Anti-bad-Duping Agency]

Following is a part of MSGDB:5,MsgCopyService:5 log and log by MsgDB Event Listener (DBOpened,HdrAdded,HdrDeleted are logged) for "filter move when outdated msf condition exists".
See attached log to bug 905576 comment #220 ( attachment 8395351 [details] ) for detail, please.

Gloda perhaps hooks events which is relevant to "something was added to filter move target folder".
However, due to "outdated msf condition", following occurs.
  "Folder Open" fails because of "outdated msf condition".
  MsgDBHdr of "moved by filter" mail is not created correctly because of "outdated msf condition".
  So, HdrAdded event is not notified to Gloda's Event Listener.
These are perhaps relevant to this bug.

> (1) 3 mails are downloaded to Inbox.
> 16:57:14.6540000 0001 UTC  EventType=MsgDBHdrAdded, messageKey=3163, messageOffset=3163, messageSize=691, summaryValid=false, MsgDB=mailbox://[e-mail]/Inbox
> 16:57:19.2910000 0001 UTC  EventType=MsgDBHdrAdded, messageKey=3854, messageOffset=3854, messageSize=693, summaryValid=false, MsgDB=mailbox://[e-mail]/Inbox
> 16:57:28.1640000 0001 UTC  EventType=MsgDBHdrAdded, messageKey=4547, messageOffset=4547, messageSize=720, summaryValid=false, MsgDB=mailbox://[e-mail]/Inbox
> (2-2) "after Classification", Move to Folder, is executed for mail #1/mail #2/mail #3
> 16:57:28.1890000 0001 UTC  CopyMessages
> 16:57:28.1890000 0002 UTC  request 44e4740 CopyMessages request - src mailbox://[e-mail]/Inbox dest mailbox://nobody@Local%20Folders/Test/DDD numItems 0 type=0
> 16:57:28.1890000 0003 UTC  request 44e4740 DoCopy - src mailbox://[e-mail]/Inbox dest mailbox://nobody@Local%20Folders/Test/DDD numItems 3 type=0
> (2-2-1) Because of outdated msf condition, folder is not properly opened.
> 16:57:28.1890000 0004 UTC  nsMsgDatabase::Open(C:\ ... \Mail\Local Folders\Test.sbd\DDD.msf, FALSE, eec2e0, TRUE)
> 16:57:28.1890000 0005 UTC  closing database    C:\ ... \Mail\Local Folders\Test.sbd\DDD.msf
> 16:57:28.1890000 0006 UTC  nsMsgDatabase::Open(C:\ ... \Mail\Local Folders\Test.sbd\DDD.msf, FALSE, eec2e0, TRUE)
> 16:57:28.1890000 0007 UTC  closing database    C:\ ... \Mail\Local Folders\Test.sbd\DDD.msf
> 16:57:28.1890000 0008 UTC  nsMsgDatabase::Open(C:\ ... \Mail\Local Folders\Test.sbd\DDD.msf, FALSE, eec2e0, TRUE)
> 16:57:28.2040000 0001 UTC  closing database    C:\ ... \Mail\Local Folders\Test.sbd\DDD.msf
> (2-2-2) But mail data of three mails are appended to Test/DDD, without required MsgDBHdr creation.
>         Because Test/DDD is not properly opened, MsgDBHdr for "moved" mail is not created correctly.
>         So, HdrAdded event is not notified to MsgDB Event Listner
> 16:57:28.3610000 0001 UTC  nsMsgDatabase::Open(C:\ ... \Mail\Local Folders\Test.sbd\DDD.msf, FALSE, 6888f50, TRUE)
> 16:57:28.3610000 0002 UTC  closing database    C:\ ... \Mail\Local Folders\Test.sbd\DDD.msf
> 16:57:28.3760000 0001 UTC  nsMsgDatabase::Open(C:\ ... \Mail\Local Folders\Test.sbd\DDD.msf, FALSE, 6888f50, TRUE)
> 16:57:28.3760000 0002 UTC  closing database    C:\ ... \Mail\Local Folders\Test.sbd\DDD.msf
> 16:57:28.3920000 0001 UTC  nsMsgDatabase::Open(C:\ ... \Mail\Local Folders\Test.sbd\DDD.msf, FALSE, 6888f50, TRUE)
> 16:57:28.3920000 0002 UTC  closing database    C:\ ... \Mail\Local Folders\Test.sbd\DDD.msf
> (2-2-3) mail #1/mail #2/mail #3 is deleted from Inbox,
>         because "move to target" is normally executed for "filter move(after Classification)".
> 16:57:28.3930000 0001 UTC  EventType=MsgDBHdrDeleted, messageKey=3163, messageOffset=3163, messageSize=691, MsgDB=mailbox://[e-mail]/Inbox
> 16:57:28.3940000 0001 UTC  EventType=MsgDBHdrDeleted, messageKey=3854, messageOffset=3854, messageSize=693, MsgDB=mailbox://[e-mail]/Inbox
> 16:57:28.3950000 0001 UTC  EventType=MsgDBHdrDeleted, messageKey=4547, messageOffset=4547, messageSize=720, MsgDB=mailbox://[e-mail]/Inbox
> 16:57:28.4030000 0001 UTC  EventType=DeleteOrMoveMsgCompleted, FolderURI=mailbox://[e-mail]/Inbox
> 16:57:28.4070000 0001 UTC  NotifyCompletion - src mailbox://[e-mail]/Inbox dest mailbox://nobody@Local%20Folders/Test/DDD
> 16:57:28.4070000 0002 UTC  request 44e4740 Clearing OK request - src mailbox://[e-mail]/Inbox dest mailbox://nobody@Local%20Folders/Test/DDD numItems 3 type=0

Comment 57

[Anthony Becker]

I had been experiencing this bug regularly (along with others, such as crashing when I compacted mailboxes) starting with Thunderbird 24.0 (or at least I think that was when).  I wonder if this is at all related to my having changed platforms from Ubuntu to MacOSX last year.  In any case, I seem to finally be free of problems (knock wood), but to get there, I had to install the ImportExport Tools add-on, do a complete export of all my mailboxes, and start over from scratch.  After the tedious process of re-importing my old mailboxes into a new profile, and recreating all my filters, the filters all work as expected, and no more crashing.

Sorry if this doesn't provide an actual fix for the bug, but if you are as frustrated as I was, it may at least relieve your frustration.

Comment 58

[WADA:World Anti-bad-Duping Agency]

FYI.

This bug is c) in next.
a) When "outdated msf condition" exists in a message folder,
   (in other words, .msf is broken or not correctly updated)
   bug 261419(and bug 4957609) occurs.
b) When bug 261419(and bug 4957609) occurs upon "Filter Move with before Junk Classification",
   phenomenon of 931303 occurs.
c) When bug 261419(and bug 4957609) occurs upon "Filter Move with after Junk Classification",
   upon manual filter run etc., and if Gloda(Global Search and Indecer) is used,
   phenomenon of this bug occurs.
In any case, cause is that "outdated msf condition" of a folder is kept.
In any case, if "outdated msf condition" exists in a folder, Rebuild-Index(Repair Folder) is automatically invoked by Tb when "Folder open"s executed by user(click folder at folder pane), in order to clear "outdated msf condition". Even when Rebuild-Index is not automatically invoked, user can do "Repair Folder" any time, if problem occurs on a folder.
Biggest problem in this bug:
  "problem on which folder" can not be known by error message...
  so user can do nothing when this bug occurred...

A method to know folder name.
   Rule set "rule-N" : consists of 3 rules.
     rule-N-0 : before Classification, if condition-N, do actions except Move to Folder
     rule-N-1 : before Classification, if condition-N, Move to Folder-N
     rule-N-2 : after  Classification, if condition-N, Move to Folder-N
     (See bug 931303 comment #58 for detail)
If "outdated msf condition" exists, phenomenon of bug 931303 occurs by rule-N-1 execution, and this bug may occur by rule-N-2 execution.
Because you already know folder name by bug 931303, you can check folder status.

Keep "file size for a message folder" as small as possible, because Rebuild-Index is time consuming job and required time is proportional to file size(and number of mails in a folder).

Comment 59

[WADA:World Anti-bad-Duping Agency]

Removing "filter doesn't work" part from bug summary, to avoid misleading/confusion, and to focus on Gloda problem only.

Comment 60

[WADA:World Anti-bad-Duping Agency]

(In reply to Anthony Becker from comment #57)
> such as crashing when I compacted mailboxes) starting with Thunderbird 24.0 (or at least I think that was when).

"Prolem in Compact when outdated msf condition exists" surely exist. For example,
  Event of CompactCompleted is notified to Folder event listener.
  However, file size is not reduced, expungedBytes is not reset to ZERO.
  i.e. Compact does do nothing or silently fails to do Compact.
IIRC, Compact in former Tb releases invoked Rebuld-Index if "outdated msf condition" exists.
This may be relevant to your "crashing when I compacted mailboxes".
Actually "Crash of Tb"? (Crash is usually for "sudden abnormal termination of Tb" with dump or crash log).
I you still see such problem with Compact, open separate bug, please.

Comment 61

[WADA:World Anti-bad-Duping Agency]

Bug 989884 is cripy report. Setting dependency instead of duping to that bug, because relation with bug 931303 will be lost by duping.

Comment 62

[JoWestla]

Hi, I just wanted to report (even if this bug is already a bit older) that this behavior vanished for me. The cause is likely that I migrated my desktop to windows 7 finally. The entire thunderbird configuration remained entirely unchanged (since stored on serverside), same applies to the serverside itself (also unchanged).

Environment:
Desktop Windows 7
Serverside Debian / Samba mounted
Thunderbird 31.4.0