Last Comment Bug 392015 - Crash of Tb, if "Rebuild Index" is executed during compacting folder is inprogess, and loosing mails [@ morkRowObject::SetRow]
: Crash of Tb, if "Rebuild Index" is executed during compacting folder ...
: crash, dataloss, fixed1.8.1.17, testcase, topcrash
Product: MailNews Core
Classification: Components
Component: Database (show other bugs)
: 1.8 Branch
: All All
: -- critical (vote)
: mozilla1.9.1a2
Assigned To: David :Bienvenu
Depends on:
  Show dependency treegraph
Reported: 2007-08-13 03:34 PDT by benoit
Modified: 2014-01-17 03:37 PST (History)
4 users (show)
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---

Bug details (118.31 KB, text/plain)
2007-08-16 05:28 PDT, benoit
no flags Details
proposed fix (2.80 KB, patch)
2008-07-08 10:11 PDT, David :Bienvenu
no flags Details | Diff | Splinter Review
use throwAlert (2.33 KB, patch)
2008-07-09 09:18 PDT, David :Bienvenu
neil: review+
neil: superreview+
dmose: approval1.8.1.17+
Details | Diff | Splinter Review

Description benoit 2007-08-13 03:34:17 PDT
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv: Gecko/20070725 Firefox/
Build Identifier: version (20070728)

During thunderbird were compacting a folder, it crashed and messages of this folder were all deleted after restarting thunderbird

Reproducible: Always

Steps to Reproduce:
1.right click on a folder
3.right click on the same folder > rebuilt index
6.all mail of the folder disappear
Actual Results:  
After the crash I restart thunderbird. Some messages that disappeared from the folder were store on a folder nstmp, but most of messages were lost.

Expected Results:  
I don't know, but it should make a backup or compact messages per messages to avoid dataloss in case of crash.

no additional theme were use
no additional modules were use
Comment 1 WADA 2007-08-13 18:11:28 PDT
> 2.Compact
> 3.right click on the same folder
> > rebuilt index
> 5.crash!!

Did you execute "Rebuild Index" at step 4 after completion of "Compact Folder" at step 2? Or you did "Rebuild Index" while "Compact Folder" is in progress?
If latter, no crash when Thunderbird trunk nightly(2007/08/09-12 build,MS Win-XP), then there is no critical problem like data loss after restart.

(1) "Compact Folder"
    => nstmp & nstmp.msf is created, and data is being copied to nstmp
(2) "Rebuild Index" while "Compact Folder" is in progress
    => Warnig dialog of "other process in progress, retry after end"
(3) Reply "OK" to dialog
    => nstmp & nstmp.msf was deleted
       <folder_name>.msf for the folder compacted was deleted 
    => Following errors in Error Console
(For compact)
> Error: [Exception... "Component returned failure code:
> 0x80550006 [nsIMsgFolder.getMsgDatabase]"  nsresult: "0x80550006 (<unknown>)"
> location: "JS frame :: chrome://messenger/content/msgMail3PaneWindow.js ::
> HandleCompactCompleted :: line 526"  data: no]
> Source File: chrome://messenger/content/msgMail3PaneWindow.js Line: 526
(For rebuild index)
> Error: uncaught exception: [Exception... "Component returned failure code:
> 0x8055000a [nsIMsgFolder.updateFolder]"  nsresult: "0x8055000a (<unknown>)"
> location: "JS frame :: chrome://messenger/content/widgetglue.js ::
> RebuildSummaryFile :: line 249"  data: no]

(4) "Rebuild Index" again several times
    => Following error in Error Console on each "Rebuild Index" request 
> Error: uncaught exception: [Exception... "Component returned failure code:
> 0x80550006 [nsIMsgFolder.getMsgDatabase]"  nsresult: "0x80550006 (<unknown>)"
> location: "JS frame :: chrome://messenger/content/widgetglue.js ::
> RebuildSummaryFile :: line 237"  data: no]

(5) Close property dialog => error messages were issued 
(6) Terminate Thunderbird, and Restart Thunderbird
    => Because no .msf, .msf recreation is successfully executed
    => Because no garbage of nstmp & nstmp.msf, no garbage in folder pane
Comment 2 WADA 2007-08-13 18:22:05 PDT
To David Bienvenu:
Crash(and dataloss for user) of Tb 2.0 is sufficiently critical problem, even if no crash when trunk. Should we keep separate bugs for crash of 2.0 and mail db damage problem of trunk? 
Comment 3 David :Bienvenu 2007-08-13 19:09:27 PDT
I'm more worried about the 2.0 problem - on the trunk, the db gets regenerated so that doesn't bother me so much.
Comment 4 WADA 2007-08-14 06:14:55 PDT
I tested with Tb using a folder of 80,000 junk mails, but crash was not re-produced.
(1) "Rebuild Index" while "Compact Folder" is in progress.
(2) Dialog of "other process is in progress", and reply "OK"
    => nstmp & nstmp.msf was deleted (i.e. compact aborted)
       (error message in error console appeared)
    => mail count of the folder became far smaller than 80,000
       (error message in error console appeared)
(3) "Rebuild Index"
    => "Rebuild Index" completed, but count became less than 80,000
(4) Click other folder and click the folder again
    => Mail count of the folder was corrected to 80,000

To benoit(the bug oper):
What is Talkback-ID of your crash?
Do you use add-on(extension)?
If yes, can you re-produce crash with -safe-mode of Thunderbird?
Comment 5 WADA 2007-08-14 08:16:38 PDT
Crash was re-produced with Thunderbird build, en-US, MS Win-XP), by same test procedure as comment #4.
Crash when step (1), "Rebuild Index" while "Compact Folder" is in progress.
nstmp & nstmp.msf remained after crash.
Talkback-ID = TB34936806Y
Comment 6 WADA 2007-08-14 08:17:18 PDT
Comment 7 timeless 2007-08-15 00:45:58 PDT
Incident ID: 34936806
Stack Signature	morkRowObject::SetRow 1dbd471f
Product ID	Thunderbird2
Build ID	2007072817
Trigger Time	2007-08-14 08:07:01.0
Platform	Win32
Operating System	Windows NT 5.1 build 2600
Module	thunderbird.exe + (0004f656)
URL visited	
User Comments	This is crash report of Bug 392015
Since Last Crash	82 sec
Total Uptime	82 sec
Trigger Reason	Access violation
Source File, Line No.	e:/builds/tinderbox/Tb-Mozilla1.8-Release/WINNT_5.0_Depend/mozilla/db/mork/src/morkRowObject.cpp, line 462
Stack Trace 	
morkRowObject::SetRow  [mozilla/db/mork/src/morkRowObject.cpp, line 462]
nsFolderCompactState::EndCopy  [mozilla/mailnews/base/src/nsMsgFolderCompactor.cpp, line 996]
nsMailboxProtocol::OnStopRequest  [mozilla/mailnews/local/src/nsMailboxProtocol.cpp, line 301]
nsInputStreamPump::OnStateStop  [mozilla/netwerk/base/src/nsInputStreamPump.cpp, line 564]
nsMsgProgress::AddRef  [mozilla/mailnews/base/src/nsMsgProgress.cpp, line 49]
Comment 8 benoit 2007-08-16 05:28:51 PDT
Created attachment 276946 [details]
Bug details

I've been able to reproduce the bug, this time I used the feedback agent. I hope this will help.

Comment 9 Wayne Mery (:wsmwk, NI for questions) 2008-02-14 11:14:45 PST
ranked #17 on talkback for TB2009 => topcrash
Comment 10 Wayne Mery (:wsmwk, NI for questions) 2008-07-05 06:26:51 PDT
is nsFolderCompactState::EndCopy the same crash?  (it's also a topcrash)

I doubt all these crashes are specifically "rebuild index during compact".

"I had created a subfolder three levels in which did not appear right away. I created it again and the system complained that a folder by that name already existed. I closed the parent folder and reopened, which succeeded in displaying the subfolder, but"
Since Last Crash	140065 sec
Total Uptime	156766 sec
Trigger Reason	Access violation
Source File, Line No.	e:/builds/tinderbox/Tb-Mozilla1.8-Release/WINNT_5.0_Depend/mozilla/mailnews/base/src/nsMsgFolderCompactor.cpp, line 995
Stack Trace 	
nsFolderCompactState::EndCopy  [mozilla/mailnews/base/src/nsMsgFolderCompactor.cpp, line 995]
nsMailboxProtocol::OnStopRequest  [mozilla/mailnews/local/src/nsMailboxProtocol.cpp, line 301]
nsInputStreamPump::OnStateStop  [mozilla/netwerk/base/src/nsInputStreamPump.cpp, line 564]

Comment 11 WADA 2008-07-07 17:32:37 PDT
I'm unable to re-produce crash with Tb Win-XP SP2) and test scenario of comment #1. Tb also issued warning when I tried to "Rebuild Index" while "Compact" is in progress. 

To benoit(bug opener):
Can you re-produce the crash problem with newer release or trunk?
Comment 12 David :Bienvenu 2008-07-08 09:03:37 PDT
I can reproduce this - the trick is to have very large messages in the folder that's being compacted, to increase the odds that you rebuild the index while in the middle of copying a single message during the compact process. In other words, the copy of the single message has to cause us to go through the event loop multiple times, and one of those times the user has to click rebuild index. This deletes the db out from under the message copy.

There are two ways to fix this - first, the rebuild index code should check if the folder is locked; if so, it should report an error and not try to rebuild the index. Second, the folder compact code could register to get told about the db getting closed out from under it, and handle that gracefully. But given that the folder compaction code has locked the folder, I think it should reasonably expect not to have the db closed.
Comment 13 David :Bienvenu 2008-07-08 10:11:52 PDT
Created attachment 328513 [details] [diff] [review]
proposed fix 

SM and TB patch.
Comment 14 2008-07-08 14:02:44 PDT
Comment on attachment 328513 [details] [diff] [review]
proposed fix 

>+    var errorMessage = gMessengerBundle.getString("operationFailedFolderBusy");
The other consumers of this string pass it to throwAlertMsg - I think it would be a good idea if we could be consistent here?
Comment 15 David :Bienvenu 2008-07-08 16:26:39 PDT
Comment on attachment 328513 [details] [diff] [review]
proposed fix 

good idea - I didn't realize throwAlertMsg was exposed in the idl. I'll rework the patch.
Comment 16 David :Bienvenu 2008-07-09 09:18:27 PDT
Created attachment 328693 [details] [diff] [review]
use throwAlert
Comment 17 David :Bienvenu 2008-07-25 15:57:28 PDT
fixed on trunk.
Comment 18 David :Bienvenu 2008-07-25 15:58:12 PDT
Comment on attachment 328693 [details] [diff] [review]
use throwAlert

simple fix for crash...
Comment 19 David :Bienvenu 2008-08-15 12:24:14 PDT
adding checkin-needed keyword for 1.8.1 branch checkin
Comment 20 Phil Ringnalda (:philor) 2008-08-15 18:53:13 PDT

Note You need to log in before you can comment on or make changes to this bug.