392015 - Crash of Tb 2.0.0.6, if "Rebuild Index" is executed during compacting folder is inprogess, and loosing mails [@ morkRowObject::SetRow]

Status: RESOLVED FIXED

(MailNews Core :: Database, defect)

Description

[benoit]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6
Build Identifier: version 2.0.0.6 (20070728)

During thunderbird were compacting a folder, it crashed and messages of this folder were all deleted after restarting thunderbird

Reproducible: Always

Steps to Reproduce:
1.right click on a folder
2.Compact
3.right click on the same folder
4.properties > rebuilt index
5.crash!!
6.all mail of the folder disappear
Actual Results:  
After the crash I restart thunderbird. Some messages that disappeared from the folder were store on a folder nstmp, but most of messages were lost.

Expected Results:  
I don't know, but it should make a backup or compact messages per messages to avoid dataloss in case of crash.

no additional theme were use
no additional modules were use

Comment 1

[WADA:World Anti-bad-Duping Agency]

> 2.Compact
> 3.right click on the same folder
> 4.properties > rebuilt index
> 5.crash!!

Did you execute "Rebuild Index" at step 4 after completion of "Compact Folder" at step 2? Or you did "Rebuild Index" while "Compact Folder" is in progress?
If latter, no crash when Thunderbird trunk nightly(2007/08/09-12 build,MS Win-XP), then there is no critical problem like data loss after restart.

(1) "Compact Folder"
    => nstmp & nstmp.msf is created, and data is being copied to nstmp
(2) "Rebuild Index" while "Compact Folder" is in progress
    => Warnig dialog of "other process in progress, retry after end"
(3) Reply "OK" to dialog
    => nstmp & nstmp.msf was deleted
       <folder_name>.msf for the folder compacted was deleted 
    => Following errors in Error Console
(For compact)
> Error: [Exception... "Component returned failure code:
> 0x80550006 [nsIMsgFolder.getMsgDatabase]"  nsresult: "0x80550006 (<unknown>)"
> location: "JS frame :: chrome://messenger/content/msgMail3PaneWindow.js ::
> HandleCompactCompleted :: line 526"  data: no]
> Source File: chrome://messenger/content/msgMail3PaneWindow.js Line: 526
(For rebuild index)
> Error: uncaught exception: [Exception... "Component returned failure code:
> 0x8055000a [nsIMsgFolder.updateFolder]"  nsresult: "0x8055000a (<unknown>)"
> location: "JS frame :: chrome://messenger/content/widgetglue.js ::
> RebuildSummaryFile :: line 249"  data: no]

(4) "Rebuild Index" again several times
    => Following error in Error Console on each "Rebuild Index" request 
> Error: uncaught exception: [Exception... "Component returned failure code:
> 0x80550006 [nsIMsgFolder.getMsgDatabase]"  nsresult: "0x80550006 (<unknown>)"
> location: "JS frame :: chrome://messenger/content/widgetglue.js ::
> RebuildSummaryFile :: line 237"  data: no]

(5) Close property dialog => error messages were issued 
(6) Terminate Thunderbird, and Restart Thunderbird
    => Because no .msf, .msf recreation is successfully executed
    => Because no garbage of nstmp & nstmp.msf, no garbage in folder pane

Comment 2

[WADA:World Anti-bad-Duping Agency]

To David Bienvenu:
Crash(and dataloss for user) of Tb 2.0 is sufficiently critical problem, even if no crash when trunk. Should we keep separate bugs for crash of 2.0 and mail db damage problem of trunk? 

Comment 3

[David :Bienvenu]

I'm more worried about the 2.0 problem - on the trunk, the db gets regenerated so that doesn't bother me so much.

Comment 4

[WADA:World Anti-bad-Duping Agency]

I tested with Tb 2.0.0.5(Win-XP) using a folder of 80,000 junk mails, but crash was not re-produced.
(1) "Rebuild Index" while "Compact Folder" is in progress.
(2) Dialog of "other process is in progress", and reply "OK"
    => nstmp & nstmp.msf was deleted (i.e. compact aborted)
       (error message in error console appeared)
    => mail count of the folder became far smaller than 80,000
       (error message in error console appeared)
(3) "Rebuild Index"
    => "Rebuild Index" completed, but count became less than 80,000
(4) Click other folder and click the folder again
    => Mail count of the folder was corrected to 80,000

To benoit(the bug oper):
What is Talkback-ID of your crash?
Do you use add-on(extension)?
If yes, can you re-produce crash with -safe-mode of Thunderbird?

Comment 5

[WADA:World Anti-bad-Duping Agency]

Crash was re-produced with Thunderbird 2.0.0.6(release build, en-US, MS Win-XP), by same test procedure as comment #4.
Crash when step (1), "Rebuild Index" while "Compact Folder" is in progress.
nstmp & nstmp.msf remained after crash.
Talkback-ID = TB34936806Y

Comment 6

[WADA:World Anti-bad-Duping Agency]

Confirming.

Comment 7

[timeless]

Incident ID: 34936806
Stack Signature	morkRowObject::SetRow 1dbd471f
Product ID	Thunderbird2
Build ID	2007072817
Trigger Time	2007-08-14 08:07:01.0
Platform	Win32
Operating System	Windows NT 5.1 build 2600
Module	thunderbird.exe + (0004f656)
URL visited	
User Comments	This is crash report of Bug 392015
Since Last Crash	82 sec
Total Uptime	82 sec
Trigger Reason	Access violation
Source File, Line No.	e:/builds/tinderbox/Tb-Mozilla1.8-Release/WINNT_5.0_Depend/mozilla/db/mork/src/morkRowObject.cpp, line 462
Stack Trace 	
morkRowObject::SetRow  [mozilla/db/mork/src/morkRowObject.cpp, line 462]
nsFolderCompactState::EndCopy  [mozilla/mailnews/base/src/nsMsgFolderCompactor.cpp, line 996]
nsMailboxProtocol::OnStopRequest  [mozilla/mailnews/local/src/nsMailboxProtocol.cpp, line 301]
nsInputStreamPump::OnStateStop  [mozilla/netwerk/base/src/nsInputStreamPump.cpp, line 564]
0x0755cd68
nsMailboxProtocol::`vftable'
nsMsgProgress::AddRef  [mozilla/mailnews/base/src/nsMsgProgress.cpp, line 49]
0x8b560c45

Comment 8

[benoit]

Attachment: Bug details

I've been able to reproduce the bug, this time I used the feedback agent. I hope this will help.

Benoit

Comment 9

[Wayne Mery (:wsmwk)]

ranked #17 on talkback for TB2009 => topcrash

Comment 10

[Wayne Mery (:wsmwk)]

is nsFolderCompactState::EndCopy the same crash?  (it's also a topcrash)

I doubt all these crashes are specifically "rebuild index during compact".

TB46631095
"I had created a subfolder three levels in which did not appear right away. I created it again and the system complained that a folder by that name already existed. I closed the parent folder and reopened, which succeeded in displaying the subfolder, but"
Since Last Crash	140065 sec
Total Uptime	156766 sec
Trigger Reason	Access violation
Source File, Line No.	e:/builds/tinderbox/Tb-Mozilla1.8-Release/WINNT_5.0_Depend/mozilla/mailnews/base/src/nsMsgFolderCompactor.cpp, line 995
Stack Trace 	
nsFolderCompactState::EndCopy  [mozilla/mailnews/base/src/nsMsgFolderCompactor.cpp, line 995]
nsMailboxProtocol::OnStopRequest  [mozilla/mailnews/local/src/nsMailboxProtocol.cpp, line 301]
nsInputStreamPump::OnStateStop  [mozilla/netwerk/base/src/nsInputStreamPump.cpp, line 564]
0x0315cc10

Comment 11

[WADA:World Anti-bad-Duping Agency]

I'm unable to re-produce crash with Tb 2.0.0.14(MS Win-XP SP2) and test scenario of comment #1. Tb 2.0.0.14 also issued warning when I tried to "Rebuild Index" while "Compact" is in progress. 

To benoit(bug opener):
Can you re-produce the crash problem with newer release or trunk?

Comment 12

[David :Bienvenu]

I can reproduce this - the trick is to have very large messages in the folder that's being compacted, to increase the odds that you rebuild the index while in the middle of copying a single message during the compact process. In other words, the copy of the single message has to cause us to go through the event loop multiple times, and one of those times the user has to click rebuild index. This deletes the db out from under the message copy.

There are two ways to fix this - first, the rebuild index code should check if the folder is locked; if so, it should report an error and not try to rebuild the index. Second, the folder compact code could register to get told about the db getting closed out from under it, and handle that gracefully. But given that the folder compaction code has locked the folder, I think it should reasonably expect not to have the db closed.

Comment 13

[David :Bienvenu]

Attachment: proposed fix

SM and TB patch.

Comment 14

[[e-mail]]

Comment on attachment 328513 [details] [diff] [review]
proposed fix 

>+    var errorMessage = gMessengerBundle.getString("operationFailedFolderBusy");
The other consumers of this string pass it to throwAlertMsg - I think it would be a good idea if we could be consistent here?

Comment 15

[David :Bienvenu]

Comment on attachment 328513 [details] [diff] [review]
proposed fix 

good idea - I didn't realize throwAlertMsg was exposed in the idl. I'll rework the patch.

Comment 16

[David :Bienvenu]

Attachment: use throwAlert

Comment 17

[David :Bienvenu]

fixed on trunk.

Comment 18

[David :Bienvenu]

Comment on attachment 328693 [details] [diff] [review]
use throwAlert

simple fix for crash...

Comment 19

[David :Bienvenu]

adding checkin-needed keyword for 1.8.1 branch checkin

Comment 20

[Phil Ringnalda (:philor)]

mailnews/base/resources/content/widgetglue.js 1.171.4.3
mail/base/content/widgetglue.js 1.15.2.5