Closed Bug 471307 Opened 16 years ago Closed 16 years ago

"Rebuild Index" feature creating corrupted .msf index files

Categories

(Thunderbird :: General, defect, P1)

x86
Windows XP
defect

Tracking

(Not tracked)

VERIFIED FIXED
Thunderbird 3.0b2

People

(Reporter: fehe, Assigned: Bienvenu)

References

Details

(Keywords: regression)

Attachments

(1 file)

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b3pre) Gecko/20081227 Lightning/1.0pre

Folder index files are being corrupted when right-clicking a folder and doing "Properties... --> General Information --> Rebuild Index".  The messages are still listed, but their content is no longer viewable.  Once this occurs, the only solution is to shutdown Shredder and manually delete the associated .msf file.

Regression Range:
Works: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b3pre) Gecko/20081219 Shredder/3.0b2pre

Broken: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b3pre) Gecko/20081220 Shredder/3.0b2pre


Reproducible: Always

Steps to Reproduce:
1. Right-click a folder (containing messages) and select "Properties... --> General Information --> Rebuild Index"
2. Click a different folder then reselect the folder in Step #1 and attempt to view any of the messages.  Notice that no message content can be displayed.
3. Shutdown Shredder and delete the corresponding .msf file for the folder
4. Start Shredder and notice that you can now view the message content.
Flags: blocking-thunderbird3?
Keywords: regression
Version: unspecified → Trunk
Confirmed with Tb trunk on MS Win-XP SP3.
> Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b3pre) Gecko/20081227 Shredder/3.0b2pre

(1) Rebuild Index(of a local mail folder)
(2) Click some mails in the folder => Message pane was not correctly updated.
    Following error in Error Console when mail click.
> Error: uncaught exception: [Exception... "Component returned failure code: 0x80550008 [nsIMsgDBHdr.markHasAttachments]" nsresult: "0x80550008 (<unknown>)"
>  location: "JS frame :: chrome://messenger/content/msgHdrViewOverlay.js :: anonymous :: line 520"  data: no]
(3) Move a mail to the troubled mail folder from other mail folder
    => Problem disappeared. Mails were displayed normally.
       Above error message was not issued, until Rebuid Index again.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Confirmed on Win Vista HP SP1 (Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b3pre) Gecko/20081228 Lightning/1.0pre Shredder/3.0b2pre).

The index corruption happens most often on account's Sent folder, after sendin a message.
yikes, I wonder if this had to do with the changes to the db opening api...I'll try to figure this out.
Assignee: nobody → bienvenu
Severity: normal → critical
Status: NEW → ASSIGNED
Flags: blocking-thunderbird3? → blocking-thunderbird3+
Priority: -- → P1
Target Milestone: --- → Thunderbird 3.0b2
In my test case, just clicking away to a different folder and then back to the original folder fixes the problem. I'm not sure if the db is getting regenerated in that case. The message key in the view is the same in both cases, so it's not that we have the wrong offset into the mailbox.
Not so here, I have the sent folder in the "locked up" state and get this:
Error: Component returned failure code: 0x80550005 [nsIMsgFolder.getMsgDatabase] = <unknown>
Source file: chrome://messenger/content/mailWidgets.xml
Line: 1929

Probably trying to open a message in a new tab, at that point.
Here is another related error:
Error: Component returned failure code: 0x80550005 [nsIMessenger.msgHdrFromURI] = <unknown>
Source file: chrome://messenger/content/messageWindow.js
Line: 382

Double clicking to open in a new window
Two things are going wrong, from what I can tell:

CloseAndBackupDB is not backing up the db, because mFlags is 0.
GetDatabaseWithReparse is failing because OpenFolderDB is returning NS_MSG_ERROR_FOLDER_SUMMARY_OUT_OF_DATE and a null mDatabase. In that case, GetDatabaseWithReparse does nothing useful.

Sometimes OpenFolderDB returns NS_MSG_ERROR_FOLDER_SUMMARY_OUT_OF_DATE and a non-null mDatabase, and then GetDatabaseWithReparse works and the bug doesn't occur.
Joe, you can probably discover about 50 things that go wrong if we can't open the database for the folder. We need to fix the underlying problem.
(In reply to comment #2)
> The index corruption happens most often on account's Sent folder, after sendin
> a message.

This is how I first ran into the problem, but didn't realize that the corruption occurred after sending messages, because I didn't observe the state of the Sent folder before sending a message.  It was only afterward, when I tried to double-check what I had sent, that I found my Sent messages were inaccessible.

Thanks for pointing this out.
OS: Windows XP → Windows NT
Summary: "Rebuild Index" feature creating corrupted .msf index files → Sending email or using "Rebuild Index" feature resulting in corrupted .msf index files
I believe the problem is that we never delete the .msf file in this case, so we reparse the folder and add headers to an existing mdb, where everything is duplicated. But debugging this is a bit challenging on the mac...
For a while now, simply opening the sent folder has resulted in an auto reindex.
Looking at the regression range, I'd suspect:
http://hg.mozilla.org/comm-central/rev/e10a3baac8d7
OS: Windows NT → Windows XP
Summary: Sending email or using "Rebuild Index" feature resulting in corrupted .msf index files → "Rebuild Index" feature creating corrupted .msf index files
I'm not sure if this can help, but I can provide two Sent.msf files: the 'broken' one (after sending a message it was impossible to view the message bodies) and the second, which was recreated after TB restart. 

The files are zipped: http://www.nncron.ru/temp/sent.zip (~540kB)
Attached patch proposed fixSplinter Review
proposed fix - this fixes it for me. Asking jcranmer for review since I think this was broken in the db open reworking code.
Attachment #354770 - Flags: superreview?(bugzilla)
Attachment #354770 - Flags: review?(Pidgeot18)
Comment on attachment 354770 [details] [diff] [review]
proposed fix

I swore I changed all of those...

From grep, this is the last one.
Attachment #354770 - Attachment is patch: true
Attachment #354770 - Flags: review?(Pidgeot18) → review+
Attachment #354770 - Flags: superreview?(bugzilla) → superreview+
fix checked in.
Status: ASSIGNED → RESOLVED
Closed: 16 years ago
Resolution: --- → FIXED
"Rebuild Index" is fixed, but the corruption on sending that was mentioned in Comment #2, and which I acknowledged in Comment #9 remains.

I have filed Bug 471682 for the corruption on sending.
Status: RESOLVED → VERIFIED
if you look far enough into the openfolderdb I think it can return a NS_ERROR_FILE_TARGET_DOES_NOT_EXIST so you may want to || it.
if you look far enough into the openfolderdb I think it can return a NS_ERROR_FILE_TARGET_DOES_NOT_EXIST so you may want to || it.
is this still considered to be a regression?  of Bug 437886?
yes, correct, it's a regression from bug 437886
Blocks: 437886
Depends on: 472446
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: