Open Bug 1093217 Opened 7 years ago Updated 2 years ago

folderSize for imap (sizeOnDisk) is lost if panacea.dat is deleted

Categories

(MailNews Core :: Backend, defect, P2)

defect

Tracking

(Not tracked)

People

(Reporter: rkent, Assigned: benc)

References

(Depends on 1 open bug, Blocks 1 open bug)

Details

Attachments

(1 file)

STR:

1) For an IMAP folder, delete panacea.dat prior to startup
2) Start Thunderbird

Expected results: In Folder Properties, you see sizeOnDisk (which represents the size of the storage on the server)

Actual results: sizeOnDisk is reported as zero.

sizeOnDisk recovers if you rebuild the folder. This is also easier to see using the Extra Folder Columns extension, which displays folderSize.
Blocks: 1032360
Attached patch 1093217.patchSplinter Review
Here's my fix. This uses existing patches from bug 813459 and bug 813249 which move folderSize to int64_t. This patch should be revisited and reviewed after those are resolved.
Assignee: nobody → kent
Status: NEW → ASSIGNED
Depends on: 813459, 813249
Sorry for the wrong dependency
No longer depends on: 813249
To make this work correctly, we need to accurately keep track of a sum of the individual message sizes in the database. But folderSize is used in mbox as an indicator of db validity, and in that case (because of possible deleted messages) the folderSize may not equal the sum of the message sizes for individual messages. For both maildir (see bug 1032360) and in IMAP, we would like to have a dbFolderInfo variable that represents the sum of all of the messages sizes in the database.

So it seems to me that the safest way to handle this will be to add a new variable to dbFolderInfo which represents the sum of message sizes, and add the hooks to properly maintain this variable as changes are made to the db. Then the folder can choose which representation it wants to report in the sizeOnDisk attribute. That will provide backwards compatibility, at the cost of confusing naming of various attributes: "sizeOnDisk" in the folder (which is what is reported in the UI), "folderSize" in dbFolderInfo and panacea.dat, and a new "totalSize" in dbFolderInfo and panacea.dat which is the sum of message sizes in the db.

I'll have to try some code to see if all of this works or not.
I think the blocking bugs have landed, you can try continuing with this patch.
OS: Windows 8.1 → All
Hardware: x86_64 → All
Priority: -- → P2
Aceman, anything we should pick up?

Anecdotal: Today I deleted my panacea.dat with the consequence that about 1000 MSF files got truncated to 0 bytes.
http://kb.mozillazine.org/Files_and_folders_in_the_profile_-_Thunderbird says:
  panacea.dat - Mail folder cache. It can be safely deleted to resolve various issues.
Well, I had heaps of problems after deleting it. Filters didn't work, all read indicators lost, etc.
Flags: needinfo?(acelists)
see also bug 131589
Flags: needinfo?(acelists)
Assignee: rkent → benc
Status: ASSIGNED → NEW

Looking at comment #6 and bug 131589:
Corrupt panacea.dat causing us to lose messages in offline store, and redownload message headers/rebuild .msf files of suffixed file name like folder-1.msf

There's a bug I've never filed but which seems related to bug 131589 and this one here: Deleting panacea.dat will cause loss of all .MSF files.

I did that once and about 1700 .MSF files got set to zero byte size. As a consequence all unread counts were lost and worse, filters into those folders stopped working. In brief: Deleting panacea.dat causes havoc.

You need to log in before you can comment on or make changes to this bug.