Closed Bug 598104 Opened 14 years ago Closed 14 years ago

Local Sent folder allowed to grow past 4GB

Categories

(MailNews Core :: Backend, defect)

x86
Windows 7
defect
Not set
critical

Tracking

(thunderbird3.1 .7-fixed)

RESOLVED FIXED
Thunderbird 3.3a1
Tracking Status
thunderbird3.1 --- .7-fixed

People

(Reporter: Bienvenu, Assigned: Bienvenu)

References

()

Details

(Keywords: dataloss, Whiteboard: [gs])

Attachments

(2 files, 1 obsolete file)

We're apparently not doing a size check on the local sent folder when appending to it, and users are getting sent folders > 4GB, without an error or warning.
Bug 387502 couldn't resolve local Sent folder case? If so, why? "Copy to Sent after mail send" doesn't open Sent folder(MsgDB)? Silently appends sent mail data to file of Sent and tries to update .msf?
Copy to sent uses a different mechanism from either of the issues fixed in that bug, I believe.
How about Drafts? AFAIK, "write failure to Drafts" falls back to Sent folder, so Drafts is similar to Sent, although "save to Drafts" is different from "copy to Sent" because "save to Drafts" has "delete previous version of draft" step.
Dear Wada

As you know, I come from #477430 comment #18 which content I won't repeat here.

You said that Linux limit for a mailbox was 2GB. I can guarantee you that I personally migrated (rescued) that user from a dying windoze box to 64-bit Ubuntu Lucid, that the Inbox mailbox was about 3.5GB, that I told the user that she should erase or archive, which she never did, happilly continuing to receive messages, and that everything bumped on 4GB, and only the procedure I described on the above comment enabled me to put an end to her panic.

Cheers

Antonio
linux limit used to be 2GB - that was fixed so that it's now 4GB
Linkifying for ease of access: bug 477430 comment #18
Severity: normal → critical
Keywords: dataloss
Attached patch proposed fixSplinter Review
I haven't tested this, or written a unit test, but this is what I have in mind.
Assignee: nobody → bienvenu
Status: NEW → ASSIGNED
Comment on attachment 487038 [details] [diff] [review]
proposed fix

This works fine - I'll try to get a unit test that hijacks the msgWindow alert to verify that the alert is thrown, but I think we're going to want this for 3.1.7 so I better get it in standard8's review q :-)
Attachment #487038 - Flags: review?(bugzilla)
bug 608449 filed for compaction of > 4GB folder failing.
Comment on attachment 487038 [details] [diff] [review]
proposed fix

Looks good, r=Standard8
Attachment #487038 - Flags: review?(bugzilla) → review+
Attached patch add unit test to detect this (obsolete) — Splinter Review
This adds a unit test for this - I had to tweak nsMsgDBFolder.cpp to use the nsMsgWindow's alert method so our test mechanism of overriding the msg window's alert would work.
Attachment #487755 - Flags: review?(bugzilla)
http://hg.mozilla.org/comm-central/rev/425b0d7f9506 - fixed on trunk.
Status: ASSIGNED → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
Comment on attachment 487038 [details] [diff] [review]
proposed fix

getting on 3.1.7 radar
Attachment #487038 - Flags: approval-thunderbird3.1.7?
FYI.
Manual recovery procedure sample, until bug 608449 will be fixed.
(Information in bug 387502 comment #107 and bug 387502 comment #118)

(1) Terminate Tb.
(2) Rename file named "Sent" to "SentX". (keep backup)
    Rename file named "Sent.msf" to "Sent". (new/cleared Sent folder)
    (same as delete "Sent.msf" then create null file of "Sent")  
(3) Restart Tb, click Sent folder, copy a mail to Sent folder.
(4) Split SentX to multiple mail folder files which are smaller than size limit.
(4-1) Create SentX-1
  SentX-1 : data from offset=0 to 2GB-1 (2GB data) + 0x0D0A([CRLF])
  if "From " line(separator of uix mbox) is split, change size.
(4-2) Restart Tb, click SentX-1, wait for rebuild-index of SentX-1.
  Show "Order Received" column, sort in ascending order.
  "Order Received" column valu is "offset in mail folder file",
  when local mail folder.
  XXX : "Order Received" column value of bottom most mail(mail of largest XXX).
(4-3) Create SentX-2
  SentX-2 : data from offset=XXX to end of SentX.
(4-4) Restart Tb, click SentX-2, wait for rebuild-index of SentX-2.
  Show "Order Received" column, sort in ascending order.
  Last mail in SentX-1 can be partial mail of first mail in SentX-2.
  Delete last mail in SentX-1, if the last mail in SentX-1 is really partial
  mail of first mail in SentX-2.
(4-5) If file size of SentX-2 is still larger than file size limit,
      split SentX-2 into SentX-2-A and SentX-2-B.

See next document for how to spliting large unix mbox file.
> http://kb.mozillazine.org/Edit_large_mbox_files
See also next document.
> http://kb.mozillazine.org/Limits_-_Thunderbird#Folders_and_messages
Comment on attachment 487755 [details] [diff] [review]
add unit test to detect this

this test does fail on the mac, but passes on Windows. I'll try to figure out what's different.
Attachment #487755 - Flags: review?(bugzilla)
I suspect the test failure is due to a problem creating the 4GB file - when it gets to the code that checks the file size, it's 0, so it's either a problem in the mark_file_region_sparse code on the mac, or in my particular test...
Attachment #487755 - Attachment is obsolete: true
Attachment #488786 - Flags: review?(bugzilla)
Comment on attachment 488786 [details] [diff] [review]
fix unit test for mac - checked in.

>+  try {
>+  copyService.CopyFileMessage(file, gLocalInboxFolder, null, false, 0,
>+                              "", copyListener, dummyMsgWindow);
>+  } catch (ex) {

nit: copyService and the following line need two more spaces indentation.
Attachment #488786 - Flags: review?(bugzilla) → review+
unit test landed...
Flags: in-testsuite+
Target Milestone: --- → Thunderbird 3.3a1
Attachment #488786 - Attachment description: fix unit test for mac → fix unit test for mac - checked in.
Comment on attachment 487038 [details] [diff] [review]
proposed fix

a=Standard8 for both the patch and the unit test.
Attachment #487038 - Flags: approval-thunderbird3.1.7? → approval-thunderbird3.1.7+
Attachment #488786 - Flags: approval-thunderbird3.1.7?
Comment on attachment 488786 [details] [diff] [review]
fix unit test for mac - checked in.

clearing 3.1.7 review request - the test needs to be tweaked quite a bit for 3.1.7, and I'm going to do that in the bug for parsing/compaction of > 4gb files.
Attachment #488786 - Flags: approval-thunderbird3.1.7?
Do you need help in testing? I am keeping on fixing 4GB files over and over on behalf of one of the users I have to assist...
My bug was 610772.  It would be great if there was some way to warn us when our Sent file is reaching 4GB.  I'm a real greenhorn and didn't know I was supposed to compact the files - which I now intend to do.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: