Note: There are a few cases of duplicates in user autocompletion which are being worked on.

Local Sent folder allowed to grow past 4GB

RESOLVED FIXED in Thunderbird 3.3a1

Status

MailNews Core
Backend
--
critical
RESOLVED FIXED
7 years ago
7 years ago

People

(Reporter: Bienvenu, Assigned: Bienvenu)

Tracking

({dataloss})

unspecified
Thunderbird 3.3a1
x86
Windows 7
dataloss
Bug Flags:
in-testsuite +

Thunderbird Tracking Flags

(thunderbird3.1 .7-fixed)

Details

(Whiteboard: [gs], URL)

Attachments

(2 attachments, 1 obsolete attachment)

(Assignee)

Description

7 years ago
We're apparently not doing a size check on the local sent folder when appending to it, and users are getting sent folders > 4GB, without an error or warning.
(Assignee)

Updated

7 years ago
Bug 387502 couldn't resolve local Sent folder case? If so, why? "Copy to Sent after mail send" doesn't open Sent folder(MsgDB)? Silently appends sent mail data to file of Sent and tries to update .msf?
(Assignee)

Comment 2

7 years ago
Copy to sent uses a different mechanism from either of the issues fixed in that bug, I believe.
How about Drafts? AFAIK, "write failure to Drafts" falls back to Sent folder, so Drafts is similar to Sent, although "save to Drafts" is different from "copy to Sent" because "save to Drafts" has "delete previous version of draft" step.
Dear Wada

As you know, I come from #477430 comment #18 which content I won't repeat here.

You said that Linux limit for a mailbox was 2GB. I can guarantee you that I personally migrated (rescued) that user from a dying windoze box to 64-bit Ubuntu Lucid, that the Inbox mailbox was about 3.5GB, that I told the user that she should erase or archive, which she never did, happilly continuing to receive messages, and that everything bumped on 4GB, and only the procedure I described on the above comment enabled me to put an end to her panic.

Cheers

Antonio
(Assignee)

Comment 5

7 years ago
linux limit used to be 2GB - that was fixed so that it's now 4GB
Linkifying for ease of access: bug 477430 comment #18
Blocks: 387502
Duplicate of this bug: 584685
Severity: normal → critical
Keywords: dataloss
(Assignee)

Updated

7 years ago
Duplicate of this bug: 607036
(Assignee)

Comment 9

7 years ago
Created attachment 487038 [details] [diff] [review]
proposed fix

I haven't tested this, or written a unit test, but this is what I have in mind.
Assignee: nobody → bienvenu
Status: NEW → ASSIGNED
(Assignee)

Comment 10

7 years ago
Comment on attachment 487038 [details] [diff] [review]
proposed fix

This works fine - I'll try to get a unit test that hijacks the msgWindow alert to verify that the alert is thrown, but I think we're going to want this for 3.1.7 so I better get it in standard8's review q :-)
Attachment #487038 - Flags: review?(bugzilla)
(Assignee)

Comment 11

7 years ago
bug 608449 filed for compaction of > 4GB folder failing.
Duplicate of this bug: 584612
Comment on attachment 487038 [details] [diff] [review]
proposed fix

Looks good, r=Standard8
Attachment #487038 - Flags: review?(bugzilla) → review+
Duplicate of this bug: 556330
(Assignee)

Comment 15

7 years ago
Created attachment 487755 [details] [diff] [review]
add unit test to detect this

This adds a unit test for this - I had to tweak nsMsgDBFolder.cpp to use the nsMsgWindow's alert method so our test mechanism of overriding the msg window's alert would work.
Attachment #487755 - Flags: review?(bugzilla)
(Assignee)

Comment 16

7 years ago
http://hg.mozilla.org/comm-central/rev/425b0d7f9506 - fixed on trunk.
Status: ASSIGNED → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
(Assignee)

Comment 17

7 years ago
Comment on attachment 487038 [details] [diff] [review]
proposed fix

getting on 3.1.7 radar
Attachment #487038 - Flags: approval-thunderbird3.1.7?
FYI.
Manual recovery procedure sample, until bug 608449 will be fixed.
(Information in bug 387502 comment #107 and bug 387502 comment #118)

(1) Terminate Tb.
(2) Rename file named "Sent" to "SentX". (keep backup)
    Rename file named "Sent.msf" to "Sent". (new/cleared Sent folder)
    (same as delete "Sent.msf" then create null file of "Sent")  
(3) Restart Tb, click Sent folder, copy a mail to Sent folder.
(4) Split SentX to multiple mail folder files which are smaller than size limit.
(4-1) Create SentX-1
  SentX-1 : data from offset=0 to 2GB-1 (2GB data) + 0x0D0A([CRLF])
  if "From " line(separator of uix mbox) is split, change size.
(4-2) Restart Tb, click SentX-1, wait for rebuild-index of SentX-1.
  Show "Order Received" column, sort in ascending order.
  "Order Received" column valu is "offset in mail folder file",
  when local mail folder.
  XXX : "Order Received" column value of bottom most mail(mail of largest XXX).
(4-3) Create SentX-2
  SentX-2 : data from offset=XXX to end of SentX.
(4-4) Restart Tb, click SentX-2, wait for rebuild-index of SentX-2.
  Show "Order Received" column, sort in ascending order.
  Last mail in SentX-1 can be partial mail of first mail in SentX-2.
  Delete last mail in SentX-1, if the last mail in SentX-1 is really partial
  mail of first mail in SentX-2.
(4-5) If file size of SentX-2 is still larger than file size limit,
      split SentX-2 into SentX-2-A and SentX-2-B.

See next document for how to spliting large unix mbox file.
> http://kb.mozillazine.org/Edit_large_mbox_files
See also next document.
> http://kb.mozillazine.org/Limits_-_Thunderbird#Folders_and_messages
(Assignee)

Comment 19

7 years ago
Comment on attachment 487755 [details] [diff] [review]
add unit test to detect this

this test does fail on the mac, but passes on Windows. I'll try to figure out what's different.
Attachment #487755 - Flags: review?(bugzilla)
(Assignee)

Comment 20

7 years ago
I suspect the test failure is due to a problem creating the 4GB file - when it gets to the code that checks the file size, it's 0, so it's either a problem in the mark_file_region_sparse code on the mac, or in my particular test...
(Assignee)

Comment 21

7 years ago
Created attachment 488786 [details] [diff] [review]
fix unit test for mac - checked in.
Attachment #487755 - Attachment is obsolete: true
Attachment #488786 - Flags: review?(bugzilla)
Comment on attachment 488786 [details] [diff] [review]
fix unit test for mac - checked in.

>+  try {
>+  copyService.CopyFileMessage(file, gLocalInboxFolder, null, false, 0,
>+                              "", copyListener, dummyMsgWindow);
>+  } catch (ex) {

nit: copyService and the following line need two more spaces indentation.
Attachment #488786 - Flags: review?(bugzilla) → review+
(Assignee)

Comment 23

7 years ago
unit test landed...
Flags: in-testsuite+
(Assignee)

Updated

7 years ago
Target Milestone: --- → Thunderbird 3.3a1
(Assignee)

Updated

7 years ago
Attachment #488786 - Attachment description: fix unit test for mac → fix unit test for mac - checked in.
Comment on attachment 487038 [details] [diff] [review]
proposed fix

a=Standard8 for both the patch and the unit test.
Attachment #487038 - Flags: approval-thunderbird3.1.7? → approval-thunderbird3.1.7+
(Assignee)

Updated

7 years ago
Attachment #488786 - Flags: approval-thunderbird3.1.7?
(Assignee)

Comment 25

7 years ago
Comment on attachment 488786 [details] [diff] [review]
fix unit test for mac - checked in.

clearing 3.1.7 review request - the test needs to be tweaked quite a bit for 3.1.7, and I'm going to do that in the bug for parsing/compaction of > 4gb files.
Attachment #488786 - Flags: approval-thunderbird3.1.7?
(Assignee)

Comment 26

7 years ago
fixed for 3.1.7 - http://hg.mozilla.org/releases/comm-1.9.2/rev/7539124e3043
status-thunderbird3.1: --- → .7-fixed

Updated

7 years ago
Duplicate of this bug: 610772
Do you need help in testing? I am keeping on fixing 4GB files over and over on behalf of one of the users I have to assist...

Comment 29

7 years ago
My bug was 610772.  It would be great if there was some way to warn us when our Sent file is reaching 4GB.  I'm a real greenhorn and didn't know I was supposed to compact the files - which I now intend to do.
Duplicate of this bug: 615347
Duplicate of this bug: 602360
You need to log in before you can comment on or make changes to this bug.