294647 - Junk mailbox grows to 2 gigs and Thunderbird hangs [Mac]

Status: RESOLVED INCOMPLETE

(MailNews Core :: Filters, defect)

Description

[Elliotte Rusty Harold]

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.8) Gecko/20050511 Firefox/1.0.4
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.8) Gecko/20050511 Firefox/1.0.4

The recent neo-nazi spam invasion pushed my local junk mail box to 2 gigabytes.
Once this happened, Thunderbird repeatedly hung with the spinning beach ball of
death.  

Reproducible: Always

Steps to Reproduce:
1. Collect two gigabytes worth of spam
2. Wait for a message to be automarked as junk
3.

Actual Results:  
Thunderbird hung with the spinning beach ball of death.  

Expected Results:  
I'm not sure. It shoudln't hang. Possibly it should stop moving mail into the
Junk folder. Possibly it shoudl create a 2nd Junk folder. It shoudl probably
alert the user to the problem and ask them if they want to trim old junk mail.

This may well be a problem beyond the Junk mail folder, of course. I've just
never had another folder get close to the 2-gig limit.


This is probably platform specific. There may be a 2-gig limit on Mac OS X 10.3
file sizes. I don't know.

Comment 1

[Ben Fowler]

(In reply to comment #0)
> This is probably platform specific. There may be a 2-gig limit on Mac OS X 10.3
> file sizes. I don't know.

The file system limit for file size is thousands of times larger. There was 
once a limit around that size, but that was fixed for Mac OS 8.5 . 

Otherwise, it seems sensible to avoid creating files over 1GB in size
automatically. A similar problem occurs with log files. Perhaps once
the standard 'Junk' folder is over 1GB, Thunderbird should switch over to
a new Junk folder per day ...

And/or automatically delete stale junk ...

Comment 2

[Jo¾e Klepec]

Thunderbird should have another option at Tools>Junk Mail Controls>Handling.
There is an option of deleting n days older mail. Please, add option to delete N
days younger mail. The current would delete all mail dated 1.1.1970, 1.1.1980,
or if today is 2.8.2005, all files week ago if I set it to 7. The addition would
for M days younger mail would delete every mail which is dated (in this case)
9.8.2005 (DD.MM.YYYY, respectively).

Comment 3

[a_geek]

Please see also bug #292857

It would be nice if this bug could be fixed in a way that
all Mozilla apps profit from it, and also be "advertised"
as fixed, so users know which version to use.

Thanks!

Comment 4

[Wayne Mery (:wsmwk)]

Elliot, et al,

Please comment as to whether you still hit 2gig limit with TB 1.5.0.4 and also please set in this bug the version of TB where it happened.

The limit should be 4gig according to bug 180154.
If you don't see it you can close this as a dup of bug 207400.


Joe in comment #2:
> Thunderbird should have another option at Tools>Junk Mail Controls>Handling.
> There is an option of deleting n days older mail. Please, add option to delete N
> days younger mail. The current would delete all mail dated 1.1.1970, 1.1.1980,
> or if today is 2.8.2005, all files week ago if I set it to 7. The addition would
> for M days younger mail would delete every mail which is dated (in this case)
> 9.8.2005 (DD.MM.YYYY, respectively).

If you want this capability the please file an enhancement bug 


a_geek comment #3:
> Please see also bug #292857
> 
> It would be nice if this bug could be fixed in a way that
> all Mozilla apps profit from it, and also be "advertised"...

a_geek, what "other apps"?

(see bug 180145 via bug 207400)

Comment 5

[Elliotte Rusty Harold]

I cannot easily reproduce it, nor do I have the time right now to deliberately inject 2 gigs of spam and see what breaks. It would be good for someone to do so, but I can neither confirm nor deny that the bug exists in current versions of Thunderbird. I would suggest adding a deliberate test for this to Thunderbird's acceptance test suite. 

I suspect the bug was originally reported with whichever version of Thunderbird was shipping on May 18, 2005. 

Comment 6

[Jo¾e Klepec]

Well,

Well, I have frequency of around 20 to 40 spamhits per day or less. It all about being wise when you browse & click @ URI.

I don't know why TB has not hanged since I started to use RPM packages from Fedora Core. It had crashed several times before - I used tar.gz packages then - but since then I also stopped using apt packaging system and started using yum. The second point is that I regularly use Smartsave extension to save junk outside the TB junk directory. 
Complexity of mail rules and mail directories has on the other hand increased.

I hope that points out a way to the possible solution.

Suggestion: 

Maybe we should setup like this: 

someuser AT someserver 

Then we set a server of totally blank homepage, except with href of this mail in it. The rest would be easy. Just getting mail down with TB, considering that all mail that it gets IS spam initially setting only one rule - marking all mail as read. I assume that this would really fast make 2Gb blowup, because there are a lot of stupid mail harvesting robots around. And it would be very convenient for all other tests, too.

The other way is that we all redirect ours spam to such mail address by a filter rule. 

--little bit OT ---
And Wayne, if I and somebody else currently don't have much time to post new bugs or solve old ones this does not mean that you're the wiseset guy on this list or even on whole planet - but you might have some good ideas which added together make faster & better software.

I personally am just overpossessed by starting my own bussines and please don't bother with my and theirs low frequency from time to time. We're all useful on more fronts then just Bugzilla - you might even not know them all. So we fight multifrontally as we can. With such mail you might in some cases get even less help then you originally wanted.

Cheers, Joze.

Comment 7

[Wayne Mery (:wsmwk)]

Magnus, David, two questions
1. have any major "delete local" bugs been fixed in last two years?
2. would "delete local" have as severe resource issues as bug 368685 for example?

(ignore my comment about file size, I don't think that was the issue.) comment 0 states local folders on mac. Elliot's doesn't state if thousands of messages were deleted i an "instant", resulting in "hang", or if it was over an extended period of time.  unknown if memory issue or cpu loop.  Also don't know what Klepec's problem is/was.


Klepec  sorry if I offended. However it's often not possible to make progress without participation from others, especially those who report problems. Also, I didn't demand anything from anyone.

Comment 8

[Wayne Mery (:wsmwk)]

I may have read too much into Elliot's description and been diverted by other comments - he doesn't mention delete.

Elliot, did hang happen by clicking junk folder or attempting to delete junk mail from junk folder

related?  bug 369255, bug 398684

Comment 9

[Jo¾e Klepec]

imho #369255 is no way related to what I have seen, the other one also not, but check please. These are possibly separate bugs in my opinion. But, these bugs may be connected. To illustrate, I'll need some space: 

When I was collecting statistical data from state webpages for my degree in public administration (done this with FF, took six months, pretty much with FF Deer Park/nightly releases of TB which share[d] some code, somebody needs to check my error posts overthere to see if something possibly matches).

I had installed 4G swap under FC4.x (one year and a half, so under FC5 too). This swap was used for the whole world only in sizes of few kilobytes or even zero. For some portals (Nigeria, China and some never heard-of-states in Pacific) I remember a very fast raise of this to 2G (surprise, surprise...). So, if anyone gets very high numbers (or very high constant or quite peaky swap usage on certain sites) this is probably this, connected with high numbers of spam, using similar domains. 
I really do have moderate spam numbers and in TB I really could live with it as I setup several rules, spamassasin, etc.. But... whenever I run yum update thunderbird, TB rules were all broken and let's setup them from zero again. Aii, this is it for #9999999 ;)
So I switched to Kmail to and forgot about TB. Really sorry folks, it hurts me too. The best of the rest, what can I say.

Comment 10

[Elliotte Rusty Harold]

It's been a while since I reported this, but my recollection is that *NO* action was required, either selecting or deleting. The hang would happen when TB itself attempted to move new junk into the overstuffed Junk folder. I did not even have to open the Junk folder in the UI.

Comment 11

[nathanael]

Same problems with Thunderbird 2.0.0.12 and windows xp.
My inbox was 1.9Go, and I wasn't able to received any mails. As soon as I tried to download my mails, the 2 Go size was over. So thunderbird crashed.
I removed all messages from the 2007 year and now it works. 

Comment 12

[Wayne Mery (:wsmwk)]

Nathanael, this is a Mac-only bug. Please get talkback id using http://kb.mozillazine.org/Talkback#Getting_an_incident_ID   With stack information from the crash, find a bug that better matches your problem. And if there is no such bug, then please file a new one.

Comment 14

[WADA:World Anti-bad-Duping Agency]

This bug cannot be DUP of Bug 403043.
  This bug   : File size limit is really 2GB on Mac OS X. See Bug 362244.
  Bug 403043 : File size limit is 4GB on MS Win. Problem when 2GB<file_size<=4GB.
               Bug 450359 & Bug 450991 on MS Win are friends of Bug 403043.
Please never add comment for phenomenon with 2GB<file_size<=4GB on MS Win.

Re-opening, and setting dependency to Bug 362244.

To bug opener and all problem reporters(on Mac X):
Does problem(hung on Mac OS X when reached 2GB limit) still occur on latest release of Thunderbird? (As of today, Thunderbird 2.0.0.19 is the newest).

Comment 15

[Kent James (:rkent)]

While this bug may not be a dup of 403043, I think its current status should be WFM since the patches for bug 228675 limits the size of the junk store to a size typically much, much small than the 2 gbyte limit.

If you want to keep it open, please define what needs to be done beyond what is already done for junk support.

Comment 16

[Wayne Mery (:wsmwk)]

this is, strictly speaking a dupe, and perhaps not the only one. but confirming and keeping this open because the NSPR bug is not easily found.

Comment 17

[Wayne Mery (:wsmwk)]

inclined to agree with comment 15 after 3.0 update goes full steam.

Comment 18

[Kent James (:rkent)]

Sorry, but re-reading this bug I misunderstood it. My comment 15 was directed at the issue of the junk training store, but this bug is directed toward the mail store for junk messages.

I think bienvenu is working on some sort of solution to the too-big mailbox problem though.

Comment 19

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #0)
> The recent neo-nazi spam invasion pushed my local junk mail box to 2 gigabytes.
> Once this happened, Thunderbird repeatedly hung with the spinning beach ball of death.  

File size is larger than 2GB(or just 2GB)?
If yes, it may be dup of bug 387502. (summary says 4GB, but 2GB on Linux/Mac)
Once file size exceeds mbox file size limit, such phenomenon happens and continues until file deletion or file size reduction by external utility program.