Closed Bug 449741 Opened 16 years ago Closed 12 years ago

Thunderbird freeze if Inbox size exceed 2.1GB and message filter is enabled (with "quarantine option" disabled)

Categories

(MailNews Core :: Database, defect)

x86
All
defect
Not set
critical

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: roman.laube, Unassigned)

References

Details

(Keywords: dataloss, hang)

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.10) Gecko/20071126 Fedora/2.0.0.10-1.fc7 Firefox/2.0.0.10
Build Identifier: 2.0.0.16 (20080708)

If Inbox size exceeds 2.1GB and message filter is set for this Inbox and new message which hits this filter comes, Thunderbird stops responding (crash).

Reproducible: Always

Steps to Reproduce:
1.get Inbox > 2.1GB
2.set filter for this Inbox ('From' contains 'MyName'-> 'Move message' to MyFolder)
3.receive message which hits this filter rule
Actual Results:  
Thunderbird became unusable

Expected Results:  
It should receive message and do what filter says

I have realized that in alfa version 3.0a2pre (2008052603) it is working. Do you plan to fix it in version 2?
As written in Bug 362244, mail folder file greater than 2GB is not supported yet
(supported by MS Win version only). Bug 343907 is an example of bug report(on Mac OS X) due to still open Bug 362244.

When reached 2GB limit while adding a mail data, it can corrupt mail related data in mail DB.
Latest mail in thread pane is the mail caused freeze? If yes, can the mail be viewed normally? No lack of mail data?
(Sort by "Order Received" column. It's offset in local mail folder file)   
Does following occur in mail folder file? (tail command perhaps can be used) 
                                           2GB
   (last-1) ---><--- (last mail/partial) ---|--- (lost mail data) -->
   (Mail data is cut off at 2GB, but mail DB has whole lengh of the mail) 
Manual "Run Filter On" on the mail folder works? (download time only problem?)

> I have realized that in alfa version 3.0a2pre (2008052603) it is working.
> Do you plan to fix it in version 2?

I think error message like "2GB limit reached" is issued by Tb Trunk. Is it true?
Please note that following.
Since Bug 362244 is not fixed yet, mail folder file larger than 2GB is still not supported by Tb trunk. Even if no freeze with Tb trunk, last mail(and following mails) can't be downloaded, unless mail folder file size is reduced by you.
(In reply to comment #1)
> As written in Bug 362244, mail folder file greater than 2GB is not supported
> yet
> (supported by MS Win version only).

I am testing it on Windows

> When reached 2GB limit while adding a mail data, it can corrupt mail related
> data in mail DB.
> Latest mail in thread pane is the mail caused freeze? If yes, can the mail be
> viewed normally? No lack of mail data?

Inbox is not corrupted, but end of destination mail folder (where message should be moved) is filled with mess.
No mail is lost. After freeze you can restart (kill->start) thunderbird, switch off message filter and all messages are received with no problem....

> Manual "Run Filter On" on the mail folder works? (download time only problem?)

Manual "Run Filter On" on the mail folder works...

> I think error message like "2GB limit reached" is issued by Tb Trunk. Is it
> true?
> Please note that following.
> Since Bug 362244 is not fixed yet, mail folder file larger than 2GB is still
> not supported by Tb trunk. Even if no freeze with Tb trunk, last mail(and
> following mails) can't be downloaded, unless mail folder file size is reduced
> by you.

In our company we commonly use Inbox ~ 3GB with no problem (Windows platfom). I was only surprised I cannot use filter on such Inbox...

Thank you for your time

When local mail folder file greater than 2GB on MS Win, you can see "negative value in Order Received column". "Order Received" column is offset value(32bit integer, unsigned when MS Win, signed when other OS) in local mail folder file. Because mail folder file file of 2GB<filesize<=4GB on MS Win can be accessed normally and sort order of "Order Received" columns is correct, it's display only issue of "Order Received" column. However, existence of this kind of issue is an evidence of misuse of "unsigned 32 bit integer" as "signed 32 bit integer" at some other places in many cases.
In Japan, it is expressed as follows;
  If you see a cockroach, at least 20 cockroaches are living in your home :-)

> Manual "Run Filter On" on the mail folder works...

Do you enable quarantine option? (Tools/Options/Privacy/Anti Virus)
Do you use Global Inbox feature?
If Global Inbox, message filter is defined for each account only? Global Inbox owner account(Local Folders usually) only? Or for both?
What does filter log say about filtering upon the crash?

(In reply to comment #0)
> Thunderbird stops responding (crash).

What is Talkback ID (or Crash Reporter ID) of your crash?
Does the crash occur with -safe-mode or with new profile/newly defined account?
Can you re-produced crash with latetst-trunk of TB, with new profile/newly defined acount?
Other questions.

Is move target folder of message filter is also "greater than 2GB" folder?
Does your problem is reproduced with "Inbox of around 3GB" and "some new mails which will be filtered"?
If yes, does the problem occur with null target mail folder folder file?

> Inbox is not corrupted, but end of destination mail folder
> (where message should be moved) is filled with mess.

What data was the "mess"? Filled from top of target mail folder file? Or appended to target mail folder file? Can you guess "what happened" by data of the "mess"?

If you encounter problem again, or if you can re-produce problem, keep back up of at least "Inbox, Inbox.msf, move_target_folder, move_target_folder.msf, message filter log". They possibly helps you to guess "what happened" or "what is wrong".
One more question.
You say "freeze" in your bug summary. But you say "crash" in your comment #0.
> Thunderbird stops responding (crash)
Which? What phenomenon do you mean by your "freeze" or "crash"? 
(In reply to comment #3)
> Do you enable quarantine option? (Tools/Options/Privacy/Anti Virus)

I had "quarantine option" disabled. But after enabling it my filter have started working ! It's some kind of magic :-) This solution would not cross my mind. I think answering rest of your questions is useless. Thank you for your time... and sorry for annoyance
(In reply to comment #6)
> I had "quarantine option" disabled. But after enabling it my filter have started working !

Thinks for valuable information.
I guess that message filter reads mail data from temporary file for single mail  during download when "quarantine option"=On, thus problem won't occur. It indicates that problem occurs only when message filter reads mail data from Inbox greater than 2GB during download(You say no problem when manual "Run Filters On"). 
I can try to do duplication test by your answer. I have test folder file greater than 4GB in addition to file greater than 2GB ;-)

> I think answering rest of your questions is useless.

You are right for many of my questions. But please answer to my question of "What kind of phenomenon really occurred in addition to 'mess' in move target folder?"
- Crash? Hang(unresponsive)?
- If Crash, what is Talkback ID or Crash Reporter ID?
- If Hang(unresponsive), Loop(CPU 100% or 50% if dual-CPU)? Or Wait(CPU 0%)?  
- Was other funny phenomena(such as retry loop on move target folder) observed? 
Question to avoid misleading: POP3? IMAP?

(In reply to comment #7)

> You are right for many of my questions. But please answer to my question of
> "What kind of phenomenon really occurred in addition to 'mess' in move target
> folder?"
> - Crash? Hang(unresponsive)?
> - If Crash, what is Talkback ID or Crash Reporter ID?
> - If Hang(unresponsive), Loop(CPU 100% or 50% if dual-CPU)? Or Wait(CPU 0%)?  

I am sorry for mess in terminology. What I mean is: thunderbird "nearly" freeze – you cannot download any message, open any message, write new message.  You can use menu (for example open "Options" window), switch between mail folders (you see mails but cannot open any).
You can close thunderbird but process “thunderbird” is still runnig in background (you have to kill it via “Task Manager”). CPU usage is 0%

> - Was other funny phenomena(such as retry loop on move target folder) observed?

I don't know what you mean....

> Question to avoid misleading: POP3? IMAP?

We are downloading messages via POP3


I am adding example of 'mess' which is apended in move target folder:"##��Y##�\#w �pw �p����#   @Q#�#R# L#  ����"
Sounds for me same problem as Bug 321371. See Bug 321371 Comment #69.
Setting dependency to 321371. If wrong, please remove.
Depends on: 321371
Thanks WADA san. I did not know this bug.

This bug is exactly same as I commented in Bug 450359 comment #10.

I think patches in Bug 450359, Bug 450698 and maybe Bug Bug 428314 fixes this issue.
Confirming based on Comment #10 & Bug 450359 comment #10 by Ikezoe san.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Severity: normal → critical
Depends on: 450698
Summary: Thunderbird freeze if Inbox size exceed 2.1GB and message filter is enabled → Thunderbird freeze if Inbox size exceed 2.1GB and message filter is enabled (with "quarantine option" disabled)
No longer depends on: 321371
Removing dependency to 321371, because main issue of that bug was different.

FYI.
I've opened Bug 450991 for phenomenon of "negative value in Order Received column", which I referred to in my Comment #3.
Component: General → Backend
OS: Windows XP → All
Product: Thunderbird → MailNews Core
QA Contact: general → backend
Component: Backend → Database
QA Contact: backend → database
Laube's problem gone?
should be if nothing is needed beyond the blocking bugs
Keywords: hang
... as of 2.0.0.18
Laube reports "in new versions everything work fine", so WFM
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.