302567 - crash (mail) because training.dat for junk is large (larger than 6.5mb)

Status: RESOLVED DUPLICATE of bug 228675

(SeaMonkey :: MailNews: Account Configuration, defect)

Description

[Dirk]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050513 Debian/1.7.8-1
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050513 Debian/1.7.8-1

my training.dat file is larger than 6.5mb and in effect mozilla "disappears"
after download of new emails.


Reproducible: Always

Steps to Reproduce:
1. Download emails
Actual Results:  
Mozilla crahes

Expected Results:  
checking for Junk

deleting training.dat solved the problem

Comment 1

[Andrew Schultz]

Do you still have the training.dat that caused it to crash or a stacktrace from
the crash?

Comment 2

[John T]

My Moz 1.7.12 training.dat was approaching that size and still worked with no problems under 1.7.12, but didn't work right with 1.8 and Seamonkey builds, causing poor junk filtering. I've switched to testing only on Seamonkey using a new training.dat that's approaching .5Mb in less than a week. I suspected other problems than a size limit, but that is worth looking into. Ongoing marking of mail as junk adds to the file size, and that can't go on forever.

Info provided on IRC about the file format indicates that the first part of the file is data for good mail, and the latter part of the file is data about junk. Dates are not included in the records.

A suggestion I made as an enhancement request is to limit the filesize somehow. IF the goodmail and badmail records are entered sequentially instead of sorted, for example adding new entries to the front of the file sections so the tail end is the really old entries, then the limit could be imposed by deleting tail-end records from the junkmail sections of the file. I wouldn't suggest removing records from the goodmail section.

Would that be a workable method of limiting the file size to, for example, 5Mb?

Comment 3

[Wayne Mery (:wsmwk)]

David, 
wasn't there a patch in the last 2 years that effectively trims (ages entries out of) the training file? I didn't find anything in lxr or in bugs and bug 215701 doesn't seem applicable here.

bug 309620 is related

Comment 4

[Karsten Düsterloh]

How should that work? We don't store any time/date info in training.dat, see <http://mxr.mozilla.org/seamonkey/source/mailnews/extensions/bayesian-spam-filter/src/nsBayesianFilter.cpp#1365>.

Comment 5

[Tony Mechelynck [:tonymec]]

1.7 branch isn't supported anymore. Relabeling Trunk/All/All and duping to bug 228675.