278483 - Proper place of Ja tokenizer code for junk mail control

Status: RESOLVED EXPIRED

(MailNews Core :: Internationalization, defect)

Description

[Noriyuki INOUE]

Where should we place Japanese tokenizer code (Bug 277354) for junk mail control?

Comment 1

[Scott MacGregor]

Noriyuki, I wonder if the junk code couldn't use a class like:

http://lxr.mozilla.org/seamonkey/source/intl/lwbrk/src/nsSampleWordBreaker.h#48

which seems to have similar code for detecting what class a character is....

Comment 2

[Noriyuki INOUE]

(In reply to comment #1)
> Noriyuki, I wonder if the junk code couldn't use a class like:
> 
> http://lxr.mozilla.org/seamonkey/source/intl/lwbrk/src/nsSampleWordBreaker.h#48
> 
> which seems to have similar code for detecting what class a character is....

I think that spriting string into semantic units is different from tokenizing
for the junk filter. For example, it is found that breaking string into bigram
or trigram without constructional analysis is more efficient way in some cases
of japanese message. So i had wrote it imdependently to test several way.
As a result, they are very similar. I will wrote a patch integrating them. There
are many conditional jumps. I may reduce them.

Comment 3

[Gervase Markham [:gerv]]

This is an automated message, with ID "auto-resolve01".

This bug has had no comments for a long time. Statistically, we have found that
bug reports that have not been confirmed by a second user after three months are
highly unlikely to be the source of a fix to the code.

While your input is very important to us, our resources are limited and so we
are asking for your help in focussing our efforts. If you can still reproduce
this problem in the latest version of the product (see below for how to obtain a
copy) or, for feature requests, if it's not present in the latest version and
you still believe we should implement it, please visit the URL of this bug
(given at the top of this mail) and add a comment to that effect, giving more
reproduction information if you have it.

If it is not a problem any longer, you need take no action. If this bug is not
changed in any way in the next two weeks, it will be automatically resolved.
Thank you for your help in this matter.

The latest beta releases can be obtained from:
Firefox:     http://www.mozilla.org/projects/firefox/
Thunderbird: http://www.mozilla.org/products/thunderbird/releases/1.5beta1.html
Seamonkey:   http://www.mozilla.org/projects/seamonkey/

Comment 4

[Gervase Markham [:gerv]]

This bug has been automatically resolved after a period of inactivity (see above
comment). If anyone thinks this is incorrect, they should feel free to reopen it.