11659 - Need i18n support for filter/search backend

Status: VERIFIED FIXED

(MailNews Core :: Internationalization, defect, P3)

Description

[nhottanscp]

The current back end is not i18n ready (e.g. old converter should be replaced
by the new one etc..).
Please put appropriate target mile stone, thanks.

Comment 1

[nhottanscp]

I summarized i18n requirement in a document below.
http://www.mozilla.org/projects/intl/mail-news-i18n-spec.html#Local mail search 
i18n

Header search 

MIME decode and charset conversion for headers - 
search term is now unicode (used to be a folder charset), 
we should compare it against MIME decoded and unicode converted header strings. 

Body search 

4.x implementation:  
    converts body text to win_csid. - no need to support this in mozilla 
proposal for mozilla: 
    convert body text to unicode. 
    QP decode for body (plain and HTML) 
    Entity (CER) and NCR decode for HTML body 

Comment 2

[Scott MacGregor]

triaging since M15 is tonight. Not an M15 stoppper.

Comment 3

[nhottanscp]

Putting beta2 for i18n beta2 criteria items. Contact bobj for question.

Comment 4

[leger]

Putting on [nsbeta2+] radar.  Need to fix by 5/16...+ for i18n search at least 
as good as 4.x.

Comment 5

[selmer (gone)]

Mass moving M16 to M17 - look for nsbeta2 before anything else.

Comment 6

[leger]

Removing [nsbeta2+][5/16]... is this mandatory for I18n RTM?

Comment 7

[nhottanscp]

yes, the user should be able to search messages, this is supported in 4.x.

Comment 8

[bobj]

Yes, searching and filtering of non-ASCII is mandatory.
This would be a major regression from previous releases.

Comment 9

[leger]

On exception list for PR2, removing 5/16...giving [nsbeta2+]Exception Feature 
status.

Comment 10

[Alec Flett]

I just landed a TON of internationalization work to search and filters
yesterday, so I'm going to take this bug and mark it fixed... I think further
i18n issues should be taken up on exception by exception basis.

Comment 11

[Alec Flett]

adding mscott back to the CC and marking fixed.

Comment 12

[Katsuhiko Momoi]

Alec, can I proceed on the assumption that specs mentioned in
[e-mail] 2000-03-24 10:09 commet have been 
implemented?

Also what is working at FE?

1. Search/Find in msgs
2. IMAP search
3. Local Folder search
4. Filtering incoming msgs

Comment 13

[Alec Flett]

yes, I think everything nhotta describes is implemented, or at least I've seen 
the code to do everything that is described.

The FE should at least be able to spit things to the screen in the next day or 
so.

Comment 14

[Alec Flett]

oh, and to comment on your list:
1) done using the same technique as the browser. But this has nothing to do with 
search/filters, so let's not include that in this bug.`
2) I don't think this is completely implemented
3) bienvenu says this is now working
4) filters should now be i18n friendly. There may be an issue with migrated 
filters from 4.x though, because I'm assuming filter values are encoded in utf8, 
and I think the 4.x filters were in the system charset. I have a seperate bug on 
that though.

Comment 15

[Alec Flett]

*** Bug 5933 has been marked as a duplicate of this bug. ***

Comment 16

[Katsuhiko Momoi]

** Checked with 8/2/2000 Win32 M17 build **

OK. I'm going to mark this verified as fixed
though I believe that there might be several things not 
working with search or filter i18n-wise. 
We have filed some bugs and we should file others as needed.

Basic IMAP search is working -- header and body.
Basic Local Folder search seems to be working for headers only.
Basif filter -- this depends on the day. Today it does even work
   for Enlgish. I trust that someone has filed a bug for this.