Closed Bug 176528 Opened 18 years ago Closed 18 years ago

need a nsISemanticUnitScanner to support intl spam mail filter

Categories

(Core :: Internationalization, defect)

defect
Not set

Tracking

()

RESOLVED FIXED
mozilla1.3alpha

People

(Reporter: ftang, Assigned: shanjian)

Details

(Keywords: intl)

Attachments

(3 files, 6 obsolete files)

beard send me a proposal. but I change it due to the difficulty to remove malloc

we need this so beard can change the spam mail filter work for non ascii
assigned
Status: NEW → ASSIGNED
Please use the MPL, not the NPL.  And the makefile needs to use the tri-license,
not the old license.
code issue? QA to yokoyama@netscape.com for now.
Keywords: intl
QA Contact: ruixu → yokoyama
wtill work in progress. 
to do list
1. add to intl module file
2. write self test code
3. debug

done list:
1. idl is ok
2. basic structure is ok
3. need to add zh detection and behavior differently (maybe?) between zh and
ja/ko
Attachment #103994 - Attachment is obsolete: true
Attachment #104186 - Attachment is obsolete: true
Comment on attachment 104353 [details] [diff] [review]
patch v3, include contract id and intl/build changes

wrong submission remove it
Attachment #104353 - Attachment is obsolete: true
Attachment #104354 - Attachment is obsolete: true
From the test result, it looks like we need some more work
Attachment #104358 - Attachment is obsolete: true
shanjian/nhotta can you r= this bug ?
beard can 1. try the patch with the test file, or if you can attach some sample
text you want to try then I can run that with the test file and give you the
result. 

Here are some test result:
Test Data:

(germany) Die richtige Karte für jeden Anlass Ob
Feiertage, Liebe und Freundschaft, Grüße und Glück-wünsche, Natur und
Tier, Sport und Spaß oder ein Bild zum Schmunzeln - bei unseren
elektronischen Karten finden Sie auf alle Fälle das richtige Motiv für
Ihren persönlichen Gruß. (spanish) Lanza amenaza Bin Laden a Occidente
La televisora Al Yazira mostró una grabación con la voz de Bin Laden
alabando los ataques terroristas en Bali y Moscú; amenazando al mundo
occidental con represalias si ataca a Irak. (french)  Mardi 12 novembre
2002 AUJOURD'HUI SUR LE WEB Le site du jour Epopée américaine Girardin,
jeune homme originaire du Maine-et-Loire, s'embarque pour les Amériques
à 20 ans après s'être fait renvoyer du collège. Ce site en Flash nous
conte l'épopée de ce petit Français à travers les Etats-Unis, entre
1848 et sa mort, à 59 ans. 

Test Result:

[germany] [Die] [richtige] [Karte] [für] [jeden] [Anlass] [Ob] [Feiertage]
[Liebe] [und] [Freundschaft] [Grüße] [und] [Glück] [wünsche] [Natur] [und]
[Tier] [Sport] [und] [Spaß] [oder] [ein] [Bild] [zum] [Schmunzeln] [bei]
[unseren] [elektronischen] [Karten] [finden] [Sie] [auf] [alle] [Fälle] [das]
[richtige] [Motiv] [für] [Ihren] [persönlichen] [Gruß] [spanish] [Lanza]
[amenaza] [Bin] [Laden] [a] [Occidente] [La] [televisora] [Al] [Yazira] [mostró]
[una] [grabación] [con] [la] [voz] [de] [Bin] [Laden] [alabando] [los] [ataques]
[terroristas] [en] [Bali] [y] [Moscú] [amenazando] [al] [mundo] [occidental]
[con] [represalias] [si] [ataca] [a] [Irak] [french] [Mardi] [12] [novembre]
[2002] [AUJOURD] [HUI] [SUR] [LE] [WEB] [Le] [site] [du] [jour] [Epopée]
[américaine] [Girardin] [jeune] [homme] [originaire] [du] [Maine] [et] [Loire]
[s] [embarque] [pour] [les] [Amériques] [à] [20] [ans] [après] [s] [être] [fait]
[renvoyer] [du] [collège] [Ce] [site] [en] [Flash] [nous] [conte] [l] [épopée]
[de] [ce] [petit] [Français] [à] [travers] [les] [Etats] [Unis] [entre] [1848]
[et] [sa] [mort] [à] [59] [ans] 


(view this with ISO-8859-1)
Here are some CJK test (please view this part wiht UTF-8)

Test Data
 (japanese) Windows 版、Mac 版、Linux
版、いずれもダウンロード可能。 Netscape 7.0
でオンラインでの可能性のすべてを体験してください。最も人気の高いブラウザを今すぐ
ダウンロード! (korean) 만약 여러분이 언론단체의
일원이거나 Netscape와 관련된 추가 정보가 필요하시다면, 저희 공보부로 전화주시거
나 (650-937-3329, 오전 8 시
부터 오후 5시까지) pradmin@netscape.com으로 전자 우편을 보내주시기 바랍니다. 여
러분의 요청사항을 빠르게
진행해드리기 위해서는 본인과 소속 출판사 이름과 기사 제목, 보도일 등에 관련된 정
보가 필요합니다. 만약 여러분께서 책을
출판하신다면, 저희 출판 관계 섹션에서 제공되는 자료를 참고하시기 바랍니다.
(chinese) Netscape 7.0
正式问世! Netscape 7.0 是为满足您在互联网上的各种需求所设计的 -
浏览网页,搜索信息,通信,购物,下载及欣赏音乐等应有尽有。它包含一个威力无比的电
子邮件工具,以及全球首屈一指的即时信息服务,即“AOL
快信信使”(AIM)。它还能为您提供更大的空间,以充分发挥您在网上的各种经验与技能。 

Test Result

[japanese] [Windows] [版] [、Mac] [版] [、Linux] [版] [、] [いずれも] [ダウン
ロード] [可] [能] [。] [Netscape] [7] [0] [で] [オンライン] [での] [可] [能]
[性] [のすべてを] [体] [験] [してください] [。] [最] [も] [人] [気] [の] [高]
[い] [ブラウザ] [を] [今] [すぐ] [ダウンロード] [!] [korean] [만약] [여러분이]
[언론단체의] [일원이거나] [Netscape와] [관련된] [추가] [정보가] [필요하시다면]
[저희] [공보부로] [전화주시거나] [650] [937] [3329] [오전] [8] [시] [부터] [오
후] [5시까지] [pradmin] [netscape] [com으로] [전자] [우편을] [보내주시기] [바랍
니다] [여러분의] [요청사항을] [빠르게] [진행해드리기] [위해서는] [본인과] [소속]
[출판사] [이름과] [기사] [제목] [보도일] [등에] [관련된] [정보가] [필요합니다]
[만약] [여러분께서] [책을] [출판하신다면] [저희] [출판] [관계] [섹션에서] [제공
되는] [자료를] [참고하시기] [바랍니다] [chinese] [Netscape] [7] [0] [正] [式]
[问] [世] [Netscape] [7] [0] [是] [为] [满] [足] [您] [在] [互] [联] [网] [上]
[的] [各] [种] [需] [求] [所] [设] [计] [的] [浏] [览] [网] [页] [,] [搜] [索]
[信] [息] [,] [通] [信] [,] [购] [物] [,] [下] [载] [及] [欣] [赏] [音] [乐]
[等] [应] [有] [尽] [有] [。] [它] [包] [含] [一] [个] [威] [力] [无] [比] [的]
[电] [子] [邮] [件] [工] [具] [,] [以] [及] [全] [球] [首] [屈] [一] [指] [的]
[即] [时] [信] [息] [服] [务] [,] [即] [“AOL] [快] [信] [信] [使] [”(AIM)。]
[它] [还] [能] [为] [您] [提] [供] [更] [大] [的] [空] [间] [,] [以] [充] [分]
[发] [挥] [您] [在] [网] [上] [的] [各] [种] [经] [验] [与] [技] [能] [。] 
Attachment #104357 - Flags: review+
read http://www.mozilla.org/scriptable/faq.html#i3 about why I use new Object in
the test html /js file
reassign to shanjian 
beard- could you please take a look at the current result and let shanjian know
is this acceptable or there are something we need to change. 
shanjian, thank you to take care this for me. 
Assignee: ftang → shanjian
Status: ASSIGNED → NEW
Scott/Seth, could one of you sr?
Status: NEW → ASSIGNED
Sorry for the delay in my getting back to you, in the results in comment #14,
there is the following token,

[”(AIM)。]

But it looks like it contains spaces to me. Is this a correct result?
Other than that, the code looks fine to me. We will need a new Mac project file
to build the .idl file.
patrick,
There is no space there. That's full-width character. Try to select each
character and you will see what i mean. btw, can you sr it? We can do a bunch of
improvement after this main structure is there. 
In order to run http://bugzilla.mozilla.org/attachment.cgi?id=106056&action=view
you need to download it to your local dist to visit it, and the file:// will
make the privlegemanager ask you to grant the permission.

while serving from the http:// I think you need to sign it (signed javascript)
in order to do that. 
Naoki/ftang, can one of you help add the mac build file?
okay, I will do it
Attachment #106439 - Flags: review+
Comment on attachment 104357 [details] [diff] [review]
patch v4, change the idl according to beard's comment

sr=beard
Attachment #104357 - Flags: superreview+
Comment on attachment 106439 [details] [diff] [review]
Mac build changes (added a new src file, created xml for idl).

sr=beard
Attachment #106439 - Flags: superreview+
fix checked in. 
Status: ASSIGNED → RESOLVED
Closed: 18 years ago
Resolution: --- → FIXED
Reopening, since this was backed out.
Status: RESOLVED → REOPENED
OS: Windows NT → All
Hardware: PC → All
Resolution: FIXED → ---
Target Milestone: --- → mozilla1.3alpha
fix checked in.
Status: REOPENED → RESOLVED
Closed: 18 years ago18 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.