9357 - Reimplement CJK detector or replace it with universal detector.

Status: RESOLVED FIXED

(Core :: Internationalization, defect, P2)

Description

[Teruko Kobayashi]

Tested 6-29 Win32 build.

This is known issue.
Step of reproduce
1. Copy above file to the local directory where Apprunner is located.
2. If the page has Meta charset, delete it
3. type index.html | DETECTCH cjkpsm 1024

It does not detect any charset.
If you use kopsm instead of cjkpsm, it detect EUC-KR correctly.

I tested cjkpsm to detect several EUC-KR pages.  cjkpsm did not detect EUC-KR pages.
http://altavista.co.kr/
http://www.chosun.com/
http://www.ibm.co.kr/
http://www.ked.co.kr/index.html
http://lg.co.kr/h_lg/
http://bbs.sbs.co.kr/

Comment 1

[Frank Tang]

Mark this as M12, I won't fix zhcnpsm, cjkpsm bug untill M12

Comment 2

[Frank Tang]

change to M20

Comment 3

[Frank Tang]

reassign to shanjian. shanjian, I am not sure my 5/15 fix (add statistic 
analysis to nsPSMDetector) fix this or now. Please try it.

Comment 4

[Frank Tang]

mark it as future

Comment 5

[Shanjian Li]

CJK detector should be replaced by universal. Or we might need to reimplement 
CJK detector using my new model. I do not want to reinvent another wheel while
we have one. Keep this bug open and kill rest of CJK detector bug.

Comment 6

[Shanjian Li]

*** Bug 57872 has been marked as a duplicate of this bug. ***

Comment 7

[Thomas]

Call description states "copy above file", but no files seems attached.
Is it possible to attach such file so it can be tested if this bug is still
reproducable?

Comment 8

[Frank Tang]

shanjian is no longer working on mozilla for 2 years and these bugs are still
here. Mark them won't fix. If you want to reopen it, find a good owner first. 

Comment 9

[Travis Chase]

Mass Re-open of Frank Tangs Won't fix debacle. Spam is his responsibility not my own

Comment 10

[Travis Chase]

Mass Re-assinging Frank Tangs old bugs that he closed won't fix and had to be
re-open. Spam is his fault not my own

Comment 11

[Serge Gautherie (:sgautherie)]

Filter on "Nobody_NScomTLD_20080620"

Comment 12

[Simon Montagu :smontagu]

Bug 426271 replaced the CJK detector with the universal detector. I've checked in a unit test for EUC-KR.