Closed Bug 79275 Opened 25 years ago Closed 25 years ago

add new tool to generate gbk/gb18030 tables

Categories

(Core :: Internationalization, defect)

x86
Windows NT
defect
Not set
normal

Tracking

()

VERIFIED FIXED
mozilla0.9.1

People

(Reporter: ftang, Assigned: ftang)

References

Details

(Keywords: intl, Whiteboard: code reviewed by btell, need super review 2001-05-07 18:09)

this is a spin off from 75928 We need to support GB18030 GB18030 differ from GBK in the following way 1. GB18030 is 1,2,4 byte encoding, GBK is only 1,2 byte encoding 2. the two byte part- there are some small difference between GBK and GB18030 We need to add a new tool to generate the following table 1. common mapping in GBK and GB18030 2. GBK 2 byte unique mapping 3. GB18030 2 byte unique mapping 4. GB18030 4 byte mapping-
here is the new tool- intl\uconv\tools\gengb18030tables.pl http://bugzilla.mozilla.org/showattachment.cgi?attach_id=33410 It read in two file CP936.TXT and GB18030 (from ftp://ftp.unicode.org) It generate 4 intermediate files gbkcommon.txt contains the common mapping between gb18030 and gbk gb18030uniq.txt contains the gb18030 2 byte unique mapping cp936uniq.txt contains the gbk 2 byte unique mapping gb180304b.txt contains the gb18030 4 byte mapping in the linear space intead of gb18030 code point. It then call umaptable and cp936tocdx.pl to generate gb18030uniq2b.u{f,t} -compresed mapping gbkuniq2b.u{f,t} -compresed mapping gb180304byte.{f,t} -compresed mapping cp936map.h Also, we need to change the cp936tocdx.pl so it 1. do not do User define character range anymore since we already do that in the intl\uconv\tools\gengb18030tables.pl 2. change the input file name to gbkcommon.txt
Whiteboard: code reviewed by btell, need super review
Whiteboard: code reviewed by btell, need super review → code reviewed by btell, need super review 2001-05-07 18:09
The cp936tocdx.pl change is in http://bugzilla.mozilla.org/ showattachment.cgi?attach_id=33409
Blocks: 75928
Changing QA contact to ylong@netscape.com for now.
QA Contact: andreasb → ylong
Reassign to ftang.
Assignee: nhotta → ftang
Keywords: intl
not critical but you might want to change these to hashes (eg: use '%' instead of '@') @gb18030tounicode = {}; @unicodetogb18030 = {}; @unicodetocp936 = {}; @cp936tounicode = {}; @tounicodecommon = {}; @gb18030tounicodeuniq = {}; @gb180304btounicode = {}; @cp936tounicodeuniq = {}; @map = {}; r=bstell@netscape.com for http://bugzilla.mozilla.org/showattachment.cgi?attach_id=33410 r=bstell@netscape.com for http://bugzilla.mozilla.org/showattachment.cgi?attach_id=33409
Status: NEW → ASSIGNED
#!/user/local/bin/perl That should at least be /usr. In my perl scripts I always use 'use strict;' at the top of the file so that I don't automagically bring variables into existence. It's saved a lot of headache for me and I would suggest but don't require that you do the same. It might show a lot of problems with your code that you wouldn't normally see. @gb18030tounicode = {}; Initializers for arrays should be of type () - for example: @db18030tounicode = (); When you initialize it with an {} you actually stuff an empty hash into the first element of the array: [blizzard@decade blizzard]$ perl use Data::Dumper; @foo = {}; print Dumper(\@foo); $VAR1 = [ {} ]; [blizzard@decade blizzard]$ perl use Data::Dumper; @foo = (); print Dumper(\@foo); $VAR1 = []; I would appreciate more comments about these transformations if possible. $h is less descriptive than $high_bit or $l for $low_bit. You should use self-describing variables when possible.
fix and check in.
Status: ASSIGNED → RESOLVED
Closed: 25 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla0.9.1
Changed QA contact to teruko@netscape.com.
QA Contact: ylong → teruko
Verified.
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.