Closed
Bug 79275
Opened 25 years ago
Closed 25 years ago
add new tool to generate gbk/gb18030 tables
Categories
(Core :: Internationalization, defect)
Tracking
()
VERIFIED
FIXED
mozilla0.9.1
People
(Reporter: ftang, Assigned: ftang)
References
Details
(Keywords: intl, Whiteboard: code reviewed by btell, need super review 2001-05-07 18:09)
this is a spin off from 75928
We need to support GB18030
GB18030 differ from GBK in the following way
1. GB18030 is 1,2,4 byte encoding, GBK is only 1,2 byte encoding
2. the two byte part- there are some small difference between GBK and GB18030
We need to add a new tool to generate the following table
1. common mapping in GBK and GB18030
2. GBK 2 byte unique mapping
3. GB18030 2 byte unique mapping
4. GB18030 4 byte mapping-
| Assignee | ||
Comment 1•25 years ago
|
||
here is the new tool- intl\uconv\tools\gengb18030tables.pl
http://bugzilla.mozilla.org/showattachment.cgi?attach_id=33410
It read in two file
CP936.TXT and GB18030 (from ftp://ftp.unicode.org)
It generate 4 intermediate files
gbkcommon.txt contains the common mapping between gb18030 and gbk
gb18030uniq.txt contains the gb18030 2 byte unique mapping
cp936uniq.txt contains the gbk 2 byte unique mapping
gb180304b.txt contains the gb18030 4 byte mapping in the linear space intead of
gb18030 code point.
It then call umaptable and cp936tocdx.pl to generate
gb18030uniq2b.u{f,t} -compresed mapping
gbkuniq2b.u{f,t} -compresed mapping
gb180304byte.{f,t} -compresed mapping
cp936map.h
Also, we need to change the cp936tocdx.pl so it
1. do not do User define character range anymore since we already do that in the
intl\uconv\tools\gengb18030tables.pl
2. change the input file name to gbkcommon.txt
Whiteboard: code reviewed by btell, need super review
| Assignee | ||
Updated•25 years ago
|
Whiteboard: code reviewed by btell, need super review → code reviewed by btell, need super review 2001-05-07 18:09
| Assignee | ||
Comment 2•25 years ago
|
||
The cp936tocdx.pl change is in http://bugzilla.mozilla.org/
showattachment.cgi?attach_id=33409
Comment 3•25 years ago
|
||
Changing QA contact to ylong@netscape.com for now.
QA Contact: andreasb → ylong
Comment 5•25 years ago
|
||
not critical but you might want to change these to hashes
(eg: use '%' instead of '@')
@gb18030tounicode = {};
@unicodetogb18030 = {};
@unicodetocp936 = {};
@cp936tounicode = {};
@tounicodecommon = {};
@gb18030tounicodeuniq = {};
@gb180304btounicode = {};
@cp936tounicodeuniq = {};
@map = {};
r=bstell@netscape.com for
http://bugzilla.mozilla.org/showattachment.cgi?attach_id=33410
r=bstell@netscape.com for
http://bugzilla.mozilla.org/showattachment.cgi?attach_id=33409
| Assignee | ||
Updated•25 years ago
|
Status: NEW → ASSIGNED
Comment 6•25 years ago
|
||
#!/user/local/bin/perl
That should at least be /usr.
In my perl scripts I always use 'use strict;' at the top of the file so that I
don't automagically bring variables into existence. It's saved a lot of
headache for me and I would suggest but don't require that you do the same. It
might show a lot of problems with your code that you wouldn't normally see.
@gb18030tounicode = {};
Initializers for arrays should be of type () - for example:
@db18030tounicode = ();
When you initialize it with an {} you actually stuff an empty hash into the
first element of the array:
[blizzard@decade blizzard]$ perl
use Data::Dumper;
@foo = {};
print Dumper(\@foo);
$VAR1 = [
{}
];
[blizzard@decade blizzard]$ perl
use Data::Dumper;
@foo = ();
print Dumper(\@foo);
$VAR1 = [];
I would appreciate more comments about these transformations if possible.
$h is less descriptive than $high_bit or $l for $low_bit. You should use
self-describing variables when possible.
| Assignee | ||
Comment 7•25 years ago
|
||
fix and check in.
Status: ASSIGNED → RESOLVED
Closed: 25 years ago
Resolution: --- → FIXED
| Assignee | ||
Updated•25 years ago
|
Target Milestone: --- → mozilla0.9.1
You need to log in
before you can comment on or make changes to this bug.
Description
•