82282 - widget/gtk/keysym2ucs.c converts keysym to fullsize katakana

Status: NEW

(Core :: Internationalization, defect)

Description

[Masaki Katakai]

This bug is from bug 74088.

It seems that keysym2ucs.c converts japanese katakana keysym to
full-size characters. I believe  we usually use half-size katakana.

This is an RFE.

Comment 1

[nhottanscp]

Reassign to ftang.

Comment 2

[Frank Tang]

Please attach your suggest patch here. and We should ask the origional author 
about this one

Comment 3

[Frank Tang]

the patch is the widget/gtk/keysym2ucs.c part in
http://bugzilla.mozilla.org/showattachment.cgi?attach_id=35731

Comment 4

[Frank Tang]

katakai, please explain why you want to change to full size katakana?

Comment 5

[Frank Tang]

ask origional author
[e-mail] [e-mail]

Comment 6

[Frank Tang]

move to future unless we have a strong reason. 
katakai, if you really want it for moz0.9.2, then please remove the "Future". 
Thanks.

Comment 7

[Markus Kuhn]

I'm the maintainer of the keysym2ucs.c file in XFree86 xterm, the latest version
of which is available on

  http://www.cl.cam.ac.uk/~mgk25/ucs/keysym2ucs.c

I have also writen the proposed update of the X11 standard for X.Org, to add an
official mapping from keysyms to UCS:

  http://www.cl.cam.ac.uk/~mgk25/ucs/X11.keysyms.diff

I have brought up the issue discussed here in the [e-mail] mailing list,
where the relevant XFree86 and X.Org i18n folks are around. Please follow that
thread:

  http://www.xfree86.org/pipermail/i18n/2001-June/001946.html

The choice I made to make 0x04xx kana keysyms to U+30xx was a somewhat arbitrary
decision, mostly based on the idea that the half-width Katakana are just
compatibility characters. On the other hand, the 0x04xx keysyms were obviously
taken from JIS X 0201, and U+FF60..U+FF9F is a one-to-one mapping of JIS X 0201,
therefore mapping to halfwidth forms would also make sense to me and I wouldn't
object, if just someone just could express and justify a clear opinion on the
subject.

I see arguments in favour of *both* mappings and welcome any justified opinion
that you might have.

In any case, it is important to understand that keysym2ucs.c is just a temporary
solution until Xlib offers to do this conversion for you. More recent X.Org and
XFree86 releases of the X window system do support UTF-8 locales, therefore you
will get Unicode values instead of keysyms from XmbLookupString now, which makes
keysym2ucs.c obsolete on newer Xlib versions. XFree86 even added a
locale-independent Xutf8LookupString to simplify the API significantly for
applications like Mozilla that use Unicode internally and where locale
dependency becomes a hassle, sadly though there is still serious resistance from
Sun (!) employees against adding this highly useful API extension to the Xlib
standard. :-(

By using the Xlib functions instead of the temporary hack keysym2ucs.c, proper
locale-dependent UCS input methods can be added in the standard way, which I
think everyone agrees is the real solution here. keysym2ucs.c was just a
temporary hack for xterm, primarily for European users who do not need a complex
input mechanism.

Suggested action: (a) Don't patch this within Mozilla, wait until we decide
whether to change this and then update keysym2ucs.c entire from the above URL.
(b) Investigate whether you can use Xutf8LookupString instead and make sure that
it becomes Sun's policy to support Xutf8LookupString in X.Org to make it
standard.

http://www.cl.cam.ac.uk/~mgk25/unicode.html#x11

Markus

-- 
Markus G. Kuhn, Computer Laboratory, University of Cambridge, UK
Email: mkuhn at acm.org,  WWW: <http://www.cl.cam.ac.uk/~mgk25/>

Comment 8

[Masaki Katakai]

Markus, and Frank,

Thank you very much for taking action for community. I agree
that Mozilla should follow keysym2ucs.c of XFree86 now. I don't
have any strong requirements for this bugfix.

So keep target milestone is Future.

Comment 9

[Adam Hauner]

-> to default owner (rather than ftang's WONTFIX)

Comment 10

[Markus Kuhn]

Just for the record, and as an update on this issue, the next version of the
X.Org X11 protocol standard, Appendix A (KEYSYM Encoding)

  http://www.cl.cam.ac.uk/~mgk25/ucs/X11.keysyms.pdf

will contain an official mapping between keysyms and whatever the most
equivalent Unicode character is. This has already been in the X.Org CVS tree as
of September 2004. This mapping table is based on my old keysym2ucs.c table. The
draft X11 standard also uses for the 0x04xx kana keysyms the normal (and
according to historic JIS X 208 mapping tables "full width") Unicode katakana
values. If you have any good arguments and strong objections against this,
please let me know in detail, otherwise this will certainly not change.

Note that these keysyms are normally meant to be mapped properly to Unicode by a
full input method, therefore this table is more an informative equivalence
mapping rather than anything that should be of concern to the end user. It is
obviously not meant to be anything anywhere near a replacement for a proper
Japanese input method.

Also note that Unicode does not make a distinction between "half width" and
"full width" characters, just like it does not distinguish between normal,
italic and bold characters for the Latin script. Unicode has only one proper
representation of the Hiragana and Katakana, and these are the ones in U+30xx.
However, since the JIS standards to which Unicode wants to be roundtrip
compatible *did* duplicate the Katakana in two presentation widths, Unicode had
to allocate in the compatibility range (really just as a HACK) separate codes
for the halfwidth Katakana. I don't think, it is good practice to use these for
anything other than preserving round-trip compatibility with JIS-encoded files,
even if the KEYSYM layout obviously was originally inspired by the JIS X 201 set
that is today often customarily mapped to the Unicode halfwidth compatibility
characters.

I don't think this particular report needs any action with regard to Mozilla,
but I'm most happy to discuss the issue of the keysym <-> Unicode equivalence
table in X11R6.9 further with the original reporter.

Comment 11

[Simon Montagu :smontagu]

Perhaps we need to update widget/src/gtk*/keysym2ucs.* 

Comment 12

[Markus Kuhn]

What is the oldest libX11 version that still has to be be supported?

Considering that the proper solution, namely Xutf8LookupString(), has now been part of the X.Org implementation for many years, I would suggest that the entire temporary hack keysym2ucs.* hack can be removed by now. Removing code is good ...

Comment 13

[Simon Montagu :smontagu]

(In reply to comment #12)
> What is the oldest libX11 version that still has to be be supported?

bsmedberg, can you answer this question (or point to someone who can)?

Comment 14

[Benjamin Smedberg]

Probably the X that comes with RHEL 5 and the equivalent latest ubuntu LTS.

Comment 15

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee didn't login in Bugzilla in the last 7 months, so the assignee is being reset.