49262 - Hankaku Katakana with ISO-2022-JP in Composer will be displayed as "?" in Browser

Status: VERIFIED FIXED

(Core :: Internationalization, defect, P1)

Description

[Teruko Kobayashi]

When you change character coding to Japanese (ISO-2022-JP) and type Hankaku 
Katakana in Composer, the Hankaku Kataka will be displayed as "?" in Browser.

Steps of reproduce
1. Open Composer
2. Select menu View|Character coding ->Multibyte->Japanese (ISO-2022-JP)
3. Type Hankaku Katakana characters
4. Select menu File|Save As to save to the locale file
5. Open Brower and open the file you saved in step #4

Notice that the Hankaku Katakana characers in the page are displayed as "?" 
even though you change the character coding to Euc-jp, Shift_JIS, or Auto-Detect 
(Japanese),

If you use Communicator 4.x to look at the file with Auto-detect (Japanese), the 
Hankaku Katakana will be displayed correctly.

Tested 2000-08-16-09 Win32, Linux, and 2000-08-16-10 Mac build.

Comment 1

[nhottanscp]

Reassing to Frank. 
This might have been fixed by his check in yesterday, so please try today's 
build to see if this still happens.

Comment 2

[Teruko Kobayashi]

This still happens in 2000-08-17-08 Win32 build.

Comment 3

[Frank Tang]

teruko, does this happen to jab2 ?

Comment 4

[Teruko Kobayashi]

This is not regression. This happens in PR2 US and JA build.

Comment 5

[Jaime Rodriguez, Jr.]

Marking nsbeta3+ per I18N bug triage.

Comment 6

[Frank Tang]

this is acutally a Unicode to ISO-2022-JP conversion issue, not a ISO-2022-JP to
Unicode conversion issue. We generate something wrong.

Comment 7

[Frank Tang]

The problem is really not in the decoding part, but encoding part. We now change 
the encoding table to remove the hankaku katakana from the list. By doing so, 
the hankaku katakana will not be able encoded in ISO-2022-JP, so it will be 
encoded as NCR instead. 
I add the new mapping table jis0201gl.uf into mozilla/uconv/ucvja
Here is the additional patch needed to fix this problem


Index: ucvja/nsUCVJADll.h
===================================================================
RCS file: /m/pub/mozilla/intl/uconv/ucvja/nsUCVJADll.h,v
retrieving revision 1.4
diff -u -r1.4 nsUCVJADll.h
--- nsUCVJADll.h        1999/11/06 03:24:05     1.4
+++ nsUCVJADll.h        2000/08/25 08:57:17
@@ -32,6 +32,7 @@
 extern PRUint16 g_ut0212Mapping[];
 
 extern PRUint16 g_uf0201Mapping[];
+extern PRUint16 g_uf0201GLMapping[];
 extern PRUint16 g_uf0208Mapping[];
 extern PRUint16 g_uf0212Mapping[];
 
Index: ucvja/nsUCvJaModule.cpp
===================================================================
RCS file: /m/pub/mozilla/intl/uconv/ucvja/nsUCvJaModule.cpp,v
retrieving revision 1.12
diff -u -r1.12 nsUCvJaModule.cpp
--- nsUCvJaModule.cpp   2000/08/17 00:01:31     1.12
+++ nsUCvJaModule.cpp   2000/08/25 08:57:17
@@ -56,6 +56,9 @@
 PRUint16 g_uf0201Mapping[] = {
 #include "jis0201.uf"
 };
+PRUint16 g_uf0201GLMapping[] = {
+#include "jis0201gl.uf"
+};
 
 PRUint16 g_uf0208Mapping[] = {
 #include "jis0208.uf"
Index: ucvja/nsUnicodeToISO2022JP.cpp
===================================================================
RCS file: /m/pub/mozilla/intl/uconv/ucvja/nsUnicodeToISO2022JP.cpp,v
retrieving revision 1.3
diff -u -r1.3 nsUnicodeToISO2022JP.cpp
--- nsUnicodeToISO2022JP.cpp    1999/11/06 03:24:06     1.3
+++ nsUnicodeToISO2022JP.cpp    2000/08/25 08:57:17
@@ -49,7 +49,7 @@
 
 static PRUint16 * g_ufMappingTables[4] = {
   g_ufAsciiMapping,
-  g_uf0201Mapping,
+  g_uf0201GLMapping,
   g_uf0208Mapping,
   g_uf0208Mapping,
 };

Comment 8

[Frank Tang]

fix and check in

Comment 9

[Teruko Kobayashi]

I verified this in 2000-08-29-08 Mac build and 2000-08-29-15 build.