OS/2 only - some big5 characters show as "?"

RESOLVED FIXED

Status

()

RESOLVED FIXED
16 years ago
15 years ago

People

(Reporter: ccwu, Assigned: jhpedemonte)

Tracking

({fixed1.7})

Trunk
x86
OS/2
fixed1.7
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(URL)

Attachments

(2 attachments, 4 obsolete attachments)

(Reporter)

Description

16 years ago
User-Agent:       Mozilla/5.0 (OS/2; U; Warp 4.5; en-US; rv:1.3a) Gecko/20021210
Build Identifier: Mozilla/5.0 (OS/2; U; Warp 4.5; en-US; rv:1.3a) Gecko/20021210

some big5 characters show as "?"
in os/2, we use cp950 as big5. this is the same in windows.
there are many similar bug reports before, and i think it should be fixed.
but it's not in os/2.
it's strange that i can see these characters in windows 2000.

Reproducible: Always

Steps to Reproduce:
1.
2.
3.



Expected Results:  
is it possible to use cp950 as big5 and big5hkscs like windows
or add one character coding "cp950"?
btw, i prefer the first one solution.
(Reporter)

Comment 1

16 years ago
*** Bug 189488 has been marked as a duplicate of this bug. ***

Comment 2

16 years ago
I've been randomly seeing ? in place of ' for many weeks in OS/2 trunk builds
using default En US charset.
OS: other → OS/2

Comment 3

16 years ago
Created attachment 111858 [details]
Reduced testcase

Comment 4

16 years ago
The character in question is F9D6 which is in the user defined range of codepage
950.

When Mozilla reads a page, it converts the contents to Unicode and then it is up
to the OS to convert it back to the native codepage or display it in Unicode.

In this case, Mozilla converts it to a Unicode char and when we ask OS/2 to
convert it back, it doesn't know what to do with it because there is no standard
for converting from Unicode to the user defined range.

We have some plans to change our code to use Mozilla converters instead of OS/2
converters and that might help this problem.
Assignee: smontagu → mkaply
Status: UNCONFIRMED → NEW
Ever confirmed: true

Comment 5

16 years ago
Looks like OS/2 problem - no reproducible on Windows, Mac OS X and linux.

Comment 6

16 years ago
More information. At issue here is the ETen extension. Here is some info from 

http://seba.ulyssis.org/thesis/big5_2.php

> A very important extension is the ETen extension. It contains two blocks,
> the first block (C6A1 - C3D3) has 365 characters (numbers, kana, ...) the
> second block (F9D6 - F9FE) only has 7 hanzi (碁 銹 裏 墻 恒 粧 嫺) and 34 line
> drawing characters.

So Mozilla maps F9D6 to the ETen extension characters in Unicode, but OS/2 does not.

As mentioned in the previous post, when we convert our fonts to use Mozilla
codepage conversions, this will be fixed.

Updated

16 years ago
Summary: some big5 characters show as "?" → OS/2 only - some big5 characters show as "?"
(Reporter)

Comment 7

16 years ago
i think this is problem of os/2 not mozilla after reading your coments,
and use Mozilla converters instead of OS/2 converters would fix this.
thank you!!

i've looked more information.
in perl and linux locale they use big5 as alias of big5-eten now.
http://www.mathematik.uni-ulm.de/help/perl5/doc/Encode/TW.html

ps.big5 < cp950 < big5-eten < big5-hkscs
   ibm didn't implement big5-hkscs in os/2 :(

Comment 8

15 years ago
*** Bug 237506 has been marked as a duplicate of this bug. ***
(Assignee)

Comment 9

15 years ago
Created attachment 144168 [details] [diff] [review]
patch

This patch changes WideCharToMultiByte and MultiByteToWideChar to use the
nsIUnicodeEncoder and nsIUnicodeDecoder, instead of OS/2's unicode converters. 
With this patch, the attached testcase displays somewhat better, but the center
character now displays as a square.  Don't know if there is anything we can do
about that, though.
Also, this patch makes the testcase in bug 237506 work.
Assignee: mkaply → pedemont
Status: NEW → ASSIGNED
(Assignee)

Updated

15 years ago
Attachment #144168 - Flags: review?(mkaply)
(Assignee)

Updated

15 years ago
Blocks: 238039

Comment 10

15 years ago
Created attachment 145104 [details] [diff] [review]
Change platform charset code

Use nsPlatformCharset to get platform charset.

remerge clipboard code.
Attachment #144168 - Attachment is obsolete: true

Comment 11

15 years ago
Comment on attachment 144168 [details] [diff] [review]
patch

r=mkaply for the pieces that are the same
Attachment #144168 - Flags: review?(mkaply) → review+

Comment 12

15 years ago
Created attachment 145119 [details] [diff] [review]
The real slim shady

oops
Attachment #145104 - Attachment is obsolete: true
(Assignee)

Comment 13

15 years ago
Created attachment 146026 [details] [diff] [review]
more fixes

(1) Fix memory issue in GetUconvObject.
(2) Set error behavior for nsIUnicodeEncoder in nsOSUni.cpp.
Attachment #145119 - Attachment is obsolete: true
(Assignee)

Comment 14

15 years ago
Created attachment 146027 [details] [diff] [review]
more fixes II

Posting correct patch.
(Assignee)

Updated

15 years ago
Attachment #146026 - Attachment is obsolete: true
(Assignee)

Updated

15 years ago
Attachment #146027 - Flags: review?(mkaply)

Updated

15 years ago
Depends on: 240596, 240597

Comment 15

15 years ago
Comment on attachment 146027 [details] [diff] [review]
more fixes II

r=mkaply
sr=blizzard (platform specfic)

once the dependent bugs are in, we need to update os2charset.properties with
all OS/2 charsets.
Attachment #146027 - Flags: superreview+
Attachment #146027 - Flags: review?(mkaply)
Attachment #146027 - Flags: review+

Comment 16

15 years ago
Comment on attachment 146027 [details] [diff] [review]
more fixes II

a=mkaply for 1.7 - OS/2 only
Attachment #146027 - Flags: approval1.7+

Comment 17

15 years ago
Fix checked in.
Status: ASSIGNED → RESOLVED
Last Resolved: 15 years ago
Keywords: fixed1.7
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.