166144 - Hebrew vowel signs (diacritics) show as question marks

Status: RESOLVED FIXED

(Core :: Layout: Text and Fonts, defect)

Description

[Cees van Veelen]

User-Agent:       Mozilla/5.0 (OS/2; U; Warp 4.5; en-US; rv:1.1) Gecko/20020827
Build Identifier: Mozilla/5.0 (OS/2; U; Warp 4.5; en-US; rv:1.1) Gecko/20020827

This page is in Unicode (UTF-8). 
I know about the bug that many fonts display hebrew vowels not below the 
consonant, but give them a spave of their own. The Cardo font solves this problem. 
This page uses the <span lang="HE"> tag.
I downloaded the page, edited the HTML code by removing  this <span> code, and
all of a sudden the vowel signs and cantilation displayed correctly.

Probably related: Pages coded in charset windows-1255 also show only the
consonant text, and the vowels as question marks. For example
http://www.mechon-mamre.org/i/t/t2801.htm

Reproducible: Always

Steps to Reproduce:
1.In preferences choose a good font to display Unicode
2.Open a page in UTF, where the actual language is Hebrew, and vowel signs are used.
3.

Actual Results:  
I saw the vowel signs as question marks

Expected Results:  
Show the vowel signs as vowel signs.

I used the Cardo font from  http://members.telocity.com/~perryd
I think this bug is not platform specific, but I cannot test it.

Comment 1

[Simon Montagu :smontagu]

Which platform are you testing on? I can't reproduce this bug on either W2K or
RedHat 7.3

Comment 2

[Cees van Veelen]

I am using OS/2 Warp

Comment 3

[Mike Kaply [:mkaply]]

OS/2 Bidi is not up to par right now due to our font deficiencies.

We are working on new font code that will resolve this.

Comment 4

[Prognathous]

Displaying Nikud is the least of our problems at the moment.
First let's get the character order right. It seems that virtually all Hebrew 
websites are displayed in the reverse order.
This applies to both logical coding (windows-1255) and visual coding (iso-8859-
8).
For some reason, only the title bars are displayed properly.

Mike, are there any Bugzilla bugs that deal with BiDi on OS/2? if not, I'll be 
happy to open them and test future fixes. I'm using OS/2 Warp4 with built-in 
Hebrew support.

Comment 5

[Simon Montagu :smontagu]

prognathous, can you file a new bug for that and attach some screenshots? mkaply
told me that Hebrew is displayed in the correct order on OS/2 versions without
built-in Hebrew support.

Comment 6

[Prognathous]

Done.

Bug #180650
"Under OS/2, the character order of Hebrew coded websites is always reversed"

Comment 7

[Prognathous]

Confirmed.

This bug occurs on OS/2 and to a lesser extent on MacOS 9.2 (fewer question 
marks), it doesn't happen on Windows and OS X.
Linux is also broken - Nikud signs are displayed, but not in the right places.

Tested using the default Mozilla settings (without the Cardo font)

Comment 8

[Shoshannah Forbes]

changed title so this bug will pop up in searches. Marking as Confirmed after
comment #7

Comment 9

[Mike Kaply [:mkaply]]

Attachment: Fix for OS/2

On OS/2, we convert to native codepages before displaying. Unfortunately, the
vowels don't exist in 862.

Fix is to do all Hebrew rendering in 1208 (UTF-8)

Comment 10

[Cees van Veelen]

I will try to add a comment.
Michael Kaply wrote:
On OS/2, we convert to native codepages before displaying. Unfortunately, the
vowels don't exist in 862.

Fix is to do all Hebrew rendering in 1208 (UTF-8)

The bug occurs in two instances:
1. the coding is in UTF-8 but there is a lang=he tag present
2. the coding is in Windows-1255

Do I understand correctly that also pages in charset windows-1255 will be
rendered in UTF-8? In that case I think the solution is very good.

Comment 11

[Mike Kaply [:mkaply]]

> Do I understand correctly that also pages in charset windows-1255 will be
> rendered in UTF-8? In that case I think the solution is very good.

Yes.

On OS/2, since our Unicode rendering doesn't work as well as other platforms, we
convert everything to a native (usually windows) codepage before displaying.
This is because most fonts don't support Unicode or UTF-8.

Unfortunately, OS/2 doesn't have support for win-1255 or win-1256, so we were
converting to 862 and 864.

Since 862 doesn't have some characters that 1255 does, it is better to convert
Hebrew to Unicode or UTF8 for display.

Comment 12

[Mike Kaply [:mkaply]]

Fix checked in.

Comment 13

[Mike Kaply [:mkaply]]

Comment on attachment 116212 [details] [diff] [review]
Fix for OS/2

Sorry these are coming so late in the game.

This is an Os/2 only fix, no risk. Makes Hebrew work better.

Comment 14

[Asa Dotzler [:asa]]

Comment on attachment 116212 [details] [diff] [review]
Fix for OS/2

a=asa (on behalf of drivers) for checkin to 1.3