392209 - [10.5] Thai vowels and tone marks displayed incorrectly on Leopard 9a499

Status: RESOLVED WORKSFORME

(Core :: Layout: Text and Fonts, defect)

Description

[Disorn Homchuenchom]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/419.3 (KHTML, like Gecko) Safari/419.3
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.9a7) Gecko/2007080209 GranParadiso/3.0a7

Thai vowels and tone marks are displayed incorrectly on Leopard 9a499. They always layout before actual layout position (as seen on Safari or other applications that accepts Thai characters).

Reproducible: Always

Steps to Reproduce:
1. Open GranParadiso (or Minefield).
2. Surf some Thai web pages. (i.e. http://www.blognone.com)

Actual Results:  
You will see an incorrect display on screen.

Expected Results:  
Layout Thai vowels and tone marks correctly.

In BonEcho (on Leopard), I see ? instead of some of Thai vowels.

Comment 1

[Disorn Homchuenchom]

Attachment: Gran Paradiso on Leopard 9a499

Comment 2

[Disorn Homchuenchom]

Attachment: BonEcho on Leopard 9a499

Comment 3

[Eli Friedman]

I don't recall any significant changes to this since alpha 7, but would you mind double-checking that it is still wrong in a trunk build?  (ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk)

Comment 4

[Eli Friedman]

Attachment: Rendering of the same page on Windows

This is a rendering of the same page on WinXP; is this rendering correct?

Comment 5

[Disorn Homchuenchom]

(In reply to comment #4)
> Created an attachment (id=276648) [details]
> Rendering of the same page on Windows
> 
> This is a rendering of the same page on WinXP; is this rendering correct?
> 

That is a proper render (with vowels and tone marks). It is still not all correct because of Thai word breaking.

I've tried one of pre alpha 8 and got a same result. May be tonight (my local time), I can try another latest trunk.

Comment 6

[Disorn Homchuenchom]

Attachment: Minefield 20070814 on Leopard 9a499 (with another bug!)

As you see, I get the same incorrect render result. This is a platform specific bug.

PS. Do you see another File menu? Click it to crash Minefield (and GranParadiso).

Comment 7

[Eli Friedman]

Thanks for testing it out.

(In reply to comment #6)
> PS. Do you see another File menu? Click it to crash Minefield (and
> GranParadiso).

No idea what's going on there; file another bug, please (one issue per bug).

Comment 8

[Disorn Homchuenchom]

(In reply to comment #7)
> 
> No idea what's going on there; file another bug, please (one issue per bug).
> 

OK. Done.

Comment 9

[Isriya Paireepairit]

This should be separated cases.

From attachment #276644 [details] (Grand Paradiso), it's the issue of this bug (vowels and tone marks are displayed one caret before the letters). See the difference from attachment #276648 [details] (Windows) at the tone marks displayed after "Unix" and "Novell" word.

On attachment #276646 [details] (Bon Echo) only 'tone marks' and 'following vowels' (the vowels that come after leading letters) are displayed as 'question mark' (?). Other types of vowel: leading vowels, above vowels and below vowels are fine.

Comment 10

[Simon Montagu :smontagu]

Attachment: reduced testcase

This would be easier for those following at home if all the screen shots used the same text.

Comment 11

[Eli Friedman]

(In reply to comment #9)
> This should be separated cases.

Well, the Fx 2.0 case doesn't really matter... that branch isn't being actively developed at the moment, so the chances of a fix for a bug like this on that branch are rather low.

Comment 12

[Keng Susumpow]

Attachment: minimal test case with fx 3 pre alpha 8 (Bug 389520 patch)

the minimal testcase is fine on grand paradiso pre alpha 8 with #389520 patch.

Comment 13

[Eli Friedman]

Hmm? It doesn't look like bug 389520 should affect this... maybe there's something special about the way the reporter set up OS X.  How does the minimal testcase look in a stock nightly build?

Comment 14

[Disorn Homchuenchom]

I've done all tests on Leopard 9a499.
I've tried Bug 389520 patch and it also displayed the same thing (incorrect locations for vowels and tone marks) except correct word breaking.

So I think, this is not related to Thai word breaking patch (Bug 389520).

Comment 15

[Eli Friedman]

Considering that this appears to be Leopard specific, I'd suggest filing a bug report with Apple... it sounds like their fault.

Comment 16

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Someone could test other ATSU-using apps Leopard and see if they behave correctly. We use ATSU in a slightly weird way so it's possible something changed that confused us.

Comment 17

[Disorn Homchuenchom]

I'm not sure whether Safari uses ATSUI or new Text Layout framework (because ATSUI may be obsolete) but Safari is fine.

Comment 18

[Disorn Homchuenchom]

Seem likes this bug causes by font rendering (and selection) strategy on Leopard (that firefox should investigate this).

I've updated Leopard to 9a500n (latest ADC seed) and this problem is still unsolved.
I've tested one web page using Tahoma and it shows partially correct rendering (the second level above tone marks render as the first level tone marks which isn't correct).

I will create a test case for this issue.

Note that I haven't installed any Thai fonts in my machine except the system bundling fonts.

Comment 19

[John Daggett (:jtd)]

(In reply to comment #18)

Could you try running one of the testcases for bug 361986 for me on 9a500n?  

https://bugzilla.mozilla.org/attachment.cgi?id=277665

Just open the URL above and click on the "Display" button.  This is a page that renders text with a large assortment of fonts.  Does the Thai text render correctly in all cases, some cases or not at all?  ATSUI uses a lot of per-font information so this may need to be something we work around per-font, depending on what's going on.  A small screenshot of incorrectly rendered text would be a huge help (use Cmd-Shift-4 to capture a selection to a PNG file).  Thanks!

Comment 20

[Disorn Homchuenchom]

Attachment: Safari 3 as a reference!

Thanks John. I modify your test case a little bit. :) I pick up Thai fonts bundled with OS to the top list (including Microsoft ones those don't show themselves as Thai fonts in Font Book: Microsoft Sans Serif, and Tahoma).

See Safari as a reference. Safari uses "Thonburi" as a default Thai characters for fonts those don't have Thai characters. Even though Ayuthaya is Thai font, it renders incorrectly here but it is Thai so no substitution.

Comment 21

[Disorn Homchuenchom]

Attachment: Minefield on Leopard 9a500n

And how about Minefield? It uses "Ayuthaya". That is why almost screenshots I posted here are rendered incorrectly.

Comment 22

[Disorn Homchuenchom]

Attachment: Minefield on Leopard 9a500n with many Thai fonts installed

OK. Last one. This screenshot was captured two days ago. (I just reinstalled Leopard yesterday.) At that time, I installed a lot of Thai fonts to my machine. (I test how Microsoft Office 2004 dealing with Thai fonts.) Minefield doesn't pick "Ayuthaya" as expect but picks up other font (I don't know what it is). Also BonEcho could display text correctly (very surprise)! But it doesn't do correctly right now (without install a lot of Thai fonts).

Comment 23

[Disorn Homchuenchom]

Last note, I miss a thing. Microsoft Sans Serif and Tahoma display Tone marks incorrectly on both Safari and Minefield.

Comment 24

[Robert O'Callahan (:roc) (email my personal email if necessary)]

OK, so this is a Mac font selection issue.

Comment 25

[John Daggett (:jtd)]

Not sure this is purely a font substitution issue.  On Leopard 9A499 the testcase from bug 361986 renders incorrectly for the Thai font Ayuthaya both in FF3 trunk latest and in Safari.  Both currently show a vowel mark by itself at the beginning of the string of Thai characters.  This should appear above the next glyph, not by itself.  

Other Thai fonts (Krungthep, Sathu, Silom) render correctly.  When font substitution occurs, Safari renders correctly (except for Andale Mono, Courier and Monaco) but FF3 trunk does not.

Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.9a8pre) Gecko/2007082204 Minefield/3.0a8pre

Safari Version 3.0 (5522.11)

Webkit nightly r25168

Logged Webkit problem as:

http://bugs.webkit.org/show_bug.cgi?id=15066

Comment 26

[Disorn Homchuenchom]

Seems like the only Thai monospace font is "Ayuthaya". That is why (I guess) other monospace fonts render incorrectly.

I try to change Ayuthaya with Tiger version but the problem isn't solved. Surprisedly, Ayuthaya displays correctly in TextEdit. So I don't think it causes by font problem.

I've installed some propriety Thai monospace fonts (Windows one) to Leopard. It overrides Ayuthaya so now all monospace texts are rendered partially correct (because it doesn't use Ayuthaya anymore).

I think this is a mac specific problem since BonEcho that All of Thai text displays with incorrect font. New Cairo engine solves some of this issue but also leaves some.

Comment 27

[David D. Kilzer (ddk)]

(In reply to comment #25)
> Logged Webkit problem as:
> http://bugs.webkit.org/show_bug.cgi?id=15066

Reported to Apple as:  <rdar://problem/5435331>

Thanks for taking the time to file a bug in WebKit's Bugzilla!

In the future, please file all Leopard-related issues (that are not WebKit-specific bugs) using <https://bugreport.apple.com/>.  Note that you may create a free "online" account using <https://connect.apple.com/> to file bugs on this system.

Comment 28

[John Daggett (:jtd)]

Tested with 9A527, problem still present.

Comment 29

[John Daggett (:jtd)]

Turned on debugging code to dump out name of substituted font and reran testcase from bug 361986.  Tested with three fonts, Ayuthaya, Silom and Verdana.

Ayuthaya - Thai font, no Devanagari or Arabic glyphs
Silom - Thai font, no Devanagari or Arabic glyphs
Verdana - Latin font, no Devanagari, Arabic or Thai glyphs

Devanagari:

10.4 Ayuthaya ==> Akshar Unicode (dislay incorrect)
9A527 Ayuthaya ==> DevanagariMT (display correct)

- same for Silom, Verdana -

Arabic: 

10.4 Ayuthaya ==> ArialMT (display incorrect)
9A527 Ayuthaya ==> AlBayan (display correct)

- same for Silom, Verdana - 

Thai:

10.4 Ayuthaya (no substitution) (display correct)
9A527 Ayuthaya (no substitution) (display incorrect)

- Silom, no substitution, display correctly in both 10.4 and 9A527

10.4 Verdana ==> Ayuthaya (display correct)
9A527 Verdana ==> Ayuthaya (display incorrect)

So the font substitution algorithm used in 9A527 fixes the problem in bug 361986 but breaks the rendering of Thai using Ayuthaya or any other non-Thai font.

Next step: see if I can find a contact at Apple I can talk to about this.

Comment 30

[John Daggett (:jtd)]

As part of work on 361986 I wrote a small ATSUI app that simply draws Devanagari / Arabic / Thai text using a set of specified fonts.  Running this app under the 9A527 seed shows the same problem with Ayuthaya when rendered via ATSUI.  So this is best handled as an Apple bug.  As part of the fix for 361986, I'm going to set up a way default fallback mechanism that allows us to avoid fonts with known problems, Ayuthaya under 10.5 is just one example of this.

Disorn, if you have a chance could you try running the ATSUI test program just to confirm that Ayuthaya renders incorrectly with it on your install of 10.5?  If any other Thai font renders incorrectly, that would be good to know also, all the ones I have with my install work except for Ayuthaya.

https://bugzilla.mozilla.org/attachment.cgi?id=279419

Comment 31

[John Daggett (:jtd)]

Attachment: Ayuthaya rendering in 9A527

Running test app from bug 361986, this is how Ayuthaya and Silom display.  Silom is correct, Ayuthaya is not.

Comment 32

[John Daggett (:jtd)]

With fix for 299222, we'll be a lot less susceptible to this, since font fallback will always occur to a known "good" font and not Ayuthaya in the default case.  Ayuthaya is still specified as the default monospace font but we can change that if needed.

I mailed the Apple engineer responsible for ATSUI, hopefully I'll hear back with his take on the problem.

Comment 33

[John Daggett (:jtd)]

Feedback from Apple:

This appears to be a font bug related to the prop table contained in the version of Ayuthaya that shipped with the 10.5 seed.  This is logged as Apple bug rdar://problem/5490856.

Comment 34

[John Daggett (:jtd)]

Testing with 9A559 seed (24 Sept 2007), problem still exists but as noted above only for pages that explicitly specify Ayuthaya.  With the changes from bug 299222, other pages shouldn't fallback to Ayuthaya except in the monospace case.

Comment 35

[John Daggett (:jtd)]

Attachment: Ayuthaya rendering on 10.5 GM

With 10.5 GM build (9A581) the font bug with Ayuthaya appears to be fixed, yay!!

Marking this as WORKSFORME

Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.9a9pre) Gecko/2007102804 Minefield/3.0a9pre