Closed Bug 719366 Opened 12 years ago Closed 10 years ago

@font-face tone-mark floating issue in Thai language when using hardware acceleration.

Categories

(Core :: Layout: Text and Fonts, defect)

9 Branch
x86_64
Windows 7
defect
Not set
major

Tracking

()

RESOLVED WORKSFORME

People

(Reporter: 95kreaninw95, Unassigned)

Details

(Keywords: fonts)

Attachments

(5 files)

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0.1) Gecko/20100101 Firefox/9.0.1
Build ID: 20111220165912

Steps to reproduce:

Used @font-face to style my texts in Thai language.


Actual results:

I found out later that it cause a tone-mark floating issue when enabled hardware acceleration.


Expected results:

The floating is unacceptable in Thai language, so it should display correctly like when turned off hardware acceleration.

I have discussed this issue at http://forums.mozillazine.org/viewtopic.php?f=9&t=2404253 and confirmed it's a bug and only happens in Firefox. The others are effect by the bug too, this bug should be fixes as soon as possible.
Severity: normal → major
Keywords: fonts
Comment on attachment 589806 [details]
A test file for this issue.

A test file for this issue.
Attachment #589806 - Attachment description: test.7z → A test file for this issue.
Attachment #589806 - Attachment mime type: text/plain → application/x-7z-compressed
Added the 2nd test case linked to the mozillazine.org topic.
Attachment #589823 - Attachment mime type: application/octet-stream → application/zip
Component: General → Layout: Text
QA Contact: general → layout.fonts-and-text
Confirming on the basis of this post:  http://forums.mozillazine.org/viewtopic.php?p=11648801#p11648801 plus the reporter.
Status: UNCONFIRMED → NEW
Ever confirmed: true
The same problem shows up with this testcase on OS X, in both Firefox _and_ Safari, showing that it is not limited to FF/Win/HWacc. (I haven't tried it on Linux yet.)

This is actually a font error: both the example fonts expect to use OpenType substitution features to adjust the position of the diacritics (by replacing the default glyphs, which are positioned quite high up, with lower versions when they occur after low characters, etc.)

However, the necessary features have been included in the fonts under the _Latin_ script, not the Thai script (nor under the OpenType "default" tag). As a result, they don't get applied to runs of Thai characters.

By changing the script tag in the OpenType tables to the correct 'thai' value, I got the expected rendering.

It seems that Uniscribe (which is used when h/w acceleration is disabled) must have some fallback or "compatibility" mode to support such fonts, whereby if the correct script is not supported in the font, and there's no DFLT script either, it applies the features from the Latin script. This is why it works with h/w acc disabled. However, DirectWrite (used with h/w acc enabled) does not appear to be doing this by default, although it looks like some Microsoft applications, at least, must be checking what scripts are supported in the font and explicitly choosing to applying the Latin behavior.

I suppose we could add similar fallback behavior to Firefox, for the sake of better rendering with fonts like this; however, the best solution is for the fonts to be fixed so that they correctly support Thai script.

Behdad, would you be open to including fallback-to-Latin (as a last resort if neither the requested script nor DFLT is available in the font) in harfbuzz? It seems like it would improve compatibility with some legacy fonts, and I can't think offhand of any problems it would create.
Jonathan, why some users are able to display the diacritics on the low position with Firefox and HWA enabled? (see my attachment)

One of users has a lang pack installed on his computer to display Asian characters in applications ("FF-Chrome" in the snapshot).
(In reply to Loic from comment #7)
> Created attachment 590163 [details]
> Snapshot FF/Chrome and FF/IE-Metro
> 
> Jonathan, why some users are able to display the diacritics on the low
> position with Firefox and HWA enabled? (see my attachment)

Are you sure HWA is actually being used in that case? It may be blocked due to graphics card or driver incompatibilities, in which case Firefox will use the old GDI/Uniscribe text system. You can check whether acceleration is enabled by looking in the Graphics section at the end of about:support.
You're right, I asked a confirmation to both users, HWA was disabled indeed:
Direct2D Enabled = false
DirectWrite Enabled = false (6.1.7600.16763 or 6.2.8102.0)
This patch fixes the rendering via harfbuzz of the SPLaksaman font, where the features are all (incorrectly) stored under the 'latn' script tag.

It does not help for TFSrivichai, which seems to require custom substitutions of Private Use codes; it has _some_ OpenType layout tables (again, tagged as 'latn') but these do not actually implement the replacements needed to improve the tone mark positions. I presume this is an example of the kind of legacy font recently discussed on the harfbuzz list:

"Legacy fonts like Tahoma rely on composition-by-substitution, and requires the
rendering engine to properly pick up [PUA] glyphs with vendor-specific
code points."

I'm not convinced we should perpetuate this practice by adding custom code to support vendor-specific PUA-based rendering - that's the kind of rathole that Unicode and OpenType are supposed to avoid.

So to recap, this patch can help for SPLaksaman, which *tries* to do the right thing by providing OpenType Layout features, but implements it in a slightly broken way (with the wrong script tag); it will not help for TFSrivichai, which requires custom vendor/font-specific support code.

Behdad, WDYT?
Attachment #590452 - Flags: review?(mozilla)
(In reply to Jonathan Kew (:jfkthame) from comment #6)
> The same problem shows up with this testcase on OS X, in both Firefox _and_
> Safari, showing that it is not limited to FF/Win/HWacc. (I haven't tried it
> on Linux yet.)

On Linux, it's rendered properly, thanks to the PUA support in Pango (and probably
the lack of hardware acceleration support :P ).

> This is actually a font error: both the example fonts expect to use OpenType
> substitution features to adjust the position of the diacritics (by replacing
> the default glyphs, which are positioned quite high up, with lower versions
> when they occur after low characters, etc.)
> 
> However, the necessary features have been included in the fonts under the
> _Latin_ script, not the Thai script (nor under the OpenType "default" tag).
> As a result, they don't get applied to runs of Thai characters.

As the maintainer of SPLaksaman, I'd say it was derived from THSarabunPSK
with all OpenType features retained. The THSarabunPSK creator based their works
on a font template prepared by a governmental organization, aiming at Windows,
Mac, and Adobe supports. Unfortunately, Adobe software doesn't recognize Thai.
So, they ended up with the 'latn' hack.

> By changing the script tag in the OpenType tables to the correct 'thai'
> value, I got the expected rendering.

I think I'll modify SPLaksaman by *adding* 'thai' and 'DFLT' script tags.
But I can't modify other fonts that are based on the original template,
including THSarabunPSK itself and TFSrivichai.

> It seems that Uniscribe (which is used when h/w acceleration is disabled)
> must have some fallback or "compatibility" mode to support such fonts,
> whereby if the correct script is not supported in the font, and there's no
> DFLT script either, it applies the features from the Latin script. This is
> why it works with h/w acc disabled. However, DirectWrite (used with h/w acc
> enabled) does not appear to be doing this by default, although it looks like
> some Microsoft applications, at least, must be checking what scripts are
> supported in the font and explicitly choosing to applying the Latin behavior.
> 
> I suppose we could add similar fallback behavior to Firefox, for the sake of
> better rendering with fonts like this; however, the best solution is for the
> fonts to be fixed so that they correctly support Thai script.

Agreed. But as I said, I can't touch those other fonts. And unfortunately,
THSarabunPSK has already been distributed widespread, as it's being enforced
as the standard font in all governmental documents.

So, the Latin fallback may be necessary.
As the reporter of the bug, thanks everyone for investigate this issue. I'm not expert with the font system. But in my opinion, it should be fixed in Firefox as the interpreter of the web, and then the font in the future. Every programs/apps have to correctly fallback the issue. Like when Firefox does when HWA disabled, like the other browsers does on the same OS.
Should this block bug 718479?
Comment on attachment 590452 [details] [diff] [review]
patch, fall back to applying 'latn' features if thai and default are both missing

Review of attachment 590452 [details] [diff] [review]:
-----------------------------------------------------------------

LGTM.  Pushed upstream without the public header file change.
Attachment #590452 - Flags: review?(mozilla) → review+
Pushed the latn-fallback patch to inbound:
https://hg.mozilla.org/integration/mozilla-inbound/rev/8ee43a7c94dd

** Please do NOT close this bug when merging to m-c; there's more to be done here. **
cc-ing smontagu for any comments from an i18n POV.

Current status: the patch just pushed to inbound will fix _some_ of the problem fonts (such as SPLaksaman) on OS X, where we use harfbuzz for Thai. It will not help for fonts such as TFSrivichai that require custom PUA-based shaping.

Moreover, it will not immediately help on Windows/D2D because we don't use harfbuzz for Thai script on Windows; we use the platform shaping libraries. We could change that by setting the appropriate bit (value 64) in gfx.font_rendering.harfbuzz.scripts; this would fix SPLaksaman for Windows/D2D, but unfortunately it would also _regress_ the PUA-based fonts like TFSrivichai under GDI, where they currently work (thanks to Uniscribe).
We have PUA shaping logic in Pango.  The code is really ugly though, so someone fully understanding the logic has to rewrite it as a state machine or something.  Donno.  Is there any docs around I can study?
(In reply to Behdad Esfahbod from comment #17)
> We have PUA shaping logic in Pango.  The code is really ugly though, so
> someone fully understanding the logic has to rewrite it as a state machine
> or something.  Donno.  Is there any docs around I can study?

My attempt to restructure it was in libthai [1], as well as the ceased pango-libthai [2] project.

  [1] http://linux.thai.net/projects/libthai
  [2] http://linux.thai.net/projects/pango-libthai

In libthai, the shaping was done in two stages: cell tokenization (thcell) and cell re-composition (thrend).

In pango-libthai, a libthai-based shaping engine was also provided. But it has not been merged into mainstream pango. Only the word break engine was done. The merge, however, would cause regression for Lao support.
Thanks Thep.  Is there any documentation?  I'm not really interested in any code without understanding the problem.  That's my design goal for HarfBuzz :).
There is no document for this currently, as  the problem looked too obvious to Thai developers. It seems the best way is to write one myself, a PUA version of this one:

  http://linux.thai.net/~thep/th-otf/

Give me some time to write it.
Yeah.  It's like how Arabic shaping is obvious for anyone who grew up learning it :).

Ok, ping me when you have something written up.
(In reply to Behdad Esfahbod from comment #21)
> Ok, ping me when you have something written up.

I've quite finished a first draft:

http://linux.thai.net/~thep/th-otf/shaping.html
Thanks.  I've implemented the SARA AM decomposition now.  The rest I believe belongs to the GPOS table to reposition, or our fallback mark positioning code (to be written).  No standing re PUA shaping as of now.
Theppitak: is there anything further we need to do here, or has this been resolved by the various harfbuzz improvements that have been incorporated? I'm wondering if we can close this bug now?
Flags: needinfo?(thep)
With the 'latn' fallback, and even Thai PUA support, in harfbuzz, I believe most Thai fonts are covered. But I cannot verify this on Windows and Mac. On Linux, it renders the test case fine.
Flags: needinfo?(thep)
OK, thanks. It looks fine here on OS X, too.

I think we should close this bug, and if any further issues show up, open new reports as needed.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: