773062 - use ScriptExtensions data where possible to resolve OpenType script for shaping "common" character runs [was: OpenType cvxx feature works intermittently]

Status: NEW

(Core :: Layout: Text and Fonts, defect)

Description

[Bob Hallissy]

Attachment: Contains "cvxx failures - short.htm" and Scheherazade font.

User Agent: Mozilla/5.0 (Windows NT 6.1; rv:13.0) Gecko/20100101 Firefox/13.0.1
Build ID: 20120614114901

Steps to reproduce:

Disable Graphite rendering by going to about:config and setting gfx.font_rendering.graphite.enabled false.

Using Firefox 13.0.1 on Windows.

With Scheherazade font (in development -- contains both OpenType and Graphite logic) installed, open and display "cvxx failures - short.html" file.  (both files in attached zip).

Confirm GrCheck line says "ThisIsNotGraphite" (this confirms FF is using OpenType rather than Graphite logic).


Actual results:

2nd column of table shows characters rendered in Scheherazade. Two test cases (U+060C comma, and U+064C dammatan) are shown, and each test case has three subcases shown on succeeding lines: 
 -- No features, 
 -- with cvxx Character Variant feature, 
 -- same but following an arabic letter. 



Expected results:

In each set of three subcases, the variant glyph should show in lines 2 and 3 but shows only on 3rd.  

To be clearer, the line that says "same (but should show as alternate)" is formatted with a character variant feature but that feature is not firing. The lines that say "same following arabic letter does show alternate" are formatted with the same character variant and prove that the cvxx lookup works.

Comment 1

[Bob Hallissy]

Attachment: screenshot highlighting text that should be variant glyph but isn't.

Comment 2

[Bob Hallissy]

Tried on Nightly 16.0a1 (2012-07-11) and cvxx doesn't work AT ALL. I'm guessing the -moz-font-features syntax has changed but I can't figure out what it should be.

Comment 3

[John Daggett (:jtd)]

I tweaked the testcase a bit to support old/new syntax, include webkit/ms prefixed versions and link directly to the font via @font-face rule.

Steps:

Open URL link.

Result: alternates show when text span includes other Arabic characters but not in isolated span.

Note: syntax is described in the CSS3 Fonts spec and in the MDN page describing font-feature-settings:

https://developer.mozilla.org/en/CSS/font-feature-settings

Comment 4

[John Daggett (:jtd)]

Attachment: screenshot, comparison of fftrunk vs. chrome

Chrome-Dev for OSX renders pretty much the same (note that this version uses HarfBuzz shaping)

Comment 5

[Jonathan Kew [:jfkthame]]

I think the issue here is that the punctuation and diacritic characters you're using are not classified as *Arabic* script (see Unicode's Scripts.txt) but as Common and Inherited respectively. This means that when we do script-run analysis, they'll normally be included as part of the surrounding run and take on its script identity. However, when you have an "Arabic" comma or diacritic in isolation, we have no basis for identifying it as an Arabic script run, and so we don't apply the 'arab' opentype features.

A workaround would be to include the relevant features and lookups under the 'latn' (or 'DFLT' - I forget exactly how we handle these) script tag in the font, so that they're used regardless of whether the characters are actually in an Arabic-script run or not.

Comment 6

[John Daggett (:jtd)]

From Scripts.txt:

060C          ; Common # Po       ARABIC COMMA
0640          ; Common # Lm       ARABIC TATWEEL

Ah, that explanation makes sense.

Close INVALID as a font "bug"?

Comment 7

[Simon Montagu :smontagu]

Is bug 737931 relevant here? From http://unicode.org/Public/6.1.0/ucd/ScriptExtensions.txt:

060C          ; Arab Syrc Thaa # Po       ARABIC COMMA
0640          ; Arab Mand Syrc # Lm       ARABIC TATWEEL

Comment 8

[Jonathan Kew [:jfkthame]]

Well... although it's arguably a font shortcoming (the designer should handle these features within all the script tags that might be needed), I also wonder if we could do things to provide more expected/intuitive behavior here.

Currently, if gfxScriptRunItemizer returns a run of entirely Common/Inherited characters, where there was no script-specific character in the context to resolve these, we end up applying 'latn' shaping at http://mxr.mozilla.org/mozilla-central/source/gfx/thebes/gfxHarfBuzzShaper.cpp#976. Maybe we could do something smarter here, perhaps comparing scripts suggested by Unicode's ScriptExtensions.txt to the script tags supported by the font?

Comment 9

[Bob Hallissy]

Thanks for the suggestions. 

Adding the relevant cvxx features to 'latn' DOES fix the problem.

Speaking as a font developer, IMO it would be nice to "do something smarter here" as JK suggests, but I realize you may have more important things to work on as this is not a show-stopper and there is a reasonable workaround.

Many thanks to John Daggett for reworking the sample to demonstrate updated feature syntax. I confess I've read various pages of CSS/font-feature-settings documentation but never figured out what I was supposed to do. A relevant example is worth 1000 words of specification :-)

Comment 10

[John Daggett (:jtd)]

(In reply to Jonathan Kew (:jfkthame) from comment #8)

> Currently, if gfxScriptRunItemizer returns a run of entirely
> Common/Inherited characters, where there was no script-specific character in
> the context to resolve these, we end up applying 'latn' shaping at
> http://mxr.mozilla.org/mozilla-central/source/gfx/thebes/gfxHarfBuzzShaper.
> cpp#976. Maybe we could do something smarter here, perhaps comparing scripts
> suggested by Unicode's ScriptExtensions.txt to the script tags supported by
> the font?

Hmmm, what precisely do you think we should do here?  I think we should rename the bug here to reflect whatever that is rather than leave the current bug title.

Comment 11

[Jonathan Kew [:jfkthame]]

If gfxScriptRunItemizer returns with script code MOZ_SCRIPT_COMMON, we should check the ScriptExtensions property for each character in the run to look for any "hints" as to the likely script(s) being used, and compare to the script(s) supported by the font being used.

Exactly how to resolve ambiguities where there are multiple "candidate" scripts is left as an exercise.....