1220708 - Lepcha script rendering

Status: RESOLVED INVALID

(Core :: Layout: Text and Fonts, defect)

Description

[serkhang]

Attachment: Firefox_Comparison_marked.png

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:41.0) Gecko/20100101 Firefox/41.0
Build ID: 20151014143721

Steps to reproduce:

Lepcha script embedded using @font-face in the experimental draft-page

http://www.darjeeling-unlimited.com/rongring.html

is displayed correctly in Unicode compliant text-editors, in webkit browsers as well as in Firefox 41 under Windows, but not in Firefox 41 under OSX 10.8 to 10.10.


Actual results:

Certain ligatures are wrongly rendered. See attached screenshot, right side (OSX).


Expected results:

See attached screenshot, left side (Windows).

Comment 1

[Boris Zbarsky [:bzbarsky]]

Interestingly, I see identical behavior in Chrome on OSX and Firefox on OSX, both showing what the right-hand side of the screenshot shows.  Safari shows what the left-hand side shows...

Comment 2

[Jonathan Kew [:jfkthame]]

I'm puzzled by the report that FF/Win and FF/OSX behave differently, and I can't reproduce that here; with FF41.0.2 on Win8.1, I see the same "incorrect" rendering as on OS X.

Anyway, ignoring that oddity, the problem here is related to the Lepcha font that's being used (MainwaringRong.woff). This font is designed to work with OpenType rendering systems that do *not* really support Lepcha script, but just apply default features (it relies entirely on 'liga' and 'mark', and expects them to be applied directly to the text as encoded).

However, in recent versions of Firefox (and Chrome) the harfbuzz engine has support for Lepcha, including in particular the automatic reordering of pre-base vowel matras such as U+1C28 VOWEL SIGN O. This conflicts with the font, which expects to see sequences of <consonant, vowel-sign-o> and render them using a ligature that combines the vowel-sign glyph (visually on the left) and the consonant, because the reordering that harfbuzz applies will put the vowel sign before the associated consonant; but the font's 'liga' rules will then ligate it with the *previous* consonant (if any).

For example, consider a word like ᰊᰌᰨᰠᰦ (character sequence <1C0A "ta", 1C0C "da", 1C28 "vowel sign o", 1C20 "sa", 1C26 "vowel sign aa">). In a renderer that does NOT have Lepcha support, the 'liga' feature in the font will form ligatures of da+o and sa+aa, resulting in the glyphs <uni1C0A, uni1C0C1C28.liga, uni1C201C26.liga>.

But with Lepcha support that includes reordering, the "o" sign will be moved in front of "da", and will then ligate with "ta", so we see <uni1C0A1C28.liga, uni1C0C, uni1C201C26.liga>, and the vowel sign "o" appears visually on the left of the "ta" with which it ligated.

For more about OpenType support for Lepcha (and many other scripts), see the Microsoft Universal Shaping Engine (USE) spec.[1]

FWIW, I suspect the same problem will show up in many applications on Windows 10 (not in earlier versions of Windows, as Lepcha support did not exist in earlier Uniscribe/DirectWrite versions), but I don't have a Win10 system on hand to verify this.

The correct solution here is for the font developer to implement Lepcha support as documented in [1], using the features specified for Indic script support (and cooperating with the Unicode-based reordering in the engine) rather than assuming a generic (non-Indic) engine with no knowledge of Lepcha script, and then ligating all the consonant-vowel combinations.

In particular, this is what should be implemented under the 'lepc' script tag. The problem here arises because the font uses the script tag 'lepc', which is interpreted as meaning that it supports the OpenType model for Lepcha rendering; but then under that tag, it has a ligature feature that was actually designed for a non-Lepcha engine.

A quick-and-dirty workaround would be to simply *remove* the 'lepc' script tags from the font's GSUB and GPOS tables, leaving the existing features only under the 'DFLT' tag. Seeing this, harfbuzz will assume that the font does not actually know about Lepcha shaping behavior and just wants generic feature application, and so it'll render "correctly" (as it does in applications that lack Lepcha support). However, implementing a correct 'lepc' script as per the USE spec is the proper long-term solution (and should allow the font to be considerably simplified).

[1] http://www.microsoft.com/typography/OpenTypeDev/USE/intro.htm

Comment 3

[serkhang]

@Jonathan Kew: My admiration and gratitude for your concise comments. You are absolutely right: Removing the 'lepc' tag is an instant workaround which I'll apply for the time being.

However, since I'm the developer of the font as well, I'll consider your hints to use the existing support for *true* Lepcha rendering support. I suspect that it may be wise, however, to follow both the tracks for some time, i.e. unless genuine rendering of Lepcha script is implemented on a wider range of applications.

If I may ask you a follow-up: Am I right in assuming that the flaws in the rendering of the second example at the same page (using Noto Sans Lepcha) are due to other reasons? Best thanks again.

Comment 4

[Jonathan Kew [:jfkthame]]

(In reply to serkhang from comment #3)
> @Jonathan Kew: My admiration and gratitude for your concise comments. You
> are absolutely right: Removing the 'lepc' tag is an instant workaround which
> I'll apply for the time being.
> 
> However, since I'm the developer of the font as well, I'll consider your
> hints to use the existing support for *true* Lepcha rendering support. I
> suspect that it may be wise, however, to follow both the tracks for some
> time, i.e. unless genuine rendering of Lepcha script is implemented on a
> wider range of applications.

Yes, at this point many applications/systems will not support the 'lepc' script behavior, and so your current approach of "ligatures for everything" is the only way to get the desired rendering.

What *might* work (I haven't tested this) is to put two separate sets of features/lookups in the font: the current "ligatures for everything" approach, under the 'DFLT' script tag, and a set of features designed for the USE specification under the 'lepc' tag. This might allow systems (harfbuzz, Windows10) that recognize the 'lepc' tag to use those features, while older systems that don't have any explicit Lepcha support continue to use the 'DFLT' features.

> If I may ask you a follow-up: Am I right in assuming that the flaws in the
> rendering of the second example at the same page (using Noto Sans Lepcha)
> are due to other reasons? Best thanks again.

I'm afraid I am not sufficiently familiar with Lepcha writing to recognize problems in the Noto example; I'd need you to point out exactly what I should be looking at. (It's very possible there are bugs in the Noto font, given how new all this technology is....)

Comment 5

[serkhang]

(In reply to Jonathan Kew (:jfkthame) from comment #4)

Best thanks for your suggestions. Be sure they'll be considered and - hopefully - implemented in the further course of font development.
 
> I'm afraid I am not sufficiently familiar with Lepcha writing to recognize
> problems in the Noto example; I'd need you to point out exactly what I
> should be looking at. (It's very possible there are bugs in the Noto font,
> given how new all this technology is....)

I don't want to hijack your time and experience, however, if you could have a look at the attached text, I would appreciate your comments regarding rendering flaws. They mainly appear with stacked superscripts and the placement of the CONSONANT SIGN KANG in combination with VOWEL SIGN I.

Comment 6

[serkhang]

Attachment: Noto_Comparison_marked.png

Comment 7

[Jonathan Kew [:jfkthame]]

(In reply to serkhang from comment #5)
> (In reply to Jonathan Kew (:jfkthame) from comment #4)
> 
> Best thanks for your suggestions. Be sure they'll be considered and -
> hopefully - implemented in the further course of font development.
>  
> > I'm afraid I am not sufficiently familiar with Lepcha writing to recognize
> > problems in the Noto example; I'd need you to point out exactly what I
> > should be looking at. (It's very possible there are bugs in the Noto font,
> > given how new all this technology is....)
> 
> I don't want to hijack your time and experience, however, if you could have
> a look at the attached text, I would appreciate your comments regarding
> rendering flaws. They mainly appear with stacked superscripts and the
> placement of the CONSONANT SIGN KANG in combination with VOWEL SIGN I.

I haven't completely analyzed what's happening here, but I suspect the font is less than perfect in various ways. I looked at the sequence <U+1C09,U+1C36,U+1C30> (from the word ᰕᰉᰧᰶᰰ), for instance, and AFAICS the font lacks mark-to-mark positioning support for the pair <U+1C36,U+1C30> found here, which is why they end up overprinting each other.

In addition, looking into the Noto Sans Lepcha font, it's clear that it was designed before the Universal Shaping Engine, and (like your MainwaringRong font) it uses hundreds of multi-character ligatures to substitute for the reordering behavior that the shaper now provides. So I think now that "real" Lepcha support exists, it's time for an update to the font.

(If you have time to provide feedback to Google about any rendering problems you find in the Noto font, when used with current Firefox or Chrome, that would be a valuable contribution.)

Comment 8

[serkhang]

@Jonathan Kew: Best thanks for your comments.

I reported the problems with Noto Sans Lepcha at

https://github.com/googlei18n/noto-fonts/issues/395

though this was in connection with text editors. Rendering issues with Chrome using the font Mingzat (SIL) were reported at

https://code.google.com/p/chromium/issues/detail?id=497537

This, however, is likely to be a 'false bug' as in the present case. Removing the lepc tag in Mainwaring Róng resolved issues in Chrome as well. Amazingly, nobody at the Chromium team handling bug reports realized the true nature of the problem.

@Admins: Obviously, rendering issues reported are not due to a bug, quite the contrary. They are the consequence of an avant-garde feature. I would not mind if this thread were closed.

Comment 9

[Mats Palmgren (inactive)]

Sounds like there isn't anything to fix in Firefox here.