728866 - support unicode composition and decomposition during harfbuzz shaping

Status: RESOLVED FIXED

(Core :: Layout: Text and Fonts, defect)

Description

[Jonathan Kew [:jfkthame]]

Harfbuzz has added support for (limited/specialized) normalization during shaping, but currently we are only taking partial advantage of this - we get reordering of combining marks into a standard order for display (e.g. see bug 662055), but we don't get canonical (de)composition of precomposed accented characters, as that depends on a couple of new callbacks that we haven't yet implemented.

The main advantage of implementing this is that some fonts - especially older ones - have a number of precomposed forms, but do not have good (or any) dynamic mark positioning code. Hence, if a page includes a decomposed letter+diacritic sequence, the mark positioning may be poor, whereas if we support composition in the shaping process, we can use the better-looking precomposed glyph.

To support this, harfbuzz needs two callbacks, one to implement a single step of decomposition, and one to compose two characters into one. Unfortunately, although our normalization component has the relevant data tables (though somewhat obsolete, see bug 728180), it does not currently provide this level of API, but we can pretty easily expose it.

Comment 1

[Jonathan Kew [:jfkthame]]

Attachment: part 1 - expose low-level character composition/decomposition functions in nsUnicodeNormalizer

For harfbuzz shaping purposes, I don't want the extra indirection of going through an nsIUnicodeNormalizer; the idea here is to expose the low-level data needed in the most direct way possible.

Comment 2

[Jonathan Kew [:jfkthame]]

Attachment: part 2 - connect harfbuzz callbacks to the unicode composition data

Comment 3

[Jonathan Kew [:jfkthame]]

Attachment: part 3 - add support for composition of Hebrew presentation forms

Unfortunately, using standard canonical composition does not give us good rendering of Hebrew with old fonts (such as those from WinXP), because the Hebrew presentation forms (U+FBxx) with dagesh, etc., are excluded from composition. However, it appears that Uniscribe et al use these glyphs if available, and I think we should do the same.

With this patch, I finally get the desired rendering of your testcase from bug 727736 using the WinXP versions of Arial and Courier, for example.

Comment 4

[Simon Montagu :smontagu]

Comment on attachment 598847 [details] [diff] [review]
part 1 - expose low-level character composition/decomposition functions in nsUnicodeNormalizer

Review of attachment 598847 [details] [diff] [review]:
-----------------------------------------------------------------

Hmm, on second thoughts perhaps Decompose should have a different name to clarify that it isn't doing a full decomposition. r=smontagu stands either way.

Comment 5

[Simon Montagu :smontagu]

Comment on attachment 598849 [details] [diff] [review]
part 3 - add support for composition of Hebrew presentation forms

Review of attachment 598849 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good.

Comment 6

[Jonathan Kew [:jfkthame]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/8ee6310cecda
https://hg.mozilla.org/integration/mozilla-inbound/rev/2c3f7f3ace89
https://hg.mozilla.org/integration/mozilla-inbound/rev/969493463a87

Comment 7

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/8ee6310cecda
https://hg.mozilla.org/mozilla-central/rev/2c3f7f3ace89
https://hg.mozilla.org/mozilla-central/rev/969493463a87

Comment 8

[Behdad Esfahbod]

Humm.  Ideally I want to support both Arabic and Hebrew presentation-form shaping in HB.  Lets start thinking about that.

Comment 9

[Behdad Esfahbod]

I added the hebrew decompositions to harfbuzz upstream now.