722600 - devanagari woff font rendering correctly in windows7 , Vista but fails in Windows XP ,Linux and MacOSX

Status: RESOLVED WORKSFORME

(Firefox :: General, defect)

Description

[Vivek Iyyer]

Attachment: shiksha.png

User Agent: Mozilla/5.0 (Windows NT 5.1; rv:9.0.1) Gecko/20100101 Firefox/9.0.1
Build ID: 20111220165912

Steps to reproduce:

In developing the site http://peterffreund.com/shiksha/shiksha_praja.html, created a woff font from a open type font. 


Actual results:

This font renders perfectly in Windows 7 and Windows Vista. I have attached how it looks in windows XP where the combination rules of the consonants have not been followed and the font is rendered in a way that all the character without a vowel is fully rendered with virama mark (http://en.wikipedia.org/wiki/Virama) below it.


Expected results:

The combination rules would force the combining consonants to be connected in a way the half the character is rendered.
I will attach the correct rendering in another post as I can attach only one file with the bug report.

Comment 1

[Vivek Iyyer]

Attachment: Correct rendering of the devanagari font

Comment 2

[Peter Freund]

Attachment: Incorrect rendering of devanagari font on Mac OSX

Comment 3

[Jonathan Kew [:jfkthame]]

(In reply to Vivek Iyyer from comment #0)
> This font renders perfectly in Windows 7 and Windows Vista. I have attached
> how it looks in windows XP where the combination rules of the consonants
> have not been followed and the font is rendered in a way that all the
> character without a vowel is fully rendered with virama mark
> (http://en.wikipedia.org/wiki/Virama) below it.

I presume you see the same problems if you install the original OpenType font directly in WinXP and use it in standard applications there. The reason is that the GSUB/GPOS tables only support the 'dev2' script tag, which is the "revised" Devanagari shaping implementation that was introduced in Vista. For the font to work under the XP version of Uniscribe, you need to support the older 'deva' script tag in addition to 'dev2', possibly with slightly different features/lookups. See the Microsoft Typography site for the "deprecated" version of the Devanagari spec, which describes the older WinXP implementation.

Comment 4

[Jonathan Kew [:jfkthame]]

(In reply to Peter Freund from comment #2)
> Created attachment 592970 [details]
> Incorrect rendering of devanagari font on Mac OSX

I'm surprised the font is used at all on Mac OS X, as it does not include AAT tables (in particular, the 'morx' table) needed for complex-script shaping on this platform. What version of the browser and of OS X are you using? When I view the URL here, the downloaded font gets ignored and the standard Devanagari MT font is used instead.

Comment 5

[Peter Freund]

We are familiar with the need to support both the older 'deva' and the newer 'deva2' script tag in order for a font to work both in XP and later systems. Our font does have both. Our font works fine in Open Office in both XP and Vista and in Linux. However the .woff version of the font does not work properly in Firefox on either XP or Linux. We concluded therefore, that there was a problem with Firefox's treatment of webfonts. From our understanding, the only difference between the .ttf version of the opentype font, and the .woff version of the same font, is that the tables have been compressed. Therefore the behavior of the .woff font in Firefox should be the same as the .ttf font in Open Office. Why isn't it?

Comment 6

[Peter Freund]

(In reply to Jonathan Kew (:jfkthame) from comment #3)
> (In reply to Vivek Iyyer from comment #0)
> > This font renders perfectly in Windows 7 and Windows Vista. I have attached
> > how it looks in windows XP where the combination rules of the consonants
> > have not been followed and the font is rendered in a way that all the
> > character without a vowel is fully rendered with virama mark
> > (http://en.wikipedia.org/wiki/Virama) below it.
> 
> I presume you see the same problems if you install the original OpenType
> font directly in WinXP and use it in standard applications there. The reason
> is that the GSUB/GPOS tables only support the 'dev2' script tag, which is
> the "revised" Devanagari shaping implementation that was introduced in
> Vista. For the font to work under the XP version of Uniscribe, you need to
> support the older 'deva' script tag in addition to 'dev2', possibly with
> slightly different features/lookups. See the Microsoft Typography site for
> the "deprecated" version of the Devanagari spec, which describes the older
> WinXP implementation.
We are familiar with the need to support both the older 'deva' and the newer 'deva2' script tag in order for a font to work both in XP and later systems. Our font does have both. Our font works fine in Open Office in both XP and Vista and in Linux. However the .woff version of the font does not work properly in Firefox on either XP or Linux. We concluded therefore, that there was a problem with Firefox's treatment of webfonts. From our understanding, the only difference between the .ttf version of the opentype font, and the .woff version of the same font, is that the tables have been compressed. Therefore the behavior of the .woff font in Firefox should be the same as the .ttf font in Open Office. Why isn't it?

Comment 7

[Jonathan Kew [:jfkthame]]

(In reply to Peter Freund from comment #6)
> We are familiar with the need to support both the older 'deva' and the newer
> 'deva2' script tag in order for a font to work both in XP and later systems.
> Our font does have both.

Exactly how was your praja.woff file created (with what tools/workflow)? I downloaded and examined the praja.woff font, and it definitely does _not_ contain the 'deva' tag, only 'dev2'. So I'm guessing that either the WOFF file was created from a different (older?) version of the TTF file that lacked 'deva' support, or you used a WOFF-creation tool that decided to "optimize" the font by removing certain features.

Comment 8

[Jonathan Kew [:jfkthame]]

For reference, the GSUB table from praja.woff, when dumped using ttx, begins like this - note that there is just a single entry in the ScriptList table:

  <GSUB>
    <Version value="1.0"/>
    <ScriptList>
      <!-- ScriptCount=1 -->
      <ScriptRecord index="0">
        <ScriptTag value="dev2"/>
        <Script>
          <DefaultLangSys>
            <ReqFeatureIndex value="65535"/>
            <!-- FeatureCount=11 -->
            <FeatureIndex index="0" value="0"/>
            <FeatureIndex index="1" value="1"/>
            <FeatureIndex index="2" value="2"/>
            <FeatureIndex index="3" value="3"/>
            <FeatureIndex index="4" value="4"/>
            <FeatureIndex index="5" value="6"/>
            <FeatureIndex index="6" value="5"/>
            <FeatureIndex index="7" value="8"/>
            <FeatureIndex index="8" value="7"/>
            <FeatureIndex index="9" value="9"/>
            <FeatureIndex index="10" value="10"/>
          </DefaultLangSys>
          <!-- LangSysCount=0 -->
        </Script>
      </ScriptRecord>
    </ScriptList>
    <FeatureList>
      <!-- FeatureCount=11 -->
      <FeatureRecord index="0">
        <FeatureTag value="abvs"/>
        <Feature>
          <!-- LookupCount=4 -->
          <LookupListIndex index="0" value="23"/>
          <LookupListIndex index="1" value="24"/>
          <LookupListIndex index="2" value="25"/>
          <LookupListIndex index="3" value="26"/>
        </Feature>
      </FeatureRecord>
      ...etc...

Comment 9

[Peter Freund]

(In reply to Jonathan Kew (:jfkthame) from comment #7)
> (In reply to Peter Freund from comment #6)
> > We are familiar with the need to support both the older 'deva' and the newer
> > 'deva2' script tag in order for a font to work both in XP and later systems.
> > Our font does have both.
> 
> Exactly how was your praja.woff file created (with what tools/workflow)? I
> downloaded and examined the praja.woff font, and it definitely does _not_
> contain the 'deva' tag, only 'dev2'. So I'm guessing that either the WOFF
> file was created from a different (older?) version of the TTF file that
> lacked 'deva' support, or you used a WOFF-creation tool that decided to
> "optimize" the font by removing certain features.

It seems that when we changed from FontLab and VOLT, to FontForge, the deva tagged script was stripped out.  We're still trying to figure out why Fontforge did that, but for sure everything works fine on an older pre-FontForge version of the font.Thanks.

Comment 10

[Peter Freund]

(In reply to Jonathan Kew (:jfkthame) from comment #4)
> (In reply to Peter Freund from comment #2)
> > Created attachment 592970 [details]
> > Incorrect rendering of devanagari font on Mac OSX
> 
> I'm surprised the font is used at all on Mac OS X, as it does not include
> AAT tables (in particular, the 'morx' table) needed for complex-script
> shaping on this platform. What version of the browser and of OS X are you
> using? When I view the URL here, the downloaded font gets ignored and the
> standard Devanagari MT font is used instead.

The hype of universality of webfonts, and in particular, the possibility that WOFF fonts could allow us to take control of the appearance of Sanskrit fonts on a web page has, alas, deluded us. We see that the browser is not in fact taking over rendering responsibilities, and that these responsibilities rest with the operating system, and hence the divide between  Windows and Linux on one side, and Apple on the other is not at all bridged by webfonts. Support for creating the necessary AAT tables for South Asian scripts is obscure to say the least. What programs are available that can do this? We haven't been able to figure out how to make Fontforge generate fonts that render on a Mac, even when we start with DevanagariMT which definitely has all the necessary tables! We understand that this is not a Firefox bug, but we don't yet see a way forward to make webfonts work for us. Can you please give us some direction?

Comment 11

[Jonathan Kew [:jfkthame]]

Unfortunately, I don't have a simple answer to offer here. Webfonts are great - as far as they go. They allow you to deliver a specific font to all users, so that they can see the same glyphs, regardless of what's available on their particular platform. But for scripts that require "complex shaping", such as Devanagari, you still face the issue that there's not a single universally-supported mechanism to implement this.

If you could use @font-face to provide a Devanagari font that includes _both_ OpenType and AAT tables, and supports both the 'deva' and 'dev2' OpenType script tags, you'd be in the best currently-possible position - though this still wouldn't work in Firefox on Android devices where there's no Indic-script support (and it wouldn't work in the native Android browser on many devices, either).

However, creating the necessary AAT tables for Indic scripts is difficult: the Apple tools and docs are obscure, and haven't been updated in years - last I checked, the Apple Font Tools only existed in PPC form, which means they won't run at all on non-Rosetta systems - and I don't think FontForge has adequate AAT support for this. A number of years ago, when I was creating AAT fonts, I built my own tools based on the TrueType GX/AAT spec that Apple provided, as there wasn't anything suitable available, but obviously that makes for a major barrier to entry. I'd guess that the (few) other third parties such as Xenotype who have created complex AAT fonts may well have rolled their own tools, too.

It's our intention to support Devanagari (and other Indic-script) OpenType fonts in Firefox on all platforms, regardless of the underlying OS support, so that you'd be able to rely on a single implementation working for all users (whether on Windows, Linux, Android, or Mac OS X), but sadly this is not yet a reality - and I can't promise a timescale for when it will happen.

Comment 12

[Ioana (away)]

WFM on the 12/09 Nightly:
Mozilla/5.0 (X11; Linux i686; rv:28.0) Gecko/20100101 Firefox/28.0

Vivek, can you still reproduce this issue?

Comment 13

[Jonathan Kew [:jfkthame]]

We now have pretty complete Indic-script support in harfbuzz (there have been numerous updates since this was filed), and this example should render correctly in all current versions of Firefox across all platforms.

Leaving this open for Vivek to confirm, but AFAIK we can then resolve this issue.

Comment 14

[Vivek Iyyer]

Thanks .. it works perfectly now. Thanks. Hoping Peter who has a Mac will submit a screenshot of the page.

Comment 15

[Vivek Iyyer]

Attachment: Correct rendering screenshot of the shamana shiksha page on peterffreund.com

Fixed. This is the correct rendering in firefox.