775060 - Bundle fonts to enable MathML rendering without site-supplied fonts

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: General, defect)

Description

[Henri Sivonen (:hsivonen)]

The MathML implementation in Gecko doesn't work right without suitable fonts. In the past, those fonts haven't been bundled with Firefox due to download size concerns, etc. In the case of Firefox OS, Mozilla is shipping a full OS presumably already with some fonts, so the download size shouldn't be such a concern. Furthermore, it would be really sad if Firefox OS didn't come with sufficient fonts to support all of Gecko's functionality.

Therefore, I suggest bundling fonts that are necessary for MathML rendering with Firefox OS. AFAICT, this would involve bundling the STIX fonts as well as Asana Math and MathJax fonts.
http://www.stixfonts.org/
http://www.ctan.org/tex-archive/fonts/Asana-Math/
(Can't find a non-broken link for the MathJax fonts.)

As far as I can tell, such fonts are available on a "free as in freedom" basis, but some fonts (STIX, Asana) don't fit with Mozilla's currently stated licensing policy. Specifically, https://www.mozilla.org/MPL/license-policy.html doesn't authorize Open Font License 1.1 for Third Party Code. I hope that the policy can be amended to allow bundled fonts that are licensed under Open Font License 1.1. (After all, allowing Open Font License 1.1 is a much better fit to Mozilla than proprietary drivers and it seems the door is already open for those in Firefox OS. And chances are that other font bundling for Unicode coverage on Firefox OS will run into the Open Font License 1.1 sooner or later anyway.)

Comment 1

[Frédéric Wang (:fredw)]

See also bug 295193 for a related issue.

Comment 2

[Gervase Markham [:gerv]]

Firefox OS includes components under a wider range of licenses than Firefox itself. I see no problem with bundling OFL 1.1 fonts with Firefox OS.

Gerv

Comment 3

[Karl Tomlinson (:karlt)]

Note that not all of those fonts are /necessary/.  They provide options.

Either only STIX or only MathJax fonts should be sufficient.

Comment 4

[Frédéric Wang (:fredw)]

Currently, I would suggest to install both STIX and MathJax fonts.

- MathJax fonts are closer to LaTeX fonts (e.g. summation symbol) and improve mathvariant support.

- STIX fonts provide a much larger set of Unicode characters and can stretch braces (see bug 732832 for MathJax fonts).

Comment 5

[Gervase Markham [:gerv]]

If B2G has a size target, I'm sure the person maintaining that will want to have a say in which fonts are bundled. But I'll leave you guys to it :-)

Gerv

Comment 6

[Frédéric Wang (:fredw)]

Any update on this? Who is in charge of selecting fonts to bundle with FirefoxOS? Do they need any help?

At the moment, including the MathJax fonts seem the best option:
- The total size for fonts used for the math text and stretchy operators is only ~200kb
- The license is Apache 2, which is already in Firefox's about:license
- Our MathML code uses them by default since they are based on the classical TeX's Computer Modern fonts.

Here is a UNIX command to get the file from the MathJax's CDN (for convenience, I also attach a zip):

WGET=wget; FONTDIR=MathJaxFonts; CDN=http://cdn.mathjax.org/mathjax/latest/; mkdir -p $FONTDIR; cd $FONTDIR; for font in Main-Regular Main-Bold Main-Italic Size1-Regular Size2-Regular Size3-Regular Size4-Regular AMS-Regular; do $WGET $CDN/fonts/HTML-CSS/TeX/otf/MathJax_$font.otf; done; $WGET $CDN/LICENSE; cd -

At the moment, other fonts like MathJax_Fraktur are used to workaround bug 114365 but I believe that's ok if we don't take them. A more serious problem is bug 732832. However, I submitted a very simple patch on bug 732832 and attached a modified Size4-Regular font that has two additional glyphs for the middle part of horizontal braces. These could be included in FirefoxOS too.

Comment 7

[Frédéric Wang (:fredw)]

Attachment: MathJax Fonts used for the MathML text and stretchy operators.

Comment 8

[Frédéric Wang (:fredw)]

Another advantage of installing the MathJax fonts on FirefoxOS is that it will speed up the rendering of Websites that use MathJax's HTML-CSS output (like it will probably be the case on Wikipedia soon). Essentially, this will avoid downloading the Web fonts when they are not in the browser cache. In that case, it may be worth installing the whole set of fonts instead (~450kb). In the previous command, use

AMS-Regular Caligraphic-Bold Caligraphic-Regular Fraktur-Bold Fraktur-Regular Main-Bold Main-Italic Main-Regular Math-BoldItalic Math-Italic Math-Regular SansSerif-Bold SansSerif-Italic SansSerif-Regular Script-Regular Size1-Regular Size2-Regular Size3-Regular Size4-Regular Typewriter-Regular

Comment 9

[Frédéric Wang (:fredw)]

Let's wait that we have support for Open Type MATH fonts before doing that.

Comment 10

[Frédéric Wang (:fredw)]

Support for Open Type MATH fonts landed in Nightly and will be available in Gecko 31. The possible candidates for free fonts are shown on the MathML torture test:

http://www.maths-informatique-jeux.com/ulule/mathml_torture_test/

We only need one of them and each one is about 400-800kb.

Asana Math => http://www.ctan.org/tex-archive/fonts/Asana-Math/ (OFL)
Latin Modern Math, TeX Gyre Pagella Math, TeX Gyre Termes Math => http://www.gust.org.pl/gustnews/news012013en (GUST license http://www.gust.org.pl/projects/e-foundry/licenses)
Neo Euler => https://github.com/khaledhosny/euler-otf (OFL)
STIX Math => http://sourceforge.net/projects/stixfonts/files/Current%20Release/STIXv1.1.1-word.zip/download (OFL)
XITS => https://github.com/khaledhosny/xits-math (OFL)

Comment 11

[Frédéric Wang (:fredw)]

Attachment: Add Latin Modern Math

This is not tested.

It seems that the current fonts.mk only supports compression of ttf files. So I've compressed the latin modern math font into woff directly since this is really only used for MathML.

Comment 12

[Jonathan Kew [:jfkthame]]

(In reply to Frédéric Wang (:fredw) |away 27/04 to 06/05 from comment #11)
> Created attachment 8412918 [details] [review]
> Add Latin Modern Math

I'm not sure about the choice of Latin Modern as the math font - as a "Modern" face (actually quite old-fashioned nowadays!) it'll look very different from the default text fonts on Firefox OS. I think we should look at each of the options used for fragments of MathML in the context of a page that uses Fira Sans and Charis SIL for text, and see how well they do (or don't) harmonize.

And we should view them on a FxOS device rather than (or in addition to) a desktop browser, as the FreeType font rendering we get there may look significantly different.

> 
> This is not tested.
> 
> It seems that the current fonts.mk only supports compression of ttf files.

This could easily be extended to support the .otf extension in a similar way; it just wasn't needed yet.

> So I've compressed the latin modern math font into woff directly since this
> is really only used for MathML.

I'd prefer us to leave this as a build-time option, since using .woff files does carry a runtime cost (time to decompress, and perhaps more importantly, RAM usage for the decompressed font data). In cases where the ROM image size is not a critical constraint, it's better to leave fonts uncompressed so that FreeType can simply memory-map the files.

Comment 13

[Jonathan Kew [:jfkthame]]

Comment on attachment 8412918 [details] [review]
Add Latin Modern Math

Clearing r? here, until we've reviewed how the various available faces look in context on a device.

Comment 14

[Frédéric Wang (:fredw)]

(In reply to Jonathan Kew (:jfkthame) from comment #12)
> I'm not sure about the choice of Latin Modern as the math font - as a
> "Modern" face (actually quite old-fashioned nowadays!) it'll look very
> different from the default text fonts on Firefox OS. I think we should look
> at each of the options used for fragments of MathML in the context of a page
> that uses Fira Sans and Charis SIL for text, and see how well they do (or
> don't) harmonize.
> 
> And we should view them on a FxOS device rather than (or in addition to) a
> desktop browser, as the FreeType font rendering we get there may look
> significantly different.

Well, I just chose it as the default because in my experience math people seem to prefer the old Computer Modern style that is used as a default by LaTeX... (for example I've seen many people saying that the MathJax font looks much better than the STIX font while the former is just an OpenType font generated from Computer Modern using an autotracer and the latter is supposed to have been made by professional font designers...)

The good news with the recent MATH support and the publication of the spec is that we are likely to have more choices now... It seems to me that the ideal would be to create a "Fira Sans Math" that renders well with "Fira Sans".

(Note that we will really be able to see the style of the MATH fonts - especially math italic variables - after bug 930504, but that bug itself depends on adding MATH fonts to mobile devices)

> This could easily be extended to support the .otf extension in a similar
> way; it just wasn't needed yet.
> 
> I'd prefer us to leave this as a build-time option, since using .woff files
> does carry a runtime cost (time to decompress, and perhaps more importantly,
> RAM usage for the decompressed font data). In cases where the ROM image size
> is not a critical constraint, it's better to leave fonts uncompressed so
> that FreeType can simply memory-map the files.

OK, I can probably try to work on that. It's just that it was not handled on the moztt repository, so I didn't try to look into it further.

Comment 15

[Jonathan Kew [:jfkthame]]

(In reply to Frédéric Wang (:fredw) |away 27/04 to 06/05 from comment #14)

> Well, I just chose it as the default because in my experience math people
> seem to prefer the old Computer Modern style that is used as a default by
> LaTeX...

> It seems to me that the
> ideal would be to create a "Fira Sans Math" that renders well with "Fira
> Sans".

Yes, that would be awesome, but obviously it'll be some time before we have such a thing available.

In the meantime, I'd like to at least look at how the size and weight of the various math fonts fit in with Fira text, particularly when used for inline math; for displayed equations, it doesn't seem so important to try and match the feel of the text as they stand apart anyway.

Comment 16

[Jonathan Kew [:jfkthame]]

I'll attach a series of screenshots from my Peak device showing simple fragments of MathML within a paragraph of text, for several of the possible math fonts.

The rendering in these examples is far from perfect at present - in particular, we have operator misalignment and poorly rendered parens due to font inflation (bug 1002526), and in some cases the rules for the fraction bar or the top of the large radical sign are not rendered, presumably due to pixel-rounding fail.

Still, this should give some idea of how the fonts look in conjunction with Fira Sans on a device.

Comment 17

[Jonathan Kew [:jfkthame]]

Attachment: screenshot using Asana on B2G

Comment 18

[Jonathan Kew [:jfkthame]]

Attachment: screenshot using Latin Modern on B2G

Comment 19

[Jonathan Kew [:jfkthame]]

Attachment: screenshot using STIX on B2G

Comment 20

[Jonathan Kew [:jfkthame]]

Attachment: screenshot using Termes on B2G

Comment 21

[Jonathan Kew [:jfkthame]]

Attachment: screenshot using Pagella on B2G

Comment 22

[Frédéric Wang (:fredw)]

(In reply to Jonathan Kew (:jfkthame) from comment #16)
> I'll attach a series of screenshots from my Peak device showing simple
> fragments of MathML within a paragraph of text, for several of the possible
> math fonts.
> 
> The rendering in these examples is far from perfect at present - in
> particular, we have operator misalignment and poorly rendered parens due to
> font inflation (bug 1002526), and in some cases the rules for the fraction
> bar or the top of the large radical sign are not rendered, presumably due to
> pixel-rounding fail.
> 
> Still, this should give some idea of how the fonts look in conjunction with
> Fira Sans on a device.

The MathML code uses font metrics for many parameters and they were not updated at all in bug 627842. So at the moment, the rendering is just wrong with font inflation. That should not be hard to fix, we "just" need to pass the inflation in every places. For example for the stretchy char, I've done a quick patch in bug 1002526 to get the right size. For the alignment of operators along the x-axis, nsMathMLmoFrame::Stretch should be updated so that the axis height is correctly computed (passing nsLayoutUtils::FontSizeInflationFor(this) as the third arg of nsLayoutUtils::GetFontMetricsForFrame will do). I'm not sure about the rulethickness, normally there is some stuff to force at least 1px. Also, we'll probably want to take the patches from bug 961365 to use the constants from the MATH table.

Comment 23

[Frédéric Wang (:fredw)]

The rendering of operators and the alignment should be better with the patches of bug 1002526. I'm not sure, but the size of scripts seems incorrect. I wonder if the font inflation confuses the CSS scriptsize computation.

Comment 24

[Frédéric Wang (:fredw)]

(In reply to Jonathan Kew (:jfkthame) from comment #16)
> I'll attach a series of screenshots from my Peak device showing simple
> fragments of MathML within a paragraph of text, for several of the possible
> math fonts.
> 
> The rendering in these examples is far from perfect at present - in
> particular, we have operator misalignment and poorly rendered parens due to
> font inflation (bug 1002526), and in some cases the rules for the fraction
> bar or the top of the large radical sign are not rendered, presumably due to
> pixel-rounding fail.
> 
> Still, this should give some idea of how the fonts look in conjunction with
> Fira Sans on a device.

FYI, I've created a GitHub repository with the 12 known OpenType MATH fonts (woff, css & test):

https://github.com/fred-wang/MathFonts
http://fred-wang.github.io/MathFonts/

Comment 25

[Raniere Silva]

I'll attach a series of screenshots from my Flatfish device (running gecko 33.0a1 with Frédéric's patch) showing simple fragments of MathML within a paragraph of text, for several of the possible math fonts, from http://fred-wang.github.io/MathFonts/.

Comment 26

[Raniere Silva]

Attachment: screenshot of flatfish using default font

Comment 27

[Raniere Silva]

Attachment: screenshot of flatfish using asana

Comment 28

[Raniere Silva]

Attachment: screenshot of flatfish using latin modern

Comment 29

[Raniere Silva]

Attachment: screenshot of flatfish using stix

Comment 30

[Raniere Silva]

Attachment: screenshot of flatfish using pagella

Comment 31

[Raniere Silva]

Attachment: screenshot of flatfish using termes

Comment 32

[Frédéric Wang (:fredw)]

OK, I've updated the GitHub pull request to use OTF files when MOZ_PRODUCT_COMPRESS_FONTS is unset. Tell me if you get better result with that.

Comment 33

[Frédéric Wang (:fredw)]

Attachment: Test Latin Modern Math

This testcase will draw a small and large sigma sign if Latin Modern Math fonts are found and a small and large black square otherwise.

Comment 34

[Raniere Silva]

Attachment: Screenshot of "Test Latin Modern Math" with patch apply (a4e6a45)

> This testcase will draw a small and large sigma sign if Latin Modern Math fonts are found and a small and large black square otherwise.

At the screenshot you can see a small sigma sign and a large black square.

Comment 35

[Frédéric Wang (:fredw)]

This is going even more important now that Wikipedia has a MathML mode (there are concerns to use big WOFF fonts on the mobile version of the Website, for which most users don't have local MATH fonts). Perhaps we can take that without waiting for bug 1002526, which does not seem to affect Flatfish. At least that will avoid missing glyphs for exotic math characters.

I've just updated the pull request to use the latest version of Latin Modern Math.

@Jonathan: did you have a chance to check comment 33 (font not used for nsMathMLChars).
@Raniere: actually, do you still have the bug?

Comment 36

[Raniere Silva]

Hi Frédéric,

I will rebuild Firefox OS with this patch this weekend and back with new information.

Comment 37

[Frédéric Wang (:fredw)]

Thanks Raniere, you might want to test on various FirefoxOS devices with the patches of bug 1002526 applied, to check if concerns of comment 16 are addressed.

Comment 38

[Raniere Silva]

Attachment: screenshot of hamachi with patches from 1002526 using default font

Frédéric,

this is the screenshot of my hamachi. When I upgrade my flatfish I will add a screenshot.

Comment 39

[Frédéric Wang (:fredw)]

Comment on attachment 8511557 [details]
screenshot of hamachi with patches from 1002526 using default font

Besides the missing top bar of square root mentioned by Jonathan, it seems that the font inflation applies badly to <mtable> elements.

Comment 40

[Raniere Silva]

I need to wait Bug 1087096 be resolved to test with flatfish.

Comment 41

[Frédéric Wang (:fredw)]

@Raniere: can you try again now that bug 1002526 is fixed? Also it seems that there is a patch on bug 1087096 to fix the build failure.

Comment 42

[Raniere Silva]

Attachment: screenshot of hamachi after 1002526 merged using default font

Looks OK to me. Let me know if you want a screenshot of some part.

Comment 43

[Frédéric Wang (:fredw)]

My current position regarding the font for MathML is:

1) Use "Latin Modern Math" by default as it has the "Computer Modern" style that TeX people are used to (cf in particular the feedback from Wikipedia users, who want style consistent with the PNG mode). We should probably wait that the GUST e-foundry group releases it under the more standard license OFL license, though.

2) Bundle "STIX Math" too as it has the largest unicode coverage for math & tech symbols. Also, the Arabic/RTL features from XITS could be integrated in that font into the future. We should probably wait that version 2 is released, though.

3) In the long term, use a hypothetic "Fira Math" font that has style consistent with the default Firefox OS font. See https://github.com/mozilla/Fira/issues/79 and https://github.com/unified-font-object/ufo-spec/issues/19.

@Jonathan: what do you think about this plan?

Comment 44

[Frédéric Wang (:fredw)]

I also wonder whether the screenshot (attachment 8442056 [details]) from the test case (attachment 8441872 [details]) is still valid? If so, that sounds like another bug to fix...

Comment 45

[John Daggett (:jtd)]

(In reply to Frédéric Wang (:fredw) from comment #43)
> My current position regarding the font for MathML is:
> 
> 1) Use "Latin Modern Math" by default as it has the "Computer Modern" style
> that TeX people are used to (cf in particular the feedback from Wikipedia
> users, who want style consistent with the PNG mode). We should probably wait
> that the GUST e-foundry group releases it under the more standard license
> OFL license, though.
> 
> 2) Bundle "STIX Math" too as it has the largest unicode coverage for math &
> tech symbols. Also, the Arabic/RTL features from XITS could be integrated in
> that font into the future. We should probably wait that version 2 is
> released, though.
> 
> 3) In the long term, use a hypothetic "Fira Math" font that has style
> consistent with the default Firefox OS font. See
> https://github.com/mozilla/Fira/issues/79 and
> https://github.com/unified-font-object/ufo-spec/issues/19.
> 
> @Jonathan: what do you think about this plan?

Frédéric, could you details the specifics of how much data this is? Is there a reason to bundle STIX fonts under OSX, which already ships with them?

Comment 46

[Jonathan Kew [:jfkthame]]

(In reply to John Daggett (:jtd) from comment #45)
> Is there
> a reason to bundle STIX fonts under OSX, which already ships with them?

This bug is about FirefoxOS.

(In reply to Frédéric Wang (:fredw) from comment #43)
> My current position regarding the font for MathML is:
> 
> 1) Use "Latin Modern Math" by default...
> 2) Bundle "STIX Math" too...

This seems fine to me in principle; my only concern is whether bundling both of them will add too much size. But I don't know what the actual figures would be.

> 3) In the long term, use a hypothetic "Fira Math" font

Maybe, though it's not entirely clear to me what this would look like.

Comment 47

[Frédéric Wang (:fredw)]

(In reply to John Daggett (:jtd) from comment #45)
> Is there a reason to bundle STIX fonts under OSX, which already ships with them?

As Jonathan said, this bug is for Firefox OS. Ideally, I'd like other OS vendors to pre-install math fonts. For OSX, the old "STIX General" set is available and provides good unicode coverage as well as support for strechy operators in WebKit & Gecko (via private tables bundled into the browser). However, both of them now have support for stretchy operators via the MATH table, so I hope Apple will upgrade to STIX 2 when it is released and/or pre-install another font with a MATH table. If you have an Apple developer account, these are problems 16841023 and 17021145. See also bug 1007090 for a plan to remove our private tables where bundling a math font might be an option.

(In reply to John Daggett (:jtd) from comment #45)
> Frédéric, could you details the specifics of how much data this is?

(In reply to Jonathan Kew (:jfkthame) from comment #46)
> This seems fine to me in principle; my only concern is whether bundling both of them will add too much size. But I don't know what the actual figures would be.

I think at least bundling "Latin Modern Math" would be enough to have the "Computer Modern" style and symbols used in standard LaTeX packages. Adding "STIX Math" would be nice to provide larger unicode coverage (and maybe Arabic/RTL features in the future, although these could be added to Latin Modern too...), but that's probably not as important. Other options are using WOFF fonts to find a trade-off between bundle size / decompression cost ; or to create a subset font with only core symbols/features preserved (e.g. https://github.com/fred-wang/MOZTIX). Here are the sizes I get with the latest Latin Modern Math release and STIX2 beta:

fred@debian:~/math-fonts$ du -sh 
720K	latinmodern-math.otf
476K	latinmodern-math.woff
384K	latinmodern-math.woff2
608K	STIXMath_120.otf
448K	STIXMath_120.woff
388K	STIXMath_120.woff2

Comment 48

[James Kitchener (:jkitch)]

Bug 1231701 solved the problem I had with bundling reliable fallback fonts.  It should be just be a matter of dropping a ttf/otf font into the same directory and adding the font name to the about:config list of fallback fonts.

It can't be a WOFF font - these are loaded asynchronously, so may not load until after reflow/rendering has been performed.  (A second pass will be performed if the font is explicitly mentioned by CSS, but not for the fallback list created in nsMathMLChar).

I'm not going to have the opportunity to work on this over the next few weeks.

Comment 49

[Jonathan Kew [:jfkthame]]

(In reply to James Kitchener (:jkitch) from comment #48)
> Bug 1231701 solved the problem I had with bundling reliable fallback fonts. 
> It should be just be a matter of dropping a ttf/otf font into the same
> directory and adding the font name to the about:config list of fallback
> fonts.

We need to clarify exactly what platform(s) are being targeted here. This bug is (was?) about Firefox OS, whereas bug 1231701 involved enabling support for bundled fonts on Linux and Windows. Are you assuming it should be broadened to cover all platforms?

Comment 50

[Bill Gianopoulos [:WG9s]]

(In reply to Jonathan Kew (:jfkthame) from comment #49)
> (In reply to James Kitchener (:jkitch) from comment #48)
> > Bug 1231701 solved the problem I had with bundling reliable fallback fonts. 
> > It should be just be a matter of dropping a ttf/otf font into the same
> > directory and adding the font name to the about:config list of fallback
> > fonts.
> 
> We need to clarify exactly what platform(s) are being targeted here. This
> bug is (was?) about Firefox OS, whereas bug 1231701 involved enabling
> support for bundled fonts on Linux and Windows. Are you assuming it should
> be broadened to cover all platforms?

It is a problem on Android, where you can not install fonts without rooting the device.  FOr other platforms, if you cant figure out how to install fonts perhaps an OS documentation or users should learn how their OS works issue.

Comment 51

[Frédéric Wang (:fredw)]

Comment on attachment 8412918 [details] [review]
Add Latin Modern Math

Given Firefox OS status, I don't think this is a priority so I'm closing this pull request.

Comment 52

[BMO Automation]

Firefox OS is not being worked on