1336058 - Remove support for the deprecated STIXGeneral set

Status: RESOLVED FIXED

(Core :: MathML, task, P5)

Description

[Frédéric Wang Nélar]

Quick history:

* First release of STIXGeneral (February 2013).

* STIX 1.1.1 released, with STIXGeneral removed (June 2013). STIX General was still in the Web fonts package for use in browsers (November 2013). However, this soon became obsolete when OpenType MATH support was added to WebKit & Gecko (2014). From the release notes: "stix-web contains a set of webfonts (.woff files) based on the previously released STIXGeneral version 1.1.0 fonts. [...] Web authors who wish to use STIX within a browser [...] should choose to use stix-web [...]"

* Firefox 31 released with OpenType MATH support (July 2014)

* Firefox 41 released, deprecating STIXGeneral (September 2015)

* STIX 2 released (December 2016), deprecating old versions: "Earlier versions of the STIX fonts are deprecated and no longer supported.  We urge all users of the STIX fonts to upgrade to the new versions as soon as possible."

I believe we should remove support for STIX General now. Then after bug 1335124, the unicode table will be the only one using *.properties file. Bug 1007090 could then be fixed by only keeping one hardcoded table for Unicode. This would already greatly simplify the nsMathMLChar code and allow to get rid of XPCOM stuff for loading *.properties files.

Last time I discussed with Karl, the only reason to preserve that support was that macOS ships the STIXGeneral by default. With the release of STIX 2 and the explicit deprecation by the STIX consortium, I think it does not make sense to preserve that support.

Comment 1

[Frédéric Wang Nélar]

Attachment: Patch

This is the minimal change to remove STIXGeneral. As said above, more cleanup can be done after Unicode is the only properties table but this can be done in bug 1007090.

Comment 2

[Frédéric Wang Nélar]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=4f8196061be88effc901197554a32217cdb6eed3

Comment 3

[Karl Tomlinson (:karlt)]

(In reply to Frédéric Wang (:fredw) from comment #0)
> Last time I discussed with Karl, the only reason to preserve that support
> was that macOS ships the STIXGeneral by default. With the release of STIX 2
> and the explicit deprecation by the STIX consortium, I think it does not
> make sense to preserve that support.

There is notable value in having math render without any need to install
additional fonts.

STIX Math is not included in MacOS, even in 10.12
(https://support.apple.com/en-nz/HT206872).

Are there any fonts shipped with MacOS that include a MATH table?

Or do you think that Unicode parts and scaling of regular glyphs are sufficient for most math?

If so, I suspect font.name-list.serif.x-math should specify Apple Symbols or another font that is on MacOS to provide a consistent set of symbols?
Which font would provide Mathematical Alphanumeric Symbols, for example?

> * STIX 2 released (December 2016), deprecating old versions: "Earlier
> versions of the STIX fonts are deprecated and no longer supported.  We urge
> all users of the STIX fonts to upgrade to the new versions as soon as
> possible."

> With the release of STIX 2
> and the explicit deprecation by the STIX consortium, I think it does not
> make sense to preserve that support.

The announcement that STIX are not planning further fixes for the 1.1 series
of fonts doesn't mean that Gecko must stop using them.

> OpenType MATH support was added to
> WebKit & Gecko (2014). From the release notes: "stix-web contains a set of
> webfonts (.woff files) based on the previously released STIXGeneral version
> 1.1.0 fonts. [...] Web authors who wish to use STIX within a browser [...]
> should choose to use stix-web [...]"

Ideally, if web authors are happy with the default font used by a browser,
they should not need to provide any web fonts at all.

> * Firefox 41 released, deprecating STIXGeneral (September 2015)

What in this Firefox version deprecated STIXGeneral?

Was it just a change in the order of font families used for math?

Comment 4

[Frédéric Wang Nélar]

(In reply to Karl Tomlinson (:karlt) from comment #3)
> There is notable value in having math render without any need to install
> additional fonts.
> 
> STIX Math is not included in MacOS, even in 10.12
> (https://support.apple.com/en-nz/HT206872).
> 
> Are there any fonts shipped with MacOS that include a MATH table?
> 
> Or do you think that Unicode parts and scaling of regular glyphs are
> sufficient for most math?
> 
> If so, I suspect font.name-list.serif.x-math should specify Apple Symbols or
> another font that is on MacOS to provide a consistent set of symbols?
> Which font would provide Mathematical Alphanumeric Symbols, for example?

No there is not any pre-installed MATH font and neither STIXGeneral nor Unicode fonts provide acceptable default rendering (the former adds more stretchy/largeop support though but it lacks OpenType MATH parameters, glyph features and so on). Maybe the changes in all.js can be ignored, so that STIX General can still be used for the Unicode-only rendering (although I'm afraid other obsolete STIX fonts will need to be added). The issue has been mentioned for a while to Apple, but I do not know what their internal plans are.

> The announcement that STIX are not planning further fixes for the 1.1 series
> of fonts doesn't mean that Gecko must stop using them.

Sure, on the other hand I do not think Gecko's development should be blocked by Apple's decisions. Note that we already removed support for old fonts in the past (https://developer.mozilla.org/en-US/docs/Mozilla/MathML_Project/FontsMozilla1.8).

> > OpenType MATH support was added to
> > WebKit & Gecko (2014). From the release notes: "stix-web contains a set of
> > webfonts (.woff files) based on the previously released STIXGeneral version
> > 1.1.0 fonts. [...] Web authors who wish to use STIX within a browser [...]
> > should choose to use stix-web [...]"
> 
> Ideally, if web authors are happy with the default font used by a browser,
> they should not need to provide any web fonts at all.

The STIX releases are confusing but that one was specifically for Web fonts. The STIX Consortium provided STIXGeneral web fonts because browsers did not support the MATH table at that time. As I said this point is obsolete and they removed STIXGeneral in follow-up releases anyway.

> > * Firefox 41 released, deprecating STIXGeneral (September 2015)
> 
> What in this Firefox version deprecated STIXGeneral?
> 
> Was it just a change in the order of font families used for math?

Actually, the deprecation happened in Gecko 31 when MATH support was added:
https://groups.google.com/forum/#!searchin/mozilla.dev.tech.mathml/Important$20information$20about$20the$20MathML$20fonts$20in$20Gecko%7Csort:relevance/mozilla.dev.tech.mathml/PlVCil2X598/LfLuZfSVKyYJ

Cleaner math selection mechanism had been added to Firefox 41 (https://developer.mozilla.org/en-US/Firefox/Releases/41#New_default_and_fallback_font_handling). For that purpose the STIXGeneral properties was merged into a single file (but 1007093) but it turned out that this did not work very well (I'm not sure of the details, but NIST people said the rendering was poor until they pre-install Latin Modern Math on their Linux machines). Support for "Standard Symbols L" and "MathJax fonts" was also removed at that time:

https://lists.w3.org/Archives/Public/www-math/2015Sep/0031.html (for some reason, the message is not in the Mozilla archives).

After bug 1335124, STIXGeneral is the only non-MATH font with special support.

At the moment, nsMathMLChar is the most complex part of MathML code and is causing performance issue and blocking future support (see bug 1007090 and bug 1009582). The point of the deprecation was to rely only on OpenType MATH table in the future and hence allow to get much cleaner code, better performance and better support. I think we have to consider whether supporting STIXGeneral for macOS is worth preventing refactoring & improvements in Gecko/MathML. Personally, I think it is not.

Comment 5

[Karl Tomlinson (:karlt)]

Screenshots of the torture test on a Mac with only OS fonts, with and without this patch, may help to make a decision here.

Comment 6

[Karl Tomlinson (:karlt)]

Comment on attachment 8832899 [details] [diff] [review]
Patch

I would like to avoid a significant regression in rendering on Mac when only
OS fonts are installed.  I'd like to see the effect this change would have on
the rendering, but I don't have access to a Mac.

If you are able to attach screenshots, please, then I can consider whether the
change is acceptable.

Comment 7

[Frédéric Wang Nélar]

Perhaps a first not controversial step would be to introduce a preference and counter+deprecated warning, so that at least users can be warned that they should upgrade to STIX Two or more generally follow https://developer.mozilla.org/en-US/docs/Mozilla/MathML_Project/Fonts ; and mac users could maybe report this to Apple. Otherwise we'll never be able to clean up the stretchy operator code.

Comment 8

[Frédéric Wang Nélar]

(In reply to Karl Tomlinson (:karlt) from comment #6)

Comment on attachment 8832899 [details] [diff] [review]
Patch

I would like to avoid a significant regression in rendering on Mac when only
OS fonts are installed. I'd like to see the effect this change would have on
the rendering, but I don't have access to a Mac.

If you are able to attach screenshots, please, then I can consider whether
the
change is acceptable.

Some time ago, I was discussing with Myles Maxfield about "font-family: math" and he pinged his font team to upgrade to STIX Two: https://webkit.slack.com/archives/CTY7Z24RZ/p1583965391109500 ; that does not mean Apple will actually do it, but let's see... I guess my proposal in comment 7 still makes sense in any case.

Comment 9

[Frédéric Wang Nélar]

FYI, STIX Two Math is now available in macOS 13 Ventura beta.

Comment 10

[Frédéric Wang Nélar]

Delan introduced a counter in bug 1630935. I'm checking the telemetry data for OS X and counting the number of doc with the stix general warning with respect to the number of doc with MathML. Below is a summary of the recent metrics:

Beta |mathml|stix |%
115  |13210 |229  |1.73
114  |13880 |442  |3.18
113  |15150 |579  |3.82
112  |13790 |1720 |12.47
111  |14580 |1380 |9.47
110  |20780 |2570 |12.37
109  |15800 |3330 |21.08
108  |14710 |3200 |21.75
107  |14650 |4180 |28.53
106  |14580 |5030 |34.50
105  |14850 |5780 |38.92
104  |15660 |5420 |34.61
103  |16840 |4820 |28.62
102  |20930 |5170 |24.70
101  |17910 |5130 |28.64
100  |17870 |5440 |30.44

Note that macOS 13 Ventura was released with STIX Two (and without STIXGeneral) on 2022/10/24, corresponding to the measurement period of beta 107. After that, the STIXGeneral usage for MathML operator stretching on macOS seems to have go down progressively from 25-35% to 1.73% of doc with MathML.

@jfkthame @emilio : What do you think? Should we consider removing support for the deprecated STIXGeneral set in MathML now? Should we wait more?

Comment 11

[Emilio Cobos Álvarez [:emilio]]

Stephen, do you know what our supported macOS version range is / when will versions previous to macOS 13 be unsupported?

In general, it seems reasonable to do that given it's a percent of all mathML content which is already low compared to total pages. But if we're unsupporting macOS <13 soon enough it might be worth waiting.

Comment 12

[Jonathan Kew [:jfkthame]]

Personally, I'd be a bit hesitant to do this just yet, with macOS 13 still less than a year old (and current Firefox still supporting versions all the way back to 10.15, according to https://www.mozilla.org/en-US/firefox/117.0a1/system-requirements/). But definitely interested to know if we have any clear schedule for unsupporting older versions.

Comment 13

[Stephen A Pohl [:spohl]]

We do not have a general schedule for dropping support for older versions and this is very much driven by number of users, maintenance overhead of supporting old versions and similar parameters. I agree with :jfkthame that versions before macOS 13 are not expected to be dropped soon.

Comment 14

[Frédéric Wang Nélar]

(In reply to Frédéric Wang (:fredw) from comment #10)

Delan introduced a counter in bug 1630935. I'm checking the telemetry data for OS X and counting the number of doc with the stix general warning with respect to the number of doc with MathML. Below is a summary of the recent metrics:

I verified the data for OS X again:

Beta |mathml|stix |1%
120  |14750 |0   |0,00 %
119  |13380 |352 |2,63 %
118  |10500 |548 |5,22 %
117  |10190 |0   |0,00 %
116  |11350 |206 |1,81 %
115  |15440 |455 |2,95 %
114  |14400 |453 |3,15 %
113  |15430 |581 |3,77 %
112  |13850 |1740|12,56 %
111  |14720 |1390|9,44 %

Comment 15

[Frédéric Wang Nélar]

I was trying to check again the public telemetry data, but it seems results for Firefox versions after beta 120 are not available.

From https://firefoxgraphics.github.io/telemetry/#view=mac I see that at least 93% of reported mac versions are 10.13 or higher.

Comment 16

[Frédéric Wang Nélar]

(In reply to Frédéric Wang (:fredw) from comment #15)

From https://firefoxgraphics.github.io/telemetry/#view=mac I see that at least 93% of reported mac versions are 10.13 or higher.

Please disregard this comment, what we need is macOS 13 or higher, but it seems the vast majority of the users are still at a 10.X version.

Comment 17

[Gregory Pappas [:gregp]]

The firefoxgraphics.github.io data includes macOS versions like "10.16", "10.17", etc. Do these exist? I though that the last macOS 10.X was 10.15 Catalina.

perhaps macOS versions 11+ are being erroneously reported as these higher 10.X versions?

Comment 18

[Frédéric Wang Nélar]

@gregp: mmh, that's a good point.

As I see https://github.com/FirefoxGraphics/telemetry/pull/7 was supposed to remove the empty "macOS 10.8 (Mountain Lion) - Breakdown" pie chart and add one for "macOS 10.15 (Catalina) - Breakdown" but it seems this change has not been merged in https://firefoxgraphics.github.io/telemetry/#view=mac ; and consequently it's even more unlikely that we support macOS versions 11+.

Also OSXNameMap, DarwinVersionToOSX and DarwinVersionToOSX in https://github.com/FirefoxGraphics/telemetry/blob/bb0cc53a61a30acb22b3973f32f1721c572d02bc/www/constants.js only seem to handle "10.X" versions here, although I don't get how we get the results 10.16, 10.17, 10.18 or 10.19

Who is maintaining telemetry these days and could fix that issue?

Comment 19

[Gregory Pappas [:gregp]]

Who is maintaining telemetry these days and could fix that issue?

I'm not sure. #gfx:mozilla.org might be a good place to ask.

Comment 20

[Frédéric Wang Nélar]

(In reply to Gregory Pappas [:gregp] from comment #19)

Who is maintaining telemetry these days and could fix that issue?

I'm not sure. #gfx:mozilla.org might be a good place to ask.

Thanks, I tried but the only reply I got so far was "10.16 is 11, etc." which is what you suggested.

In any case, with this interpretation, the current values would be

macOS 15 (Sequoia) - 0.5%
macOS 14 (Sonoma) - 43.1%
macOS 13 (Ventura) - 14.5%
macOS 12 (Monterey) - 12.5%
macOS 11 (Big Sur) - 6.7%
macOS 10.15 (undefined) - 6.7%
macOS 10.13 (High Sierra) - 6.4%
macOS 10.14 (Mojave) - 3.1%
macOS 10.12 (Sierra) - 2.5%
macOS 10.11 (El Capitan) - 2.0%
macOS 10.10 (Yosemite) - 1.2%
macOS 10.9 (Mavericks) - 0.9%

So there is still a significant amount of macos < 13.

Comment 21

[Frédéric Wang Nélar]

Comment from emilio at https://phabricator.services.mozilla.com/D219084#7872847:

I checked the use counters and they hover at around 0.00004%. So I think this is worth trying!

Comment 22

[Frédéric Wang Nélar]

Attachment: Bug 1336058 - Remove support for the deprecated STIXGeneral set. r=emilio

• StaticPrefList.yaml: remove preference, it has been disabled in all platforms since Firefox 134 (bug 1912870) and non-macos platforms since Firefox 83 (bug 1664488).
• nsMathMLChar.cpp: remove dead code.
• semantics-005.html: Use a Web font and a rare stretchy arrow (not in mathfontUnicode.properties) to avoid existing bugs with some local fonts (bugs 1309429, 1648335 and 1931159).

Comment 23

[Frédéric Wang Nélar]

https://treeherder.mozilla.org/jobs?repo=try&landoCommitID=145615

Comment 24

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/6a7e97645deb
https://hg.mozilla.org/integration/autoland/rev/45f48199b916
Remove support for the deprecated STIXGeneral set. r=layout-reviewers,emilio

Comment 25

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/45f48199b916

Comment 26

[Gregory Pappas [:gregp]]

These prefs are removed:

• mathml.stixgeneral_operator_stretching.disabled

Comment 27

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/54396 for changes under testing/web-platform/tests

Comment 28

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot