297465 - (mglyph) Add support for mglyph

Status: NEW

(Core :: MathML, defect)

Description

[Martin Mokrejs]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b2) Gecko/20050612
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b2) Gecko/20050612

See this testcase too:
http://www.w3.org/Math/testsuite/testsuite/Presentation/TokenElements/mglyph/rec-mglyph2.xml

Reproducible: Always

Comment 1

[rbs]

mglyph is not yet implemented.

Comment 2

[Gervase Markham [:gerv]]

This is an automated message, with ID "auto-resolve01".

This bug has had no comments for a long time. Statistically, we have found that
bug reports that have not been confirmed by a second user after three months are
highly unlikely to be the source of a fix to the code.

While your input is very important to us, our resources are limited and so we
are asking for your help in focussing our efforts. If you can still reproduce
this problem in the latest version of the product (see below for how to obtain a
copy) or, for feature requests, if it's not present in the latest version and
you still believe we should implement it, please visit the URL of this bug
(given at the top of this mail) and add a comment to that effect, giving more
reproduction information if you have it.

If it is not a problem any longer, you need take no action. If this bug is not
changed in any way in the next two weeks, it will be automatically resolved.
Thank you for your help in this matter.

The latest beta releases can be obtained from:
Firefox:     http://www.mozilla.org/projects/firefox/
Thunderbird: http://www.mozilla.org/products/thunderbird/releases/1.5beta1.html
Seamonkey:   http://www.mozilla.org/projects/seamonkey/

Comment 3

[Gervase Markham [:gerv]]

This bug has been automatically resolved after a period of inactivity (see above
comment). If anyone thinks this is incorrect, they should feel free to reopen it.

Comment 4

[Martin Mokrejs]

Still wrong with:
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a1) Gecko/20051122 SeaMonkey/1.5a

./configure --disable-optimize --enable-debug='-g3 -O0' --enable-debug-modules=all --enable-debugger-info-modules --enable-detect-webshell-leaks --enable-svg --enable-svg-renderer-libart --enable-image-decoders=all --with-qtdir=/usr/qt/3 --enable-application=suite --disable-freetype2 --enable-default-toolkit=gtk2 --enable-xft --disable-gssapi

Comment 5

[Martin Mokrejs]

Still an issue with current trunk build.

Comment 6

[Martin Mokrejs]

Still an issue with current cvs HEAD build:

./configure --disable-optimize --enable-debug='-g3 -O0 -ggdb' --enable-debug-modules=all --enable-debugger-info-modules --enable-detect-webshell-leaks --disable-svg --enable-image-decoders=all --with-qtdir=/usr/qt/3 --enable-application=suite --disable-freetype2 --enable-jprof --enable-default-toolkit=gtk2 --enable-xft --disable-gssapi

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a2pre) Gecko/20070126 SeaMonkey/1.5a

Comment 7

[Frédéric Wang (:fredw)]

It seems that the former use of <mglyph/> (access to non-standard fonts) will be deprecated in MathML 3 and replaced by something similar to <img/>

http://www.w3.org/TR/2009/WD-MathML3-20090604/chapter3.html#id.3.2.9.4

Comment 8

[Frédéric Wang (:fredw)]

Attachment: Testcase (with respect to MathML 3)

Comment 9

[Frédéric Wang (:fredw)]

If someone wants to work on this, the idea is to create a new class nsMathMLmglyphFrame that inherits from nsMathMLContainerFrame. Then do as for nsSVGImageFrame (or nsImageBoxFrame, nsBulletFrame...): create a class nsMathMLmglyphListener that inherits from nsStubImageDecoderObserver. Use this class in the implementation of nsMathMLmglyphFrame to handle the image.
Note however that mglyph is a bit different from the other MathML elements: it is similar to a character and can only be child of a token element. This means that additional work will probably be needed.

Comment 10

[Frédéric Wang (:fredw)]

Some reftests based on those from the MathML testsuite:

http://devel.mathjax.org/testing/testsuite/MathMLToDisplay/Presentation/TokenElements/mglyph/

(use nativeMathML=true to see Firefox's rendering instead of MathJax's one: http://devel.mathjax.org/testing/testsuite/MathMLToDisplay/Presentation/TokenElements/mglyph/mglyph3-ref.html?nativeMathML=true)

Comment 11

[Frédéric Wang (:fredw)]

(In reply to Frédéric Wang from comment #9)
> If someone wants to work on this, the idea is to create a new class
> nsMathMLmglyphFrame that inherits from nsMathMLContainerFrame. Then do as
> for nsSVGImageFrame (or nsImageBoxFrame, nsBulletFrame...): create a class
> nsMathMLmglyphListener that inherits from nsStubImageDecoderObserver. Use
> this class in the implementation of nsMathMLmglyphFrame to handle the image.
> Note however that mglyph is a bit different from the other MathML elements:
> it is similar to a character and can only be child of a token element. This
> means that additional work will probably be needed.

I'm now wondering whether it would be possible to reuse nsImageFrame...

Comment 12

[Frédéric Wang (:fredw)]

Attachment: testcase

Comment 13

[Frédéric Wang (:fredw)]

Attachment: Patch V1

Comment 14

[Frédéric Wang (:fredw)]

Attachment: Patch V2 - part 1

Asking feedback to Jonas on part 1.

<mglyph/> is a MathML element which provides a mechanism for displaying images to represent non-standard symbols. So I'm trying to create a class that inherits from both nsMathMLElement and nsImageLoadingContent. I looked at how it is done for HTML/SVG elements. However, MathML is a bit different since we don't implement the MathML DOM and so don't need a DOM class for each MathML element...

I took implementation of functions from nsHTMLImageElement, I hope what I did is correct and that I did not forget anything. I'm not really sure to understand all these macros to define/implement functions and whether I should add a definition in dom/base/nsDOMClassInfoClasses.h for nsMathMLMglyphElement. So any help would be appreciated...

Comment 15

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Comment on attachment 587843 [details] [diff] [review]
Patch V2 - part 1

Review of attachment 587843 [details] [diff] [review]:
-----------------------------------------------------------------

First off, I don't really know anything about mathml, so you should check with the layout team if this is a feature that we really want to implement. I say this since it seems like it largely duplicates functionality. This is especially the case if MathML3 deprecates the element.

That aside, it really is a shame that we have to duplicate so much code in each image loading element. I'd be very greatful if you wanted to factor out more stuff into the nsImageLoadingContent class, but that's obviously something that's orthogonal to this bug.

::: content/mathml/content/src/nsMathMLMglyphElement.cpp
@@ +100,5 @@
> +  return rv;
> +}
> +
> +nsresult
> +nsMathMLMglyphElement::SetAttr(PRInt32 aNameSpaceID, nsIAtom* aName,

Rather than overriding SetAttr and UnsetAttr, it would be somewhat cleaner to just hook into AfterSetAttr. See nsSVGImageElement for an example (though that uses svg-specific image loading functions, which is not what you want to do here. Keep the function calls that you currently have).

@@ +113,5 @@
> +      return NS_OK;
> +    }
> +
> +    // A hack to get animations to reset. See bug 594771.
> +    mNewRequestsWillNeedAnimationReset = true;

We don't use this mNewRequestsWillNeedAnimationReset hack in for example svg images, so I think we should avoid it here too.

Comment 16

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

cc'ing roc and dbaron to get input on if this is a feature that we want to implement.

Comment 17

[David Baron :dbaron:]

Why was it deprecated and why is it something we want to implement?  (Are downloadable fonts a better solution here?)

Comment 18

[Karl Tomlinson (:karlt)]

(In reply to David Baron [:dbaron] from comment #17)
> Why was it deprecated and why is it something we want to implement?

What was deprecated was the fontfamily and index attributes of MathML2's mglyph element.  There is no intention to implement that functionality.
http://www.w3.org/TR/MathML2/chapter3.html#presm.mglyph

The reason given was
"Originally, mglyph was designed to provide access to non-standard fonts. Since this functionality was seldom implemented, nor were downloadable web fonts widely available, this use of mglyph has been deprecated."
http://www.w3.org/TR/MathML3/chapter3.html#presm.mglyph

MathML3's mglyph (which is not marked as deprectated) is quite different and instead uses an image to display "non-standard symbols" "where existing Unicode characters are not adequate".

> (Are downloadable fonts a better solution here?)

Downloadable fonts would have the advantage over the referenced "widely recognized image formats", "GIF, JPEG and PNG", that the glyphs scale well.

However, downloadable fonts are awkward for non-Unicode characters because it would be necessary to use the Private Use Area.  The fallback mechanism would not be as good as the alt text for non-visual rendering or for when downloaded fonts are disabled.

A vector image format with mglyph seems the best approach here.

Comment 19

[Karl Tomlinson (:karlt)]

(In reply to David Baron [:dbaron] from comment #17)
> why is it something we want to implement?

I didn't answer this.  The MathML spec still includes mglyph, though I don't know the use cases that motivated this.  It acknowledges that Unicode characters "should meet almost all users needs".  The reason it gives for mglyph is "MathML recognizes that mathematics is not static and that new characters and symbols are added when convenient. Characters that become well accepted will likely be eventually incorporated by the Unicode Consortium or other standards bodies, but that is often a lengthy process."

Frédéric, do you know of any demand for mglyph support, in MathJax or elsewhere?

Comment 20

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Does the patch as implemented here allow us to point a mglyph src at a SVG image? That sounds like it would be ideal. I *think* that works, but we should verify before checking in.

Comment 21

[Frédéric Wang (:fredw)]

Attachment: testcase 1

Comment 22

[Frédéric Wang (:fredw)]

Attachment: testcase 2

Comment 23

[Frédéric Wang (:fredw)]

I know that mglyph is implemented at least in MathJax and MathPlayer, but I don't have an example in mind where mglyph is necessary. Some people probably need it and asked to keep it, for otherwise the Math WG would just have deprecated mglyph and not converted it into the version we have in MathML3.

I tested the three patches with an SVG image and that works (I added it to testcase 1). However, they make the browser crash with testcase 2 (it looks to happen when the image are not in cached) so I'll have to work on it any further.

Comment 24

[Frédéric Wang (:fredw)]

Attachment: Patch - part 1

Comment 25

[Frédéric Wang (:fredw)]

Attachment: Patch - part 2

Comment 26

[Frédéric Wang (:fredw)]

Attachment: Patch - part 3

Comment 27

[Frédéric Wang (:fredw)]

I don't plan to work on this in the short term and don't want to continue to unbitrot the patches. I uploaded the latest versions from my patch queue. Feel free to take the bug and finish the work on it.

Comment 28

[Frédéric Wang (:fredw)]

(In reply to Karl Tomlinson (:karlt) from comment #19)
> (In reply to David Baron [:dbaron] from comment #17)
> > why is it something we want to implement?
> 
> I didn't answer this.  The MathML spec still includes mglyph, though I don't
> know the use cases that motivated this.  It acknowledges that Unicode
> characters "should meet almost all users needs".  The reason it gives for
> mglyph is "MathML recognizes that mathematics is not static and that new
> characters and symbols are added when convenient. Characters that become
> well accepted will likely be eventually incorporated by the Unicode
> Consortium or other standards bodies, but that is often a lengthy process."
> 
> Frédéric, do you know of any demand for mglyph support, in MathJax or
> elsewhere?

There have recently been discussions in Webkit about the implementation of <semantics>. It seems that it is important for people to have some fallback images and <mglyph> or <annotation> can be used to to so. In an HTML5 environment, we can also just use our <img> element but I guess some people still want to make this fallback available with the other techniques.

https://bugs.webkit.org/show_bug.cgi?id=100626

I'm affraid the patches are bitrotten again and I did not get the time to see where the crash comes from. But if we continue the work on this bug, I guess we should also support image with <annotation> too and share the implementation with <mglyph>.

See also bug 745131 for <semantic> improvements.