Closed Bug 534970 (mathml-operator-dict) Opened 16 years ago Closed 14 years ago

[MathML3] Update MathML Operator Dictionary

Categories

(Core :: MathML, defect)

defect
Not set
normal

Tracking

()

RESOLVED FIXED
mozilla5

People

(Reporter: fredw, Assigned: fredw)

References

Details

(Whiteboard: [2.0-approved-patches-landed])

Attachments

(16 files, 15 obsolete files)

1.97 KB, patch
karlt
: review+
Details | Diff | Splinter Review
6.80 KB, patch
karlt
: review+
Details | Diff | Splinter Review
4.48 KB, patch
karlt
: review+
Details | Diff | Splinter Review
3.78 KB, patch
karlt
: review+
Details | Diff | Splinter Review
5.32 KB, patch
karlt
: review+
Details | Diff | Splinter Review
7.77 KB, patch
karlt
: review+
Details | Diff | Splinter Review
76.69 KB, patch
karlt
: review+
Details | Diff | Splinter Review
39.20 KB, patch
karlt
: review+
Details | Diff | Splinter Review
15.64 KB, patch
karlt
: review+
Details | Diff | Splinter Review
28.46 KB, patch
karlt
: review+
Details | Diff | Splinter Review
3.86 KB, patch
karlt
: review+
Details | Diff | Splinter Review
2.88 KB, patch
karlt
: review+
Details | Diff | Splinter Review
19.37 KB, patch
karlt
: review+
Details | Diff | Splinter Review
76.72 KB, patch
karlt
: review+
Details | Diff | Splinter Review
4.64 KB, patch
Details | Diff | Splinter Review
2.61 KB, patch
karlt
: review+
Details | Diff | Splinter Review
MathML 3 comes with a new format and updated entries for Operator Dictionary. Bug 519126 is about using this new compact format. The present bug aims at updating the entries in mathfont.properties.
Depends on: 213932
Alias: mathml-operator-dict
This script allows to compare our version of the dictionary against the official one. Most changes are new operators or lspace/rspace values, so I think we can take them safely. Probably we need more care for changes related to stretching. The script is able to output the details of the differences as well as an updated version of our dictionary. Entries in differences.txt are marked with "stretching", "conflict", "new", "obsolete" so you can use "grep -A2" to select a particular subset. The updated version takes the values of the W3C dictionary, except that it keeps our "direction", preserve our comment if any (end of line after the sharp "#") and keeps the "obsolete" entries. Entries are sorted with respect to the W3C dictionary, except for the obsolete ones, which are put at the end and sorted by lexical order over the key.
After applying the changes of attachment 423544 [details] [diff] [review] and attachment 441533 [details] [diff] [review] (for mathfont.properties) and comparing with http://monet.nag.co.uk/~dpc/draft-spec/appendixc.xml, the script returns: - 92 obsolete entries (24 of them are related to stretching) - 196 unchanged entries - 102 conflicting entries (9 of them are related to stretching) - 760 new entries (87 of them are related to stretching)
Assignee: nobody → fred.wang
Status: NEW → ASSIGNED
With the patch of comment 4, the differences are: - 92 obsolete entries (24 of them are related to stretching) - 214 unchanged entries - 84 conflicting entries (5 of them are related to stretching) - 760 new entries (87 of them are related to stretching) The 5 remaining entries whose stretchiness has changed are: [conflict][stretching] - operator.\u005F.infix (low line) -lspace:2 rspace:2 stretchy +lspace:1 rspace:1 -- [conflict][stretching] - operator.\u2225.infix (parallel to) -lspace:5 rspace:5 stretchy +lspace:5 rspace:5 -- [conflict][stretching] - operator.\u2223.infix (divides) -lspace:5 rspace:5 stretchy +lspace:5 rspace:5 -- [conflict][stretching] - operator.\u2216.infix (set minus) -lspace:3 rspace:3 stretchy +lspace:4 rspace:4 -- [conflict][stretching] - operator.\u002F.infix (solidus) -lspace:3 rspace:3 stretchy +lspace:1 rspace:1
Attachment #442184 - Attachment is obsolete: true
There was a bug in my script for multiple char operators. The new values are: - 54 obsolete entries (24 of them are related to stretching) - 214 unchanged entries - 122 conflicting entries (5 of them are related to stretching) - 722 new entries (87 of them are related to stretching)
Attachment #442182 - Attachment is obsolete: true
Attachment #442466 - Attachment is obsolete: true
Among the 24 obsolete entries that are stretchy, there is at least two annoying ones for the "scale strechy" patch: operator.\u22C2.infix (&Intersection;) operator.\u22C3.infix (&Union;) In Joe Java's test, in "De Morgan's laws" for sets there is something like: <mrow><mi>X</mi> <mo>=</mo> <mo>&Intersection;</mo> <mi>Y</mi> </mrow> Here, the &Intersection; operator is considered as infix (so stretchy and not largeop) rather than prefix (not stretchy and largeop). Without the patch, we don't see the difference. But with the patch applied, the former stretches the symbol in the vertical direction only (to cover the height of the formula) while the latter simply increases width and height by twice (it's what we want). To fix the problem, we can: - make these operators non-strechy. But still they won't be largeop. - remove them. For infix operators, the MathML WG uses &#x2229; and &#x222A; instead. - add a rule for determining the form, that takes into account this kind of "equal-like" operators.
Attached image bug infix &Intersection; (obsolete) —
(In reply to comment #8) > - remove them. For infix operators, the MathML WG uses &#x2229; and &#x222A; > instead. Removing the infix operators so that U+22C2 N-ARY INTERSECTION is always treated like its prefix form seems reasonable to me. i.e. It seems that N-ARY INTERSECTION should not be used for a binary operator.
> [conflict][stretching] - operator.\u2225.infix (parallel to) > -lspace:5 rspace:5 stretchy > +lspace:5 rspace:5 Note bug 414294.
Blocks: 414294
This patch adds stretchy vertical/horizontal arrows and should fix bug 516292.
Attachment #442730 - Flags: review?(karlt)
Attachment #442729 - Flags: review? → review?(karlt)
Attachment #442458 - Attachment is obsolete: true
Order of application is: attachment 442729 [details] [diff] [review], attachment 442730 [details] [diff] [review] and attachment 442735 [details] [diff] [review]. The two first can be safely taken now. For the third one, we have to wait the "scale stretchy" patch, otherwise integrals will be too small.
New values: - 52 obsolete entries (22 of them are related to stretching) - 293 unchanged entries - 122 conflicting entries (5 of them are related to stretching) (see comment 5) - 643 new entries (8 of them are related to stretching) The 8 remaining "new stretchy entries" are: operator.\u2044.infix (fraction slash) operator.\u2215.infix (division slash) operator.\u21D6.infix (north west double arrow) operator.\u21D7.infix (north east double arrow) operator.\u21D8.infix (south east double arrow) operator.\u21D9.infix (south west double arrow) operator.\u2921.infix (north west and south east arrow) operator.\u2922.infix (north east and south west arrow) For the diagonal arrows, I think we should wait bug 552290. For the two, see bug 414294 comment 15.
I've updated the set of patches to add a constraint on directions. Order of application of patches is: 1) attachment 442729 [details] [diff] [review] 2) attachment 442730 [details] [diff] [review] 3) attachment 534970 [details] [diff] [review] 4) attachment 446279 [details] [diff] [review] 5) attachment 446281 [details] [diff] [review]
Attachment #446279 - Flags: review?(karlt)
Attachment #442729 - Flags: review?(karlt) → review+
Attachment #442730 - Flags: review?(karlt) → review+
Attachment #443008 - Flags: review?(karlt) → review+
Comment on attachment 446279 [details] [diff] [review] make operator with a stretchy form use the same direction for all their forms [pushed] Noting that this is for bug 524275 comment 11.
Attachment #446279 - Flags: review?(karlt) → review+
This new property is for bug 414277.
and this is the patch for mathfont.properties
Attachment #442638 - Attachment is obsolete: true
Attachment #447036 - Attachment is obsolete: true
Attachment #454961 - Flags: review+
Attachment #447037 - Flags: review+
Maybe we can also take the two patches adding the scripts? They are not used in Gecko code, but rather are just checking the operator dictionary, so I think it would be safe to take them. The Math WG has updated its dictionary, so the result changed: - 49 obsolete entries (19 of them are related to stretching) - 285 unchanged entries - 155 conflicting entries (7 of them are related to stretching) - 645 new entries (38 of them are related to stretching) In a first time, I think I'm going to provide a patch which reorders the entries of mathfont.properties with respect to the script output, but without modifying them. Hence it would be easier to see the future changes. Next, I'll give a patch adding the new entries that do not have a stretchy or largeop property. This should be the case for most of them.
Attached patch sort entries of the dictionary (obsolete) — Splinter Review
Attachment #463330 - Flags: review?(karlt)
Here are the new values after applying the two patches: - 49 obsolete entries (19 of them are related to stretching) - 884 unchanged entries - 155 conflicting entries (7 of them are related to stretching) - 46 new entries (38 of them are related to stretching) The 8 remaining non-stretchy new entries are the following largeops: [new entry] - operator.\u2A03.prefix (n-ary union operator with dot) [new entry] - operator.\u2A05.prefix (n-ary square intersection operator) [new entry] - operator.\u2A07.prefix (two logical and operator) [new entry] - operator.\u2A08.prefix (two logical or operator) [new entry] - operator.\u2A09.prefix (n-ary times operator) [new entry] - operator.\u2A0A.prefix (modulo two sum) [new entry] - operator.\u2AFC.prefix (large triple vertical bar operator) [new entry] - operator.\u2AFF.prefix (n-ary white vertical bar)
Attachment #463669 - Flags: review?(karlt)
The new values after applying the two last patches: - 49 obsolete entries (19 of them are related to stretching) - 996 unchanged entries - 77 conflicting entries (7 of them are related to stretching) - 12 new entries (12 of them are related to stretching)
- 49 obsolete entries (19 of them are related to stretching) - 1066 unchanged entries - 7 conflicting entries (7 of them are related to stretching) - 12 new entries (12 of them are related to stretching)
For these infix forms, one should rather use operator.\u2A3F.infix = lspace:4 rspace:4 # amalgamation or coproduct operator.\u2227.infix = lspace:4 rspace:4 # &wedge; operator.\u2228.infix = lspace:4 rspace:4 # &vee; This patch should allow to pass all the tests in Topics/LargeOp/ (from the MathML3 testsuite).
Attachment #463829 - Flags: review?(karlt)
Comment on attachment 463330 [details] [diff] [review] sort entries of the dictionary This patch seems to be removing a "mirrorable" flag. That means it would only apply after attachment 450849 [details] [diff] [review] is applied I guess, but then it would be undoing much of that attachment. (There are a couple of minsize=0 attributes removed too, but I think that's fine because I don't think those values change anything.) I like the idea of sorting these entries though.
Attachment #463330 - Flags: review?(karlt) → review-
(In reply to comment #30) > Maybe we can also take the two patches adding the scripts? Yes, that makes sense.
Attachment #463331 - Flags: review?(karlt) → review+
Attachment #463669 - Flags: review?(karlt) → review+
Attachment #463671 - Flags: review?(karlt) → review+
Comment on attachment 463818 [details] [diff] [review] fix conflicts for entries becoming accent Have you checked whether drawing the arrows larger (which accent should cause) looks reasonable?
Attachment #463819 - Flags: review?(karlt) → review+
Attachment #463829 - Flags: review?(karlt) → review+
Requesting approval2.0. This patch is necessary in some situations to make the patch in bug 414277 behave as expected. The patch contains no code but modifies data only. Much of this data can be set or overridden by the author but these values provide sensible defaults.
Attachment #463997 - Flags: approval2.0?
(In reply to comment #41) > Comment on attachment 463330 [details] [diff] [review] > sort entries of the dictionary > > This patch seems to be removing a "mirrorable" flag. > That means it would only apply after attachment 450849 [details] [diff] [review] is applied I guess, but > then it would be undoing much of that attachment. Sorry, it's a mistake. We are likely to take those patches before implementing RTL directionality, so it's better to apply them before attachment 450849 [details] [diff] [review]. Fortunately, the script does not support the mirrorable flag yet, so it has been removed and I finally obtained the expected result. > > (There are a couple of minsize=0 attributes removed too, but I think that's > fine because I don't think those values change anything.) > There was a bug in the script, minsize was removed when it took the value 0. However, this is also what the MathML dictionary does and anyway I think we don't parse it.
Attachment #454962 - Attachment is obsolete: true
Attachment #463330 - Attachment is obsolete: true
Attachment #464150 - Flags: review?(karlt)
Attachment #464150 - Flags: review?(karlt) → review+
Attachment #464149 - Flags: review?(karlt)
Attachment #454963 - Flags: review?(karlt)
Comment on attachment 442729 [details] [diff] [review] remove infix entries for N-ary Union and Intersection [pushed] http://hg.mozilla.org/mozilla-central/rev/170dc0922c23
Attachment #442729 - Attachment description: remove infix entries for N-ary Union and Intersection → remove infix entries for N-ary Union and Intersection [pushed]
Comment on attachment 442730 [details] [diff] [review] add entries for vertical and horizontal arrows [pushed] http://hg.mozilla.org/mozilla-central/rev/8ddb819c17fd
Attachment #442730 - Attachment description: add entries for vertical and horizontal arrows → add entries for vertical and horizontal arrows [pushed]
Attachment #443008 - Attachment description: add entries for stretchy fences → add entries for stretchy fences [pushed]
Comment on attachment 446279 [details] [diff] [review] make operator with a stretchy form use the same direction for all their forms [pushed] http://hg.mozilla.org/mozilla-central/rev/4428b9c468cd
Attachment #446279 - Attachment description: make operator with a stretchy form use the same direction for all their forms → make operator with a stretchy form use the same direction for all their forms [pushed]
Attachment #447037 - Attachment description: add private data "integral" → add private data "integral" [pushed]
Comment on attachment 454961 [details] [diff] [review] update some entries whose stretchiness has changed [pushed] http://hg.mozilla.org/mozilla-central/rev/77e3880307ea
Attachment #454961 - Attachment description: update some entries whose stretchiness has changed → update some entries whose stretchiness has changed [pushed]
(In reply to comment #43) > Comment on attachment 463818 [details] [diff] [review] > fix conflicts for entries becoming accent > > Have you checked whether drawing the arrows larger (which accent should cause) > looks reasonable? The new rendering looks good to me.
Attachment #463818 - Flags: review?(karlt) → review+
Comment on attachment 464149 [details] [diff] [review] Add a perl script to compare MathML operator dictionaries and check validity of our private data [pushed] rs=me
Attachment #464149 - Flags: review?(karlt) → review+
Comment on attachment 454963 [details] [diff] [review] updateOperatorDictionary.pl: add private data "integral" >+ print $file_syntax_errors "warning: operator contains the term \"intgral\" in its comment, but is not integral\n"; \"integral\"
Attachment #454963 - Flags: review?(karlt) → review+
Comment on attachment 464149 [details] [diff] [review] Add a perl script to compare MathML operator dictionaries and check validity of our private data [pushed] http://hg.mozilla.org/mozilla-central/rev/38bc6e6dbf06
Attachment #464149 - Attachment description: Add a perl script to compare MathML operator dictionaries and check validity of our private data → Add a perl script to compare MathML operator dictionaries and check validity of our private data [pushed]
Comment on attachment 480859 [details] [diff] [review] updateOperatorDictionary.pl: add private data "integral" (fix typo) [pushed] http://hg.mozilla.org/mozilla-central/rev/0e03f2057b4c
Attachment #480859 - Attachment description: updateOperatorDictionary.pl: add private data "integral" (fix typo) → updateOperatorDictionary.pl: add private data "integral" (fix typo) [pushed]
Depends on: post2.0
this will be used after MathML bidi is implemented
Whiteboard: [approved-patches-landed]
Karl, if there are any more patches to land here, please land them on cedar.
Attachment #463997 - Attachment is obsolete: true
Attachment #500676 - Flags: review+
Whiteboard: [approved-patches-landed] → [2.0-approved-patches-landed][waiting tryserver results before landing remainder on cedar]
Comment on attachment 463818 [details] [diff] [review] fix conflicts for entries becoming accent Making U+21DB an accent broke the scale-stretchy-3 reftest (Bug 605605 comment 22) as its munder didn't expect mo to be rendered differently to mspace because it was an accent. I've addressed this locally by making all the munders in that test use accentunder=false.
Yes, it seems a reasonable fix. Do you plan to attach a patch?
Status: ASSIGNED → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
Whiteboard: [2.0-approved-patches-landed][waiting tryserver results before landing remainder on cedar] → [2.0-approved-patches-landed]
Target Milestone: --- → mozilla2.2
No longer depends on: post2.0
OK, so I guess we should open a separate bug to fix the scale-stretchy-3 reftest?
The remaining differences: - 47 obsolete entries (19 of them are related to stretching) - 7 conflicting entries (7 of them are related to stretching) [conflict][stretching] - operator.\u002F.infix (solidus) -lspace:3 rspace:3 stretchy +lspace:1 rspace:1 [conflict][stretching] - operator.\u005F.infix (low line) -lspace:2 rspace:2 stretchy +lspace:1 rspace:1 [conflict][stretching] - operator.\u007C\u007C.infix (multiple character operator: ||) -lspace:4 rspace:4 +lspace:2 rspace:2 stretchy fence symmetric [conflict][stretching] - operator.\u2216.infix (set minus) -lspace:3 rspace:3 stretchy +lspace:4 rspace:4 [conflict][stretching] - operator.\u2223.infix (divides) -lspace:5 rspace:5 stretchy +lspace:5 rspace:5 [conflict][stretching] - operator.\u2225.infix (parallel to) -lspace:5 rspace:5 stretchy +lspace:5 rspace:5 [conflict][stretching] - operator.\u2758.infix (light vertical bar) -lspace:5 rspace:5 stretchy symmetric +lspace:5 rspace:5 - 13 new entries (12 of them are related to stretching) [new entry][stretching] - operator.\u2044.infix (fraction slash) - +lspace:4 rspace:4 stretchy [new entry][stretching] - operator.\u21D6.infix (north west double arrow) - +lspace:5 rspace:5 stretchy [new entry][stretching] - operator.\u21D7.infix (north east double arrow) - +lspace:5 rspace:5 stretchy [new entry][stretching] - operator.\u21D8.infix (south east double arrow) - +lspace:5 rspace:5 stretchy [new entry][stretching] - operator.\u21D9.infix (south west double arrow) - +lspace:5 rspace:5 stretchy [new entry][stretching] - operator.\u2215.infix (division slash) +lspace:4 rspace:4 stretchy [new entry][stretching] - operator.\u27F0.infix (upwards quadruple arrow) +lspace:5 rspace:5 stretchy [new entry][stretching] - operator.\u27F1.infix (downwards quadruple arrow) +lspace:5 rspace:5 stretchy [new entry][stretching] - operator.\u2921.infix (north west and south east arrow) +lspace:5 rspace:5 stretchy [new entry][stretching] - operator.\u2922.infix (north east and south west arrow) +lspace:5 rspace:5 stretchy [new entry] - operator.\u2ADD\u0338.infix (nonforking with slash) +lspace:5 rspace:5 [new entry][stretching] - operator.\u2B45.infix (leftwards quadruple arrow) +lspace:5 rspace:5 stretchy [new entry][stretching] - operator.\u2B46.infix (rightwards quadruple arrow) +lspace:5 rspace:5 stretchy
(In reply to comment #65) > OK, so I guess we should open a separate bug to fix the scale-stretchy-3 > reftest? That was fixed here: http://hg.mozilla.org/mozilla-central/rev/0e519f6626c7 (Please use a separate bug if providing any further operator dictionary patches, though.)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: