Bug 534970 (mathml-operator-dict)

[MathML3] Update MathML Operator Dictionary

RESOLVED FIXED in mozilla5

Status

()

defect
RESOLVED FIXED
10 years ago
8 years ago

People

(Reporter: fredw, Assigned: fredw)

Tracking

(Blocks 1 bug)

Trunk
mozilla5
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [2.0-approved-patches-landed])

Attachments

(16 attachments, 15 obsolete attachments)

1.97 KB, patch
karlt
: review+
Details | Diff | Splinter Review
6.80 KB, patch
karlt
: review+
Details | Diff | Splinter Review
4.48 KB, patch
karlt
: review+
Details | Diff | Splinter Review
3.78 KB, patch
karlt
: review+
Details | Diff | Splinter Review
5.32 KB, patch
karlt
: review+
Details | Diff | Splinter Review
7.77 KB, patch
karlt
: review+
Details | Diff | Splinter Review
76.69 KB, patch
karlt
: review+
Details | Diff | Splinter Review
39.20 KB, patch
karlt
: review+
Details | Diff | Splinter Review
15.64 KB, patch
karlt
: review+
Details | Diff | Splinter Review
28.46 KB, patch
karlt
: review+
Details | Diff | Splinter Review
3.86 KB, patch
karlt
: review+
Details | Diff | Splinter Review
2.88 KB, patch
karlt
: review+
Details | Diff | Splinter Review
19.37 KB, patch
karlt
: review+
Details | Diff | Splinter Review
76.72 KB, patch
karlt
: review+
Details | Diff | Splinter Review
4.64 KB, patch
Details | Diff | Splinter Review
2.61 KB, patch
karlt
: review+
Details | Diff | Splinter Review
Assignee

Description

10 years ago
MathML 3 comes with a new format and updated entries for Operator Dictionary. Bug 519126 is about using this new compact format. The present bug aims at updating the entries in mathfont.properties.
Assignee

Updated

9 years ago
Depends on: 213932
Assignee

Updated

9 years ago
Alias: mathml-operator-dict
Assignee

Comment 1

9 years ago
This script allows to compare our version of the dictionary against the official one. Most changes are new operators or lspace/rspace values, so I think we can take them safely. Probably we need more care for changes related to stretching.

The script is able to output the details of the differences as well as an updated version of our dictionary. Entries in differences.txt are marked with "stretching", "conflict", "new", "obsolete" so you can use "grep -A2" to select a particular subset. The updated version takes the values of the W3C dictionary, except that it keeps our "direction", preserve our comment if any (end of line after the sharp "#") and keeps the "obsolete" entries. Entries are sorted with respect to the W3C dictionary, except for the obsolete ones, which are put at the end and sorted by lexical order over the key.
Assignee

Comment 2

9 years ago
After applying the changes of attachment 423544 [details] [diff] [review] and attachment 441533 [details] [diff] [review] (for mathfont.properties) and comparing with http://monet.nag.co.uk/~dpc/draft-spec/appendixc.xml, the script returns:

- 92 obsolete entries (24 of them are related to stretching)
- 196 unchanged entries
- 102 conflicting entries (9 of them are related to stretching)
- 760 new entries (87 of them are related to stretching)
Assignee

Comment 4

9 years ago
Assignee: nobody → fred.wang
Status: NEW → ASSIGNED
Assignee

Comment 5

9 years ago
With the patch of comment 4, the differences are:

- 92 obsolete entries (24 of them are related to stretching)
- 214 unchanged entries
- 84 conflicting entries (5 of them are related to stretching)
- 760 new entries (87 of them are related to stretching)

The 5 remaining entries whose stretchiness has changed are:

[conflict][stretching] - operator.\u005F.infix (low line)
-lspace:2 rspace:2 stretchy
+lspace:1 rspace:1
--
[conflict][stretching] - operator.\u2225.infix (parallel to)
-lspace:5 rspace:5 stretchy
+lspace:5 rspace:5
--
[conflict][stretching] - operator.\u2223.infix (divides)
-lspace:5 rspace:5 stretchy
+lspace:5 rspace:5
--
[conflict][stretching] - operator.\u2216.infix (set minus)
-lspace:3 rspace:3 stretchy
+lspace:4 rspace:4
--
[conflict][stretching] - operator.\u002F.infix (solidus)
-lspace:3 rspace:3 stretchy
+lspace:1 rspace:1
Assignee

Comment 6

9 years ago
Attachment #442184 - Attachment is obsolete: true
Assignee

Comment 7

9 years ago
There was a bug in my script for multiple char operators. The new values are:

- 54 obsolete entries (24 of them are related to stretching)
- 214 unchanged entries
- 122 conflicting entries (5 of them are related to stretching)
- 722 new entries (87 of them are related to stretching)
Attachment #442182 - Attachment is obsolete: true
Attachment #442466 - Attachment is obsolete: true
Assignee

Comment 8

9 years ago
Among the 24 obsolete entries that are stretchy, there is at least two annoying ones for the "scale strechy" patch:

operator.\u22C2.infix (⋂)
operator.\u22C3.infix (⋃)

In Joe Java's test, in "De Morgan's laws" for sets there is something like:

<mrow><mi>X</mi> <mo>=</mo> <mo>&Intersection;</mo> <mi>Y</mi>  </mrow>

Here, the &Intersection; operator is considered as infix (so stretchy and not largeop) rather than prefix (not stretchy and largeop). Without the patch, we don't see the difference. But with the patch applied, the former stretches the symbol in the vertical direction only (to cover the height of the formula) while the latter simply increases width and height by twice (it's what we want).

To fix the problem, we can:
- make these operators non-strechy. But still they won't be largeop.
- remove them. For infix operators, the MathML WG uses &#x2229; and &#x222A; instead.
- add a rule for determining the form, that takes into account this kind of "equal-like" operators.
Assignee

Comment 9

9 years ago
Posted image bug infix &Intersection; (obsolete) —
(In reply to comment #8)
> - remove them. For infix operators, the MathML WG uses &#x2229; and &#x222A;
> instead.

Removing the infix operators so that U+22C2 N-ARY INTERSECTION is always treated like its prefix form seems reasonable to me.
i.e. It seems that N-ARY INTERSECTION should not be used for a binary operator.
> [conflict][stretching] - operator.\u2225.infix (parallel to)
> -lspace:5 rspace:5 stretchy
> +lspace:5 rspace:5

Note bug 414294.
Blocks: 414294
Assignee

Comment 13

9 years ago
This patch adds stretchy vertical/horizontal arrows and should fix bug 516292.
Attachment #442730 - Flags: review?(karlt)
Assignee

Updated

9 years ago
Attachment #442729 - Flags: review? → review?(karlt)
Assignee

Comment 14

9 years ago
Attachment #442458 - Attachment is obsolete: true
Assignee

Comment 15

9 years ago
Order of application is: attachment 442729 [details] [diff] [review], attachment 442730 [details] [diff] [review] and attachment 442735 [details] [diff] [review].

The two first can be safely taken now. For the third one, we have to wait the "scale stretchy" patch, otherwise integrals will be too small.
Assignee

Comment 17

9 years ago
New values:

- 52 obsolete entries (22 of them are related to stretching)
- 293 unchanged entries
- 122 conflicting entries (5 of them are related to stretching) (see comment 5)
- 643 new entries (8 of them are related to stretching)

The 8 remaining "new stretchy entries" are:

operator.\u2044.infix (fraction slash)
operator.\u2215.infix (division slash)
operator.\u21D6.infix (north west double arrow)
operator.\u21D7.infix (north east double arrow)
operator.\u21D8.infix (south east double arrow)
operator.\u21D9.infix (south west double arrow)
operator.\u2921.infix (north west and south east arrow)
operator.\u2922.infix (north east and south west arrow)

For the diagonal arrows, I think we should wait bug 552290. For the two, see bug 414294 comment 15.
Assignee

Comment 21

9 years ago
I've updated the set of patches to add a constraint on directions.

Order of application of patches is:
1) attachment 442729 [details] [diff] [review]
2) attachment 442730 [details] [diff] [review]
3) attachment 534970 [details] [diff] [review]
4) attachment 446279 [details] [diff] [review]
5) attachment 446281 [details] [diff] [review]
Assignee

Updated

9 years ago
Attachment #446279 - Flags: review?(karlt)
Attachment #442729 - Flags: review?(karlt) → review+
Attachment #442730 - Flags: review?(karlt) → review+
Attachment #443008 - Flags: review?(karlt) → review+
Comment on attachment 446279 [details] [diff] [review]
make operator with a stretchy form use the same direction for all their forms [pushed]

Noting that this is for bug 524275 comment 11.
Attachment #446279 - Flags: review?(karlt) → review+
Assignee

Comment 25

9 years ago
This new property is for bug 414277.
Assignee

Comment 26

9 years ago
and this is the patch for mathfont.properties
Assignee

Updated

9 years ago
Attachment #442638 - Attachment is obsolete: true
Assignee

Updated

9 years ago
Attachment #447036 - Attachment is obsolete: true
Attachment #454961 - Flags: review+
Attachment #447037 - Flags: review+
Assignee

Comment 30

9 years ago
Maybe we can also take the two patches adding the scripts? They are not used in Gecko code, but rather are just checking the operator dictionary, so I think it would be safe to take them.

The Math WG has updated its dictionary, so the result changed:

- 49 obsolete entries (19 of them are related to stretching)
- 285 unchanged entries
- 155 conflicting entries (7 of them are related to stretching)
- 645 new entries (38 of them are related to stretching)

In a first time, I think I'm going to provide a patch which reorders the entries of mathfont.properties with respect to the script output, but without modifying them. Hence it would be easier to see the future changes. Next, I'll give a patch adding the new entries that do not have a stretchy or largeop property. This should be the case for most of them.
Assignee

Comment 31

9 years ago
Attachment #463330 - Flags: review?(karlt)
Assignee

Comment 33

9 years ago
Here are the new values after applying the two patches:

- 49 obsolete entries (19 of them are related to stretching)
- 884 unchanged entries
- 155 conflicting entries (7 of them are related to stretching)
- 46 new entries (38 of them are related to stretching)

The 8 remaining non-stretchy new entries are the following largeops:

[new entry] - operator.\u2A03.prefix (n-ary union operator with dot)
[new entry] - operator.\u2A05.prefix (n-ary square intersection operator)
[new entry] - operator.\u2A07.prefix (two logical and operator)
[new entry] - operator.\u2A08.prefix (two logical or operator)
[new entry] - operator.\u2A09.prefix (n-ary times operator)
[new entry] - operator.\u2A0A.prefix (modulo two sum)
[new entry] - operator.\u2AFC.prefix (large triple vertical bar operator)
[new entry] - operator.\u2AFF.prefix (n-ary white vertical bar)
Assignee

Comment 34

9 years ago
Attachment #463669 - Flags: review?(karlt)
Assignee

Comment 36

9 years ago
The new values after applying the two last patches:

- 49 obsolete entries (19 of them are related to stretching)
- 996 unchanged entries
- 77 conflicting entries (7 of them are related to stretching)
- 12 new entries (12 of them are related to stretching)
Assignee

Comment 39

9 years ago
- 49 obsolete entries (19 of them are related to stretching)
- 1066 unchanged entries
- 7 conflicting entries (7 of them are related to stretching)
- 12 new entries (12 of them are related to stretching)
Assignee

Comment 40

9 years ago
For these infix forms, one should rather use

operator.\u2A3F.infix = lspace:4 rspace:4 # amalgamation or coproduct
operator.\u2227.infix = lspace:4 rspace:4 # &wedge;
operator.\u2228.infix = lspace:4 rspace:4 # &vee;

This patch should allow to pass all the tests in Topics/LargeOp/ (from the MathML3 testsuite).
Attachment #463829 - Flags: review?(karlt)
Comment on attachment 463330 [details] [diff] [review]
sort entries of the dictionary

This patch seems to be removing a "mirrorable" flag.
That means it would only apply after attachment 450849 [details] [diff] [review] is applied I guess, but
then it would be undoing much of that attachment.

(There are a couple of minsize=0 attributes removed too, but I think that's
fine because I don't think those values change anything.)

I like the idea of sorting these entries though.
Attachment #463330 - Flags: review?(karlt) → review-
(In reply to comment #30)
> Maybe we can also take the two patches adding the scripts?

Yes, that makes sense.
Attachment #463331 - Flags: review?(karlt) → review+
Attachment #463669 - Flags: review?(karlt) → review+
Attachment #463671 - Flags: review?(karlt) → review+
Comment on attachment 463818 [details] [diff] [review]
fix conflicts for entries becoming accent

Have you checked whether drawing the arrows larger (which accent should cause) looks reasonable?
Attachment #463819 - Flags: review?(karlt) → review+
Attachment #463829 - Flags: review?(karlt) → review+
Requesting approval2.0.  This patch is necessary in some situations to make the patch in bug 414277 behave as expected.
The patch contains no code but modifies data only.  Much of this data can be set or overridden by the author but these values provide sensible defaults.
Attachment #463997 - Flags: approval2.0?
Assignee

Comment 45

9 years ago
(In reply to comment #41)
> Comment on attachment 463330 [details] [diff] [review]
> sort entries of the dictionary
> 
> This patch seems to be removing a "mirrorable" flag.
> That means it would only apply after attachment 450849 [details] [diff] [review] is applied I guess, but
> then it would be undoing much of that attachment.

Sorry, it's a mistake. We are likely to take those patches before implementing RTL directionality, so it's better to apply them before attachment 450849 [details] [diff] [review]. Fortunately, the script does not support the mirrorable flag yet, so it has been removed and I finally obtained the expected result.

> 
> (There are a couple of minsize=0 attributes removed too, but I think that's
> fine because I don't think those values change anything.)
> 

There was a bug in the script, minsize was removed when it took the value 0. However, this is also what the MathML dictionary does and anyway I think we don't parse it.
Assignee

Comment 47

9 years ago
Attachment #454962 - Attachment is obsolete: true
Attachment #463330 - Attachment is obsolete: true
Attachment #464150 - Flags: review?(karlt)
Attachment #464150 - Flags: review?(karlt) → review+
Assignee

Updated

9 years ago
Attachment #464149 - Flags: review?(karlt)
Assignee

Updated

9 years ago
Attachment #454963 - Flags: review?(karlt)
Comment on attachment 442729 [details] [diff] [review]
remove infix entries for N-ary Union and Intersection [pushed]

http://hg.mozilla.org/mozilla-central/rev/170dc0922c23
Attachment #442729 - Attachment description: remove infix entries for N-ary Union and Intersection → remove infix entries for N-ary Union and Intersection [pushed]
Comment on attachment 442730 [details] [diff] [review]
add entries for vertical and horizontal arrows [pushed]

http://hg.mozilla.org/mozilla-central/rev/8ddb819c17fd
Attachment #442730 - Attachment description: add entries for vertical and horizontal arrows → add entries for vertical and horizontal arrows [pushed]
Comment on attachment 443008 [details] [diff] [review]
add entries for stretchy fences [pushed]

http://hg.mozilla.org/mozilla-central/rev/7aee654600f2
Attachment #443008 - Attachment description: add entries for stretchy fences → add entries for stretchy fences [pushed]
Comment on attachment 446279 [details] [diff] [review]
make operator with a stretchy form use the same direction for all their forms [pushed]

http://hg.mozilla.org/mozilla-central/rev/4428b9c468cd
Attachment #446279 - Attachment description: make operator with a stretchy form use the same direction for all their forms → make operator with a stretchy form use the same direction for all their forms [pushed]
Comment on attachment 447037 [details] [diff] [review]
add private data "integral" [pushed]

http://hg.mozilla.org/mozilla-central/rev/8a52e79553b6
Attachment #447037 - Attachment description: add private data "integral" → add private data "integral" [pushed]
Comment on attachment 454961 [details] [diff] [review]
update some entries whose stretchiness has changed [pushed]

http://hg.mozilla.org/mozilla-central/rev/77e3880307ea
Attachment #454961 - Attachment description: update some entries whose stretchiness has changed → update some entries whose stretchiness has changed [pushed]
Assignee

Comment 54

9 years ago
(In reply to comment #43)
> Comment on attachment 463818 [details] [diff] [review]
> fix conflicts for entries becoming accent
> 
> Have you checked whether drawing the arrows larger (which accent should cause)
> looks reasonable?

The new rendering looks good to me.
Attachment #463818 - Flags: review?(karlt) → review+
Comment on attachment 464149 [details] [diff] [review]
Add a perl script to compare MathML operator dictionaries and check validity of our private data [pushed]

rs=me
Attachment #464149 - Flags: review?(karlt) → review+
Comment on attachment 454963 [details] [diff] [review]
updateOperatorDictionary.pl: add private data "integral"

>+            print $file_syntax_errors "warning: operator contains the term \"intgral\" in its comment, but is not integral\n";

\"integral\"
Attachment #454963 - Flags: review?(karlt) → review+
Comment on attachment 464149 [details] [diff] [review]
Add a perl script to compare MathML operator dictionaries and check validity of our private data [pushed]

http://hg.mozilla.org/mozilla-central/rev/38bc6e6dbf06
Attachment #464149 - Attachment description: Add a perl script to compare MathML operator dictionaries and check validity of our private data → Add a perl script to compare MathML operator dictionaries and check validity of our private data [pushed]
Comment on attachment 480859 [details] [diff] [review]
updateOperatorDictionary.pl: add private data "integral" (fix typo) [pushed]

http://hg.mozilla.org/mozilla-central/rev/0e03f2057b4c
Attachment #480859 - Attachment description: updateOperatorDictionary.pl: add private data "integral" (fix typo) → updateOperatorDictionary.pl: add private data "integral" (fix typo) [pushed]
Depends on: post2.0
Assignee

Comment 60

9 years ago
this will be used after MathML bidi is implemented
Whiteboard: [approved-patches-landed]

Comment 61

8 years ago
Karl, if there are any more patches to land here, please land them on cedar.
Attachment #463997 - Attachment is obsolete: true
Attachment #500676 - Flags: review+
Whiteboard: [approved-patches-landed] → [2.0-approved-patches-landed][waiting tryserver results before landing remainder on cedar]
Comment on attachment 463818 [details] [diff] [review]
fix conflicts for entries becoming accent

Making U+21DB an accent broke the scale-stretchy-3 reftest (Bug 605605 comment 22) as its munder didn't expect mo to be rendered differently to mspace because it was an accent.  I've addressed this locally by making all the munders in that test use accentunder=false.
Assignee

Comment 63

8 years ago
Yes, it seems a reasonable fix. Do you plan to attach a patch?
No need.

http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=d9c2c2e76a37&tochange=0d20320645d0
Status: ASSIGNED → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
Whiteboard: [2.0-approved-patches-landed][waiting tryserver results before landing remainder on cedar] → [2.0-approved-patches-landed]
Target Milestone: --- → mozilla2.2
No longer depends on: post2.0
Assignee

Comment 65

8 years ago
OK, so I guess we should open a separate bug to fix the scale-stretchy-3 reftest?
Assignee

Comment 66

8 years ago
The remaining differences:

- 47 obsolete entries (19 of them are related to stretching)

- 7 conflicting entries (7 of them are related to stretching)

[conflict][stretching] - operator.\u002F.infix (solidus)
-lspace:3 rspace:3 stretchy
+lspace:1 rspace:1

[conflict][stretching] - operator.\u005F.infix (low line)
-lspace:2 rspace:2 stretchy
+lspace:1 rspace:1

[conflict][stretching] - operator.\u007C\u007C.infix (multiple character operator: ||)
-lspace:4 rspace:4
+lspace:2 rspace:2 stretchy fence symmetric

[conflict][stretching] - operator.\u2216.infix (set minus)
-lspace:3 rspace:3 stretchy
+lspace:4 rspace:4

[conflict][stretching] - operator.\u2223.infix (divides)
-lspace:5 rspace:5 stretchy
+lspace:5 rspace:5

[conflict][stretching] - operator.\u2225.infix (parallel to)
-lspace:5 rspace:5 stretchy
+lspace:5 rspace:5

[conflict][stretching] - operator.\u2758.infix (light vertical bar)
-lspace:5 rspace:5 stretchy symmetric
+lspace:5 rspace:5

- 13 new entries (12 of them are related to stretching)

[new entry][stretching] - operator.\u2044.infix (fraction slash)
-
+lspace:4 rspace:4 stretchy

[new entry][stretching] - operator.\u21D6.infix (north west double arrow)
-
+lspace:5 rspace:5 stretchy

[new entry][stretching] - operator.\u21D7.infix (north east double arrow)
-
+lspace:5 rspace:5 stretchy

[new entry][stretching] - operator.\u21D8.infix (south east double arrow)
-
+lspace:5 rspace:5 stretchy

[new entry][stretching] - operator.\u21D9.infix (south west double arrow)
-
+lspace:5 rspace:5 stretchy

[new entry][stretching] - operator.\u2215.infix (division slash)
+lspace:4 rspace:4 stretchy

[new entry][stretching] - operator.\u27F0.infix (upwards quadruple arrow)
+lspace:5 rspace:5 stretchy

[new entry][stretching] - operator.\u27F1.infix (downwards quadruple arrow)
+lspace:5 rspace:5 stretchy

[new entry][stretching] - operator.\u2921.infix (north west and south east arrow)
+lspace:5 rspace:5 stretchy

[new entry][stretching] - operator.\u2922.infix (north east and south west arrow)
+lspace:5 rspace:5 stretchy

[new entry] - operator.\u2ADD\u0338.infix (nonforking with slash)
+lspace:5 rspace:5

[new entry][stretching] - operator.\u2B45.infix (leftwards quadruple arrow)
+lspace:5 rspace:5 stretchy

[new entry][stretching] - operator.\u2B46.infix (rightwards quadruple arrow)
+lspace:5 rspace:5 stretchy
(In reply to comment #65)
> OK, so I guess we should open a separate bug to fix the scale-stretchy-3
> reftest?

That was fixed here:
http://hg.mozilla.org/mozilla-central/rev/0e519f6626c7

(Please use a separate bug if providing any further operator dictionary patches, though.)
You need to log in before you can comment on or make changes to this bug.