777529 - Superscript 1, 2, and 3 characters are marked as misspelled.

Status: RESOLVED FIXED

(Core :: Spelling checker, defect)

Description

[Brian Beloian]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:14.0) Gecko/20100101 Firefox/14.0.1
Build ID: 20120713134347

Steps to reproduce:

I typed the superscript 1, 2, or 3 (¹ ² ³) characters in a textarea.


Actual results:

The characters were marked as misspelled.


Expected results:

As numbers, they should not have been checked in the first place.

Comment 1

[(no longer active)]

Thanks for the bug report, Brian!

We even have a comment here about this: <http://mxr.mozilla.org/mozilla-central/source/extensions/spellcheck/src/mozInlineSpellWordUtil.cpp#952>  :-)

Simon: is there a proper Unicode way of determining this?

Comment 2

[Simon Montagu :smontagu]

mozilla::unicode::GetGenCategory(ch) == nsIUGenCategory::kNumber 
 
(that doesn't account for supplementary characters, but neither does the current code)

Comment 3

[(no longer active)]

(In reply to comment #2)
> mozilla::unicode::GetGenCategory(ch) == nsIUGenCategory::kNumber 
> 
> (that doesn't account for supplementary characters, but neither does the
> current code)

So that will only evaluate to true for '0'-'9'?

Comment 4

[Simon Montagu :smontagu]

Is that what you want? I thought you wanted to identify any number in any script.

There are three "Number" categories in the Unicode Character Database (UCDB): Nd, Decimal number; Nl, Letter number; and No, Other number.

Nd includes sets of digits from 0-9 used in a decimal positional system, like ASCII 0-9, Arabic ٠-٩, Devanagari ०-९, etc., etc., etc.

Nl includes things like Roman numerals, U+3007 IDEOGRAPHIC NUMBER ZERO etc.

No includes the subscript and superscript digits and fractions and all the circled numbers, parenthesized numbers and so on.

There should be complete lists of which Unicode code points are in which category at http://www.fileformat.info/info/unicode/category/Nd/list.htm http://www.fileformat.info/info/unicode/category/No/list.htm and http://www.fileformat.info/info/unicode/category/Nl/list.htm, but that site seems to be down right now.

mozilla::unicode::GetGenCategory(ch) == nsIUGenCategory::kNumber will evaluate to true for codepoints in any of those categories. For a more fine-grained test one could use mozilla::unicode::GetGeneralCategory(ch) and test against some combination of HB_UNICODE_GENERAL_CATEGORY_DECIMAL_NUMBER,  HB_UNICODE_GENERAL_CATEGORY_LETTER_NUMBER and HB_UNICODE_GENERAL_CATEGORY_OTHER_NUMBER.

Comment 5

[(no longer active)]

Thanks for the explanation.  I think we'll want to use mozilla::unicode::GetGenCategory(ch) == nsIUGenCategory::kNumber then.  Aryeh, can you please take a look at this?

Comment 6

[Aryeh Gregor (:ayg) (no longer with Mozilla)]

Attachment: Patch

Are reftests really the best way to test this?  If so, is there any good way to make sure the test isn't passing for some unrelated reason?  It seems fragile.  For instance, initially I forgot to add the focus() lines, and wouldn't have realized that made the test worthless if I didn't have a habit of reverting the code changes and making sure the test fails.

Comment 7

[(no longer active)]

Comment on attachment 647173 [details] [diff] [review]
Patch

Please name the reftest something like spellcheck-superscript.html to keep the names in sync with the rest of the spell checker reftests.  Also please use needs-focus.

In order to make the test more reliable, you should also add another version of the test which has another piece of text in the textarea (such as blahblah) which will get marked as misspelling, and you should mark the other one as !=.  This will make sure that you'll get no misspelling markers in the case of the super-script characters alone, and you'll get some when a misspelled text gets added to the textarea.

r=me with the above.

Comment 8

[Aryeh Gregor (:ayg) (no longer with Mozilla)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/a1db720c9c36

Comment 9

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/a1db720c9c36