969980 - Implement case-insensitive search correctly using case-folding

Status: RESOLVED FIXED

(Core :: Find Backend, defect)

Description

[Marios Titas]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:26.0) Gecko/20100101 Firefox/26.0 (Beta/Release)
Build ID: 20140106153735

Steps to reproduce:

Firefox does implement case-insensitive search but the implementation is wrong. It appears that firefox converts all text to lower case and then does ordinary search. While this does work correctly for most cases, it fails for others. The correct way to do that is to use the case-folding function which has been designed exactly for that purpose. In other words, if you want to determine if two strings s1 and s2 are the same in a case-insensitive way, then it is wrong to do it like so: toLower(s1)==toLower(s2). The correct way is caseFold(s1)==caseFold(s2).



Actual results:

In a page that contains the letter "σ" (U+03C3 GREEK SMALL LETTER SIGMA), I hit Ctrl+F and type the letter "ς" (U+03C2 GREEK SMALL LETTER FINAL SIGMA). Firefox fails to find the "σ" in the page. This is because toLower("σ")!=toLower("ς").


Expected results:

Searching for "ς" should also find all σ's. Google Chrome does that. Note that
caseFold("σ")==caseFold("ς").

Comment 1

[[:Aleksej]]

bug 812837 or bug 202251?

Comment 2

[Marios Titas]

This is definitely not related to bug 812837. The problem with Turkic i's is a separate issue.

I think that this is also not related to bug 202251; in that bug, the discussion centers around accents and diacritics. What I am talking about here is good old case insensitive search. Unlike the removing accents of accents, this should not be a contentious issue: if we want case insensitive search, we have to do it right, i.e. by using case folding instead of lowercasing.

Comment 3

[Alex Henrie]

Attachment: Bug 969980 - Use case folding for case-insensitive searches.

Comment 4

[Alex Henrie]

The patch I just uploaded adds a ToFoldedCase function and uses it instead of ToLowerCase, which resolves the problem with the letter sigma. Now, I realize that ToFoldedCase would have to return a string instead of a single char in order to support characters that become two characters when uppercased (for example, spaß = SPASS in German). However, I decided to not bother with that because I don't think we currently have a way to add that kind of complexity to nsFind without killing performance. I did put ToFoldedCase in the "internal API" section so that it can be changed in the future to return a string.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cfcbab38d04c
Use case folding for case-insensitive searches. r=Ehsan

Comment 6

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/cfcbab38d04c

Comment 7

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/cfcbab38d04c