1547801 - Bookmarks with the Russian tag name are incorrectly displayed in the Library

Status: RESOLVED FIXED

(Firefox :: Bookmarks & History, defect, P2)

Description

[Franky]

Attachment: 1.png

User Agent: Mozilla/5.0 (Windows NT 6.1; rv:61.0) Gecko/20100101 Firefox/61.0

Steps to reproduce:

I don't know when it started, but the first time I discovered this problem a year ago. My system language is Russian, I use the English version of the browsers - Firefox Quantum and Firefox Developer Edition. The problem with tags is observed in all versions of the browser that I checked this weekend, ranging from 61 to 66.

Actual results:

If the first letter of the tag in Russian is lowercase, the bookmark displayed. If the tag in Russian starts with a large letter, the bookmark is not displayed.

Expected results:

Tags that have a Latin name don't have this problem.

https://support.mozilla.org/ru/questions/1257346#answer-1216906

Comment 1

[Franky]

Comment on attachment 9061465 [details]
1.png

Example

Comment 2

[Mark Banner (:standard8)]

Thank you for the report. I've just checked and it does appear to be a regression - looks like bug 1421664 caused it.

Comment 3

[Mark Banner (:standard8)]

Oh, for posterity, the regression range is https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=dcd10220d55aea46db212314c46d25a96a7be243&tochange=f4fcdaef616807d1e31a849db0c60a3a4bdc8778

Comment 4

[Marco Bonardo [:mak]]

MAybe the same cause as Bug 1449939, probably strictly related.

Comment 5

[Mark Banner (:standard8)]

In the sql logs I noticed:

[Parent 4388: Main Thread]: D/mozStorage TRACE_STMT on 0x1256b77a0: 'SELECT b.fk, h.url, b.title AS page_title, h.rev_host, h.visit_count, h.last_visit_date, null, b.id, b.dateAdded, b.lastModified, b.parent, null AS tags , h.frecency, h.hidden, h.guid,null, null, null, b.guid, b.position, b.type, b.fk FROM moz_bookmarks b JOIN moz_places h ON b.fk = h.id WHERE NOT EXISTS (SELECT id FROM moz_bookmarks WHERE id = b.parent AND parent = 4) AND NOT h.url_hash BETWEEN hash('place', 'prefix_lo') AND hash('place', 'prefix_hi')  AND ( h.id  IN (SELECT bms.fk FROM moz_bookmarks bms JOIN moz_bookmarks tags ON bms.parent = tags.id WHERE tags.parent =  4  AND lower(tags.title) IN (  'Птицы'  )  GROUP BY bms.fk HAVING count(*) >=  1  ) ) ORDER BY 3 COLLATE NOCASE ASC'

Specifically AND lower(tags.title) IN ( 'Птицы' )

So we're comparing a lower case tag title with the upper case form.

I took a look at the code surrounding this, and found we're converting from utf-16 to utf-8 and then converting into lower case.

The function definitions around ToLowerCase for nsACString imply they only handle ascii. Hence, I'm just about to attach a patch to do the lower case transition first, and then convert to utf-8. This seems to fix the issue.

Comment 6

[Mark Banner (:standard8)]

Attachment: Bug 1547801 - Correctly handle case-insensitive matching in tag queries for international strings. r?mak

Comment 7

[Mark Banner (:standard8)]

Just to note, I think given this has been in for a while already, and we're getting late-ish for 67, we can wontfix it there and include it in 68.

Comment 8

[Franky]

(In reply to Mark Banner (:standard8) from comment #7)

Just to note, I think given this has been in for a while already, and we're getting late-ish for 67, we can wontfix it there and include it in 68.

Thank you!

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/139a838ca42f
Correctly handle case-insensitive matching in tag queries for international strings. r=mak

Comment 10

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/139a838ca42f