1672680 - Search shortcuts can not use IDEOGRAPHIC SPACE(U+3000)

Status: VERIFIED FIXED

(Firefox :: Address Bar, defect, P2)

Description

[open_ffff23]

Attachment: search.png

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:83.0) Gecko/20100101 Firefox/83.0

Steps to reproduce:

address bar enter
"b　firefox"
"(shortcuts key)(U+3000)(search keyword)"

Actual results:

Searched with the default search engine instead of the "shortcut key" search engine

Expected results:

As well as SPACE(U+0020)

"b firefox"
"b　firefox"

Search on the search engine where the shortcut key is registered.

Comment 1

[Harry Twyford]

Thanks for the report. Do you have a use case where U+3000 is commonly used?

Comment 2

[Drew Willcoxon :adw]

U+3000 is an ideographic space character in CJK. The reporter's screenshot shows that they're using a Japanese build, and U+3000 is the space character you get when you type a space while in Japanese-language mode in your IME. So this is probably actually pretty common for CJK users, and we should fix it. The screenshot appears to show that this used to work in 82, so I'll mark this as a regression.

Comment 3

[Drew Willcoxon :adw]

Here's the problem (which I created): https://searchfox.org/mozilla-central/rev/ff85081e3b05f89f9a5921b489b6e01ed4a88c25/toolkit/components/places/UnifiedComplete.jsm#1095 So this is a regression due to search mode, which is why it worked in 82.

We (I) should probably audit all calls to startsWith(" ") and similar. We should be using regexps with \s.

Comment 4

[Drew Willcoxon :adw]

Same problem here, which is also my fault: https://searchfox.org/mozilla-central/rev/ff85081e3b05f89f9a5921b489b6e01ed4a88c25/browser/components/urlbar/UrlbarProviderSearchSuggestions.jsm#493

Comment 5

[Drew Willcoxon :adw]

Attachment: Bug 1672680 - Properly handle non-ASCII spaces in search alias and restriction token searches.

This fixes all the problems created by bug 1658964 and modifies existing tests
so we check a variety of spaces.

Comment 6

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

[Tracking Requested - why for this release]: keywords/aliases/searchmode malfunction for some locales

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7d86a83672c5
Properly handle non-ASCII spaces in search alias and restriction token searches. r=mak

Comment 8

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/7d86a83672c5

Comment 9

[Alexandra Martin]

Confirmed as verified fixed on Windows 10 x64, MacOS 10.14 and Ubuntu 20.04 x64 for Nightly 84.0a1 (20201026092834), en-US and ja versions, after a "b" shortcut was added for Bing in about:preferences#search.

Comment 10

[Drew Willcoxon :adw]

Comment on attachment 9183388 [details]
Bug 1672680 - Properly handle non-ASCII spaces in search alias and restriction token searches.

Beta/Release Uplift Approval Request

• User impact if declined: CJK users will see a regression where the space character they type with their IME will no longer trigger search shortcuts in the urlbar.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: This bug has already been verified on 84 in comment 9, but for STR, please see comment 0. You can copy and paste the quoted strings in that comment into the urlbar.
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This isn't a small patch, but most of it is tests, and the core changes are small. We now have lots of tests for this, and of course we have lots of urlbar tests in general.
• String changes made/needed:

Comment 11

[Pascal Chevrel:pascalc (PTO until April 26)]

Comment on attachment 9183388 [details]
Bug 1672680 - Properly handle non-ASCII spaces in search alias and restriction token searches.

Approved for 83 beta 5, thanks.

Comment 12

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/9f4696b061ac

Comment 13

[Alexandra Martin]

Confirmed as verified fixed on Beta 83.0b5 (20201027175448) en-US and ja versions, on Windows 10 x64, MacOS 10.14, and Ubuntu 20.04 x64.