1240208 - Give a hint to on-screen keyboards that the awesomebar accepts URLs (e.g. to display '/' and TLDs)

Status: VERIFIED FIXED

(Firefox :: Address Bar, defect)

Description

[Matthew N. [:MattN]]

STR:
1) Focus the awesomebar on a device with an on-screen keyboard e.g. Windows 10

Expected result:
The keyboard layout is optimized for URL (and search term) entry e.g. with '/', TLDs, '.', etc. accessible). The space bar should also be available to enter in multiple search terms.

Actual result:
The default keyboard layout is used

==

@inputmode="url"[1] is useful for this purpose but its support is currently lacking on desktop and may not have shipped in release (bug 857355).

@type="url" implies @inputmode="url" but also adds validation, trimming, invalid URL styling, and possibly other behaviour changes that may not be wanted for the awesomebar.

[1] https://html.spec.whatwg.org/#attr-fe-inputmode

Comment 2

[:Gijs (he/him)]

Attachment: MozReview Request: Bug 1240208 - use inputmode in XUL to get better hinting for IME and on-screen keyboards, r?masayuki

Review commit: https://reviewboard.mozilla.org/r/33049/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/33049/

Comment 3

[:Gijs (he/him)]

https://reviewboard.mozilla.org/r/33049/#review29813

::: widget/windows/WinIMEHandler.cpp:530
(Diff revision 1)
>    if (aHTMLInputType.IsEmpty() || aHTMLInputType.EqualsLiteral("text")) {
> +    if (aHTMLInputInputmode.EqualsLiteral("url")) {
> +      static const InputScope inputScopes[] = { IS_URL };
> +      scopes = &inputScopes[0];
> +      arraySize = ArrayLength(inputScopes);

FWIW, I am not thrilled with the repetitiveness of this code, but it seems the rest of the code does the same thing. I'm also not sure why it's written this way, so I kept it in roughly the same style.

I only implemented support for url, email, tel and numeric, as the others seem to relate to different types of text input that I don't know that we do existing hinting for (latin-name, katakana, ...), so that seemed a bit out of scope. I used https://html.spec.whatwg.org/multipage/forms.html#attr-fe-inputmode for reference. This table https://html.spec.whatwg.org/multipage/forms.html#concept-input-apply implies that they should also apply for type="password" but that didn't seem to make sense for any of url/email/tel/numeric (well, hmm, maybe for numeric?)

Comment 4

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)(away 4/27 - 5/6)]

Comment on attachment 8714370 [details]
MozReview Request: Bug 1240208 - use inputmode in XUL to get better hinting for IME and on-screen keyboards, r?masayuki

https://reviewboard.mozilla.org/r/33049/#review29915

I think that this patch is fine. But smaug should also review about exposing inputmode attribute in chrome. I feel it's no problem, but I'd like to check his opinion.

::: widget/windows/WinIMEHandler.cpp:530
(Diff revision 1)
>    if (aHTMLInputType.IsEmpty() || aHTMLInputType.EqualsLiteral("text")) {
> +    if (aHTMLInputInputmode.EqualsLiteral("url")) {
> +      static const InputScope inputScopes[] = { IS_URL };
> +      scopes = &inputScopes[0];
> +      arraySize = ArrayLength(inputScopes);

I think I or you should clean this up in other bug. But for now, this is okay.

Comment 5

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)(away 4/27 - 5/6)]

Comment on attachment 8714370 [details]
MozReview Request: Bug 1240208 - use inputmode in XUL to get better hinting for IME and on-screen keyboards, r?masayuki

Don't you mind to expose inputmode attribute of <input> element in chrome?

Comment 6

[Olli Pettay [:smaug][[e-mail]]]

Comment on attachment 8714370 [details]
MozReview Request: Bug 1240208 - use inputmode in XUL to get better hinting for IME and on-screen keyboards, r?masayuki

https://reviewboard.mozilla.org/r/33049/#review29973

I don't have problem with that, but since I'm not too familiar with all the relevant xul bits here, is url actually the right mode? I mean, location bar can be used for random searches too.
But I guess it should be good enough hint, and there isn't probably anything better.

Comment 7

[Pulsebot]

https://hg.mozilla.org/integration/fx-team/rev/7e8f5a46f8ba

Comment 8

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/7e8f5a46f8ba

Comment 9

[:Gijs (he/him)]

Comment on attachment 8714370 [details]
MozReview Request: Bug 1240208 - use inputmode in XUL to get better hinting for IME and on-screen keyboards, r?masayuki

Approval Request Comment
[Feature/regressing bug #]: Windows on-screen keyboard support
[User impact if declined]: missing ".com" and "/" keys on the on-screen keyboard when trying to use the location bar, making it quite annoying to do so for anything that isn't just a search. This was pointed out by a surprisingly high number of users on input, as well as being noted in a few bugs. This is essentially parity-edge, parity-chrome, parity-ie - all other browsers on Windows do the right thing here.
[Describe test coverage new/current, TreeHerder]: nope. :-(
[Risks and why]: medium to low. This is one of the slightly more involved patches regarding our OSK support, but equally, it makes a large difference, and it builds on features that we've had for years now, but that aren't yet enabled for web content (I'm not entirely sure why). This patch avoids enabling those features for web content, only enabling them for chrome (Firefox) documents, like browser.xul , in order to ensure we don't change behaviour on the web. I think this contains the risk sufficiently to be able to uplift to aurora beta (also considering release is still a month away).
[String/UUID change made/needed]: no.

Comment 10

[Sylvestre Ledru [:Sylvestre]]

Comment on attachment 8714370 [details]
MozReview Request: Bug 1240208 - use inputmode in XUL to get better hinting for IME and on-screen keyboards, r?masayuki

Improve the windows situation, taking it.

Comment 11

[Sylvestre Ledru [:Sylvestre]]

Should be in 45 beta 4.

Comment 12

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/7e8c5c720d9d

Comment 13

[Carsten Book [:Tomcat]]

has problems uplifting to beta:
grafting 327699:7e8c5c720d9d "Bug 1240208 - use inputmode in XUL to get better hinting for IME and on-screen keyboards, r=masayuki r=smaug, a=sylvestre"
merging browser/base/content/urlbarBindings.xml
merging dom/events/IMEStateManager.cpp
merging widget/windows/TSFTextStore.cpp
merging widget/windows/TSFTextStore.h
merging widget/windows/WinIMEHandler.cpp
merging widget/windows/WinIMEHandler.h
warning: conflicts while merging browser/base/content/urlbarBindings.xml! (edit, then use 'hg resolve --mark')
abort: unresolved conflicts, can't continue
(use hg resolve and hg graft --continue)

Comment 14

[:Gijs (he/him)]

https://hg.mozilla.org/releases/mozilla-beta/rev/cb8ee60c69f4

Comment 15

[Carsten Book [:Tomcat]]

setting flags

Comment 16

[Cornel Ionce [:noni] [Hubs QA]]

On a MS Pro 2 device running Windows 10 x64 using Firefox 44.0b1, build ID 2015121702820, the old keyboard is displayed.
The correct OSK is displayed on the same device using the following builds:
*latest 47.0a1 Nightly, build ID 20160217030229
*latest 46.0a2 Aurora, build ID 20160216004009
*Fx 45.0b6, build ID 20160215141016.