Status

defect
P1
normal
RESOLVED FIXED
7 years ago
7 years ago

People

(Reporter: ghtobz, Assigned: mounir)

Tracking

({feature})

unspecified
B2G C1 (to 19nov)
Dependency tree / graph

Firefox Tracking Flags

(blocking-basecamp:+, firefox18 fixed, firefox19 fixed)

Details

(Whiteboard: [label:Keyboard & IME][label:mentored][LOE:S] QARegressExclude)

Attachments

(3 attachments)

[GitHub issue by mounirlamouri on 2012-08-27T14:23:37Z, https://github.com/mozilla-b2g/gaia/issues/3857]
inputmode has just be added to the input element. We might prefix it but as soon as the decision of prefixing (or not) is done, we should have the VKB using inputmode to decide whether to show uppercase/lowercase/titlecase/whatever.

See bug 746142.

CC @etiennesegonzac
[GitHub comment by autonome on 2012-09-25T03:43:26Z]
/cc @RudyLu
[GitHub comment by autonome on 2012-09-26T05:47:10Z]
@RudyLu or @mounirlamouri  can you explain what's needed to fix this, so other people can work on it?
Assignee: nobody → cpeterson
Status: NEW → ASSIGNED
Whiteboard: [label:Keyboard & IME][label:mentored] → [label:Keyboard & IME][label:mentored][LOE:S]
The keyboard does not get any direct access to the element it is providing input for, but the 'focuschange' (terrible name) event it gets from mozKeyoard carries a lot of information with it, including the full content of the input field and the type attribute of the input field.  You'll need to modified b2g/chrome/content/forms.js to pass the inputmode value as well.  That file passes it to mozKeyboard, which will automatically include it in the focuschange event.

The actual implementation of the behaviors described by the inputmode attribute goes deep into the predictive text engine stuff that I've been working on.  I have a meta-bug filed as #797155 tracking a number of things that will end up interacting with this.

If you'd like you could treat this bug as just getting the input mode value into the mozKeyboard.onfocuschanged event, and then allow me to hack on the text prediction stuff to honor the inputmode value that is passed.
Assignee: cpeterson → dflanagan
With cpeterson's permission, I've assigned this to myself.

I'm going to fold the necessary forms.js changes into my patch for #796080, and have marked this bug as depending on that one.
Depends on: 796080
Blocks: 797867
Adding this as another child bug of my text predictions meta-bug
Blocks: 797155
David, do you have time to fix this asap?  I've received lots of feedback that users aren't able to search and it's a major pain point. 

Can we just go with your original proposal so we have a solution in place?  

Replace "/" with a space bar given it's available in the alternate keyboard already.
(In reply to Chris Lee [:clee] from comment #6)
> David, do you have time to fix this asap?  I've received lots of feedback
> that users aren't able to search and it's a major pain point. 
> 
> Can we just go with your original proposal so we have a solution in place?  
> 
> Replace "/" with a space bar given it's available in the alternate keyboard
> already.

I think you are in the wrong bug.
Chris, this bug is about adding support for inputMode so that the keyboard can be told whether to capitalise the first letter of text entered (and other similar options).

This would allow the browser app to use the standard text keyboard instead of the url keyboard for the awesomebar (without capitalising the first letter), which is being discussed in https://github.com/mozilla-b2g/gaia/issues/3566
Chris,

If we need a temporary workaround to the awesome bar problem (#797867), we probably could add a space to the URL keyboard layout until this bug is fixed.  If you want to propose that in that bug, and get it marked blocking, I think the keyboard hack would be easy to do.

This bug is blocked on #796080 which is marked "resolved" but needs to land in the aurora tree before anyone will be able to use it.  Getting something to happen on #796080 and fixing it right would be better, though.
The HTML spec leaves the default input mode up to the UA. So what should our default behavior be if inputmode is not set?

I assume that we want predictions on by default.  Authors will have to set input mode to turn that off.

What about capitalization at the start of sentences?  Also on by default?  I'm going to assume so, unless I hear otherwise.
So it turns out that the patch I made for #796080 doesn't work. Input elements support the inputmode attribute, but only for attribute values (like 'digit') that gecko recognizes. If you set inputmode="verbatim" on an input field, and then query the inputmode property, you'll get "auto".  This means that I've got make another patch to forms.js and manage to get it checked into m-c and m-a.  I'm testing out a patch now.
Well darn. Now the keyboard just isn't working, so I can't test my patch.
I've got a gecko (forms.js) patch ready to fix the inputmode thing described above.  And I've got a corresponding gaia patch that turns off auto-capitialization for "verbatim" and "latin" modes.  That's just a start, but it is enough to fix #797867.

But I can't test my patches because I have to build from m-c and #800817 has made the entire keyboard unusable with the latest builds.
My previous patch was always sending the value "auto" to gaia. This corrects it to correctly query inputmode.
Attachment #670958 - Flags: review?(cpeterson)
The attached patch corrects that code that was sending the inputmode value to the MozKeyboard.  

The related pull request at https://github.com/mozilla-b2g/gaia/pull/5796 modifies the Gaia keyboard to use inputmode. It doesn't implement all of the features, but it turns off autocapitalization for verbatim and latin modes, which is what we'll need to fix the browser url bar.
Comment on attachment 670958 [details] [diff] [review]
b2g chrome patch to pass the inputmode attribute to the gaia keyboard

Review of attachment 670958 [details] [diff] [review]:
-----------------------------------------------------------------

LGTM!
Attachment #670958 - Flags: review?(cpeterson) → review+
asking for checking r=cpeterson a=blocking-basecamp
Keywords: checkin-needed
Is this supposed to be going into aurora too? Please note that in the whiteboard when requesting checkin if that's the case.
Whiteboard: [label:Keyboard & IME][label:mentored][LOE:S] → [label:Keyboard & IME][label:mentored][LOE:S][checkin-needed-aurora]
https://hg.mozilla.org/integration/mozilla-inbound/rev/137d665dd6d9
https://hg.mozilla.org/releases/mozilla-aurora/rev/3374394cc092

(Leaving open until this lands on m-c)
Keywords: checkin-needed
Whiteboard: [label:Keyboard & IME][label:mentored][LOE:S][checkin-needed-aurora] → [label:Keyboard & IME][label:mentored][LOE:S]
Thanks Ryan.  I thought I had to wait 'till it landed in m-i before I requested m-a checkin.  In the future, I'll just set checkin-needed-aurora at the same time that I set checkin-needed
https://hg.mozilla.org/mozilla-central/rev/137d665dd6d9
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Reopening this because the patch that landed was only a partial implementation of inputmode.  Just enough to enable a fix to the browser URL bar capitialization problem.  I still need to support input mode for word suggestions.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
David, before landing anything in Gaia or B2G that will make any other inputmode values from Gecko to be recognized, please make sure you spoke with someone from the WebAPI or DOM team about that.
Oh, I see this has already be done...

https://github.com/mozilla-b2g/gaia/pull/5796 needs to be backed out. We can't land that kind of stuff: it will modify web facing features without the consent of any person working on that kind of stuff and it will also have a completely incoherent behaviour: setting inputmode to some value will have an effect in Gecko but not in Gaia and some other values will be no-op in Gecko but will change the virtual keyboard.

We should whether fully move to what Hixie specified or keep our current design but we should clearly not have a part of our code following a design and another part following another design.
Mounir,

My understanding was that gecko doesn't currently do anything with inputmode except incorrectly reflect it through the inputmode property. I'm not convinced that there is any harm in implementing it to spec in the vkb.

In any case, there isn't time in the v1 schedule to coordinate this with anybody.  We need something now. If I used mozinputmode would you be okay with that?
For all features exposed to the web we need to make sure that we are implementing something that we are happy to support for a very long time.

Mounir: what exact input modes do you recommend we support for now? I would expect this list to be heavily influenced by what works in other browsers since that's most likely what content out there is using.
For what its worth, my plan was to implement these modes (from the HTML spec) for the Gaia virtual keyboard:

"verbatim": no capitalization, no word suggestions
"latin": word suggestions but no capitalization
"latin-prose": word suggestions and capitalization at the start of sentences

I'd also consider implementing "latin-name" to provide word suggestions and capitalize each word, but since we wouldn't be offering completion from contacts lists, I probably wouldn't do this one at all, or would make it a synonym for "latin-prose".

https://bugzilla.mozilla.org/show_bug.cgi?id=797867 demonstrates that we need this ability to turn off auto-capitalization on individual input fields, so we need a feature like this. I'm happy to use a prefix if that allows us to move more quickly on this.
(In reply to David Flanagan [:djf] from comment #25)
> Mounir,
> 
> My understanding was that gecko doesn't currently do anything with inputmode
> except incorrectly reflect it through the inputmode property.

Gecko has some inputmode values, they are just not the one Hixie spec'd. This explains why it is not reflecting correctly for you.

> I'm not
> convinced that there is any harm in implementing it to spec in the vkb.

Firefox Android is currently using the inputmode as currently implemented by Gecko.
Okay, so what does gecko currently implement?
Flags: needinfo?(mounir)
I've given up waiting for Mounir to respond with somethign less cryptic and did my own research.

inputmode is not documented on MDN. The only thing I found about it was a link to bug 746142

I just read through that bug.  The person who did the patches used values similar to those in the old XForms spec. Then Hixie added inputmode to the HTML spec (and mentioned it in the bug)  Then after that the patches were landed without updating them to match the spec. They didn't land in m-c until 8/27, which I assume mean that this is still Aurora, and has not made it to beta. 

The values are undocumented, and have never made it even to beta, and they differ egregiously from the values that were in the HTML spec when they landed.  This means that no one is depending on the existing Fennec behavior yet, and that it would be ridiculous to keep that behavior since it violates the spec.

So I suggest that we should just update Fennec to match the spec and then just match the spec in Gaia as well.

In an attempt to answer Jonas' question about what works in other browsers, I tried googling "inputmode attribute" and all I really found was references to the XForms spec and Opera's implementation from 2008.  Maybe I'm not looking in the right place, but it doesn't sound like this is in active use at all.

If we want maximum flexibility, we could implement the attribute so that it takes a space-separate list of tokens, as the XForms spec requires, and then we could implement both the HTML spec values and the XForms spec values, which includes things like "predictOn" and "predictOff" and is closer to what we currently have. See http://www.w3.org/TR/2006/REC-xforms-20060314/sliceE.html for the XForms values.

Still, I suggest we just stick to the spec for now. If we have to support other values for web compat we can add that, but we should start with what Hixie has proposed first.

Jonas, Mounir: what do you think of this plan?  Who can work on Fennec to change inputmode before the outdated attribute values make to to Beta?  And can I proceed to implement the Gaia keyboard to spec?
Flags: needinfo?(mounir)
Thank you for your patience David. Bug 746142 comment 9 list the supported values and the expected behaviour in plain words.

For information, the HTML Live specifications are not words of gods. The background of this feature is that we (Mozilla) wanted to have it done so we did implement something. Hixie heard about that so he had a look at it and wrote something in the specifications. What is in the specifications is not the result of a group discussion finding a consensus in the less bad proposal. It is one person thinking.
So, to make it clearer, at the WHATWG, here are how things work (roughly):
1.
 a. A UA implements something, Hixie specs it, OR
 b. Hixie specs something and a UA implement it, OR
 c. Hixie refuses to spec something but a UA still implements it;
2.
 a. UAs follow the initial implementation, OR
 b. UAs have different implementation (following the spec, the first implementation or the moon rotation), OR
 c. No other UA than the first one have an implementation;
3.
 a. Ideally, a consensus is found after some point, specs are modified if needed, UA matches the spec, OR
 b. It's a mess.

We are currently at the first step. In other words, our implementation as much importance as the spec if not more because depending on it, other UA might have a choice between following Hixie's lead or our lead. Note that with that model, UAs have more power than the specifications because two matching implementations will very likely put the implementation as a de-facto standard and the specifications will have no choice than matching.

Also, there is indeed no web compat here because we are the only UA doing something in that area.

Finally, I'm trying to think about what we should ship. It will unlikely be what we are currently doing but clearly not the 100% of what the specs is requesting.
Thanks for the clarification, Mounir. I assumed that Hixie's spec was based on at least some public discussion on the mailing list.  It sounds like it is early enough in the process that we should use a prefix on the attribute values.

The use case that I find interesting for inputmode is to give content authors control over auto capitalization and auto correction (or word suggestion) on a field-by-field basis.  I suggest that inputmode be only for hints to virtual keyboards and input methods.  

Of the existing values that gecko supports now:

- 'numeric': 0-9, +, -, comma, dot;

The HTML spec has this too, but it really seems to me that this would be better as a new value of the type attribute and not an input mode.

- 'digit': 0-9 only;

This seems like <input type="number">, right?

- 'uppercase': A-Z only
- 'lowercase': a-z only

Apps can achieve these by filtering the user's input; it seems unnecessary to put a virtual keyboard into a special mode for this. These might be filters or validation constraints, but they're not input modes.

- 'titlecase': uppercase character for each new word [1]

Hixie proposes this feature for latin-name. How would this work on the desktop with a traditional physical keyboard? Would it alter the user's input? I think it should not, and so it is a type of auto-capitalization

- 'autocapitalized': first letter is uppercased

This is like 'latin-prose'. I assume we also auto capitalize at the start of sentences as well.  And, I assume that this only applies to virtual keyboards.  It doesn't actually alter the user's input, it just automatically engages the shift key when it thinks that the user wants that.

We also clearly need values that control auto correction and the display of word suggestions.  Hixie's version has them and ours does not.

I'd say that the spec is closer to useful than what we've got now.  How about moz-verbatim, moz-latin, moz-latin-prose and moz-latin-name to start?
(In reply to David Flanagan [:djf] from comment #32)
> - 'numeric': 0-9, +, -, comma, dot;
> 
> The HTML spec has this too, but it really seems to me that this would be
> better as a new value of the type attribute and not an input mode.

This is equivalent to <input type='number'> somehow. Except that the type will come with a UI.

> - 'digit': 0-9 only;
> 
> This seems like <input type="number">, right?

No. See above.

> - 'titlecase': uppercase character for each new word [1]
> 
> Hixie proposes this feature for latin-name. How would this work on the
> desktop with a traditional physical keyboard? Would it alter the user's
> input? I think it should not, and so it is a type of auto-capitalization

This is not as simple as 'titlecase' <-> 'latin-name'. 'latin-name' is one use case 'titlecase' would solve but 'latin-name' isn't exactly only about using title case and 'titlecase' isn't only for names. For example, 'latin-name' except the UA to suggest names from the contact list and have names prefix not capitalized when required. Not as simple as 'titlecase'.

In my opinion, we should support:
- verbatim,
- latin,
- latin-prose,
- digit

cjk modes is out of our scope for the moment. 'tel', 'email' and 'url' modes do not look particularly useful. I doubt we even want them (for the moment at least). 'numeric' seems to have less use cases than 'digit'.

My main problem is to figure out what to do with latin-name. Seems like a lot of apps might want it but I'm afraid this could easily be abused to do titlecase. In the other hand, if we only have latin-titlecase but no latin-name, we break part of the intentions of the specs...

Oh, also. All input elements should default to 'latin' (with the variation regarding their type). Textarea elements should default to 'latin-prose'. IOW, we shouldn't uppercase the first letter of input fields unless this is asked.
No longer blocks: 797867
Okay, this sounds good. 

I know exactly how to support verbatim, latin and latin-prose in the latin input method of the Gaia vkb.

I don't know what, if anything, I am supposed to do with 'digit' mode. That probably won't be supported for v1.

Is there a corresponding bug for the Fennec keyboard?
The latin input method of the gaia vkb offers three kinds of typing assistance:

- word suggestions
- auto capitalization
- punctuation help.  space space turns into period space and space period (after a suggestion is inserted) turns into period space.

The HTML spec is pretty clear about how inputmode affects suggestions and capitalization.  But I'm not sure about punctuation.  I'm going to implement it in latin-prose only.  Mounir: let me know if you think this should be the default for latin as well.
(In reply to Mounir Lamouri (:mounir) from comment #33)...
> 
> Oh, also. All input elements should default to 'latin' (with the variation
> regarding their type). Textarea elements should default to 'latin-prose'.
> IOW, we shouldn't uppercase the first letter of input fields unless this is
> asked.

Right now, actually, my gaia code uses verbatim input mode if the type is anything other than "text" or "textarea". I don't think we want suggestions during email address or url or password entry, for example. (Should I handle type=search specially, though?)

For text and textarea, I allow inputmodes "verbatim", "latin" and "latin-prose".  If the attribute is not specified, or is any other value, then it default to "latin" for type=text and "latin-prose" for type=textarea.

Mounir: does that sound like what you want? Are there any input types that you really do want me to default to latin?
https://github.com/mozilla-b2g/gaia/pull/6007 makes the Gaia vkb honor modes "verbatim", "latin" and "latin-prose", with textarea defaulting to latin-prose and type="text and type="search" defaulting to "latin".  type="password", type="url" and all other input fields use verbatim mode and ignore the inputmode attribute.

Mounir: does that sound right? If you're satisfied with my implementation, please close this issue (and open a new one for the matching implementation in Fennec, I suppose).
Flags: needinfo?(mounir)
Priority: -- → P1
We're marking this bug with the C1 milestone since it follows the criteria of "unfinished feature work" (see https://etherpad.mozilla.org/b2g-convergence-schedule).

If this work is not finished by Nov19, this bug will need an exception and will be called out at the upcoming Exec Review.
Target Milestone: --- → B2G C1 (to 19nov)
Keywords: feature
Depends on: 809865
(In reply to David Flanagan [:djf] from comment #37)
> https://github.com/mozilla-b2g/gaia/pull/6007 makes the Gaia vkb honor modes
> "verbatim", "latin" and "latin-prose", with textarea defaulting to
> latin-prose and type="text and type="search" defaulting to "latin". 
> type="password", type="url" and all other input fields use verbatim mode and
> ignore the inputmode attribute.
> 
> Mounir: does that sound right? If you're satisfied with my implementation,
> please close this issue (and open a new one for the matching implementation
> in Fennec, I suppose).

We talked about that last week with Jonas and we were not able to find out a solution that we thought was good so I think the best solution here is to hide the feature from the web and do whatever we want for Gaia. I opened a bug and wrote a patch to disable inputmode for Firefox 17 and Firefox 18 and I think Gaia should use x-inputmode attribute (there will be no IDL implementation).

I understand that changing inputmode to x-inputmode might be annoying but I think that is a good compromise between not having a crappy UI and not messing up with the Web. Hopefully grep and sed should be enough.
Flags: needinfo?(mounir)
What needs to happen in this bug now?
Summary: [vkb] use inputmode → [vkb] use x-inputmode
Attachment #680429 - Flags: review?(21)
Attachment #680429 - Attachment description: Use x-inputmode → Use x-inputmode in forms.js
Attachment #680430 - Flags: review?(21)
(In reply to Dietrich Ayala (:dietrich) from comment #40)
> What needs to happen in this bug now?

Land the two patches in attached.
Comment on attachment 680429 [details] [diff] [review]
Use x-inputmode in forms.js

Review of attachment 680429 [details] [diff] [review]:
-----------------------------------------------------------------

::: b2g/chrome/content/forms.js
@@ +285,5 @@
>    // in in Gaia, and the inputmode property returns "auto" for any value
>    // that gecko does not support. So we must query the inputmode attribute
>    // with getAttribute() rather than just using the inputmode property here.
>    // See https://bugzilla.mozilla.org/show_bug.cgi?id=746142
> +  let inputmode = element.getAttribute('x-inputmode');

The comment sounds out-of-date now.

Also there is already x-moz-errormessage and mozactionhint as non-standard extensions. I'm not used to non-standards naming but we should still be consistent in some ways no? (or is it on purpose?)
David says there is no further work required for him on this one -- reassigning to :mounir.
Assignee: dflanagan → mounir
Comment on attachment 680430 [details] [diff] [review]
Use x-inputmode in Gaia

Review of attachment 680430 [details] [diff] [review]:
-----------------------------------------------------------------

Note that #809686 and #809695 are both adding more inputmode attributes. I've made notes in those two bugs that they should be changed to use x-inputmode.

::: apps/email/index.html
@@ +377,4 @@
>          </h2>
>          <div class="sup-form">
>            <input class="sup-info-name sup-form-text" type="text"
> +                 x-inputmode="titlecase" placeholder="NamE" required />

titlecase?  We don't support that in Gaia, and I don't think there is any plan (or any bug filed) to support it for v1.

Wouldn't it be better to use name here to at least attempt to follow the spec? 

That is probably out of scope for this bug, however.

::: test_apps/uitest/tests/inputmode.html
@@ +30,4 @@
>  
>      none: <textarea></textarea><br>
> +    latin: <textarea x-inputmode="latin"></textarea><br>
> +    verbatim: <textarea x-nputmode="verbatim"></textarea><br>

typo here: missing an i
Mounir,

Thanks for these patches. I would give the gaia patch an r+ (except for the typo) if you asked me.
Comment on attachment 680429 [details] [diff] [review]
Use x-inputmode in forms.js

Review of attachment 680429 [details] [diff] [review]:
-----------------------------------------------------------------

As Vivien notes, please remove the comment, or maybe replace it with a comment explaining the x- prefix.

Also please confirm that an 'x-' prefix is what we want here instead of 'moz'.  We are implementing the current spec, so its not exactly experimental and a moz prefix might be okay.
Attachment #680429 - Flags: review?(21) → review+
Comment on attachment 680430 [details] [diff] [review]
Use x-inputmode in Gaia

Review of attachment 680430 [details] [diff] [review]:
-----------------------------------------------------------------

Please fix the typo in inputmode.html and, if those other two input mode bugs have landed, you'll need to update those files, too.  Otherwise, let's get this landed and close the bug!
Attachment #680430 - Flags: review?(21) → review+
Also, Mounir, when you land these patches, please send email to dev-gaia about the change to x-inputmode and to warn people about the version skew issue where auto capitialization and word suggestions might now work as expected until gecko and gaia are in sync again.
Typo fixed. Comment updated. Email sent to dev-gaia.
Thanks Mounir! Closing the bug now.
Status: REOPENED → RESOLVED
Closed: 7 years ago7 years ago
Resolution: --- → FIXED
The bug will be marked RESOLVED FIXED when the patch will land in m-c. This is bugzilla's process.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Ah, right.  Sorry.  I'm used to marking resolved after landing patches on github.
https://hg.mozilla.org/mozilla-central/rev/d2e0669c3baa
Status: REOPENED → RESOLVED
Closed: 7 years ago7 years ago
Resolution: --- → FIXED
Whiteboard: [label:Keyboard & IME][label:mentored][LOE:S] → [label:Keyboard & IME][label:mentored][LOE:S] QARegressExclude
You need to log in before you can comment on or make changes to this bug.