Extend app manifest to describe a keyboard app. (IME app.)

RESOLVED FIXED

Status

Firefox OS
Gaia::Keyboard
P1
normal
RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: rudyl, Assigned: GaryChen)

Tracking

({dev-doc-needed})

unspecified
ARM
Gonk (Firefox OS)
dev-doc-needed
Dependency tree / graph

Firefox Tracking Flags

(blocking-b2g:koi+)

Details

(Whiteboard: [ucid:SystemPlatform1], [FT:System-Platform], [Sprint: 2])

** For V2 - Keyboard **

Here we describe the keyboard registry mechanism, i.e. use app. manifest to let the system know that it is an keyboard app.


1.  An Open Web App. would be considered as a keyboard app. if it contains mozKeyboard permission.


2.  How to describe that a keyboard app. can support multiple input modules?
[Draft] We can define this in app’s manifest with multiple entry points:

"entry_points": {
    "en": {
      "launch_path": "/input-en.html",
      "name": "English layout",
      "inputMode": ['text', 'url', 'number']
    },
    "es": {
      "launch_path": "/input-es.html",
      "name": "Spanish layout",
      "inputMode": ['text', 'url', 'number']
    }
}

Each entry point, like “en”, represent a specific keyboard layout or a input module that this keyboard app. supports.
We also extend the original entry_points design with “inputMode” property so that we could do some filter on which keyboard could support which input type (<input type=”xxx”>).
Blocks: 816869

Updated

5 years ago
Assignee: nobody → ehung
I move mozKeyboard.onfocuschange listener from keyboard/keyboard.js to system/keyboard_manager.js, and try to create and launch a iframe for a keyboard when an input field is getting focus. 
Now I'm getting stuck on bug 838547, we need to figure out why the focus can't be kept on the input field.

Updated

5 years ago
Depends on: 838547
You are probably going to need a bug for enabling entry points for third-party app developers. We originally turned it off for v1 on the basis of high risk vs. low value.

You'll need some bug like:

* Add support for entry points on app install
* Add support for entry points on app updates

Possibly more if there's others I'm not thinking of.
Thanks for reminding me. I will file these issues later.
We are going to use entry points to describe keyboard layouts, but this is a pure 
Gaia work: keyboard manager in System app parses keys and values to know all installed keyboards. So I guess it won't face any permission problem in third-party keyboard app install/updates.
Here is my proposed manifest of a third-party keyboard app:

  "entry_points": {
    "english": {
      "launch_path": "/index.html#en",
      "description": "English layout",
      "type_group": ["text", "number"]
    },
    "spanish": {
      "launch_path": "/index.html#es",
      "description": "Spanish layout",
      "type_group": ["text", "number"]
    },
    "number": {
      "launch_path": "/index.html#numberLayout",
      "description": "Number layout",
      "type_group": ["number"]
    }
  }

In this example, "english", "spanish", and "number" are different keyobard layout provide by this app. These keys are layout names, values are path to launch this layout, layout description, and support input type_group.

Note we won't use specific "inputMode" definition but use "type_group" instead.
The "type_group" intends to group all input types into a few groups, so we don't have to match exactly the same input type to launch a keyboard layout. This gives more flexibility to determine whether a layout could be displayed, and let keyboard app developers no need to enumerate possible input types.

We will submit patches within a couple of weeks, the patches will adapt to this new manifest definition. 
Any comments are welcome.
Note - if you plan on using entry_points as a way to define your third-party keyboard app, then you will need to re-enable the entry_points app property for third-party app developers again. Right now the property is only exposed to Gaia itself, not third-party app developers, since we originally deemed having this property exposed held high risk, low value.

To get entry_points re-enabled, you'll need to get basic flows with third-party apps working such as app install, app updates, etc.
Ack. Forgot about the fact I wrote that exact comment 5 in comment 2 a month back. Sorry for the duplicated comments.
I think that the keyboard app gets a special treatment so using entry points there does not mean we have to fix their support in general.
Hi Jason and Fabrice, thanks for the feedback. So let's use entry points first.
Whiteboard: u=user c=keyboard s=ux-most-wanted
QA Contact: wachen → whsu
Whiteboard: u=user c=keyboard s=ux-most-wanted

Updated

4 years ago
Blocks: 891669
OS: Mac OS X → Gonk (Firefox OS)
Hardware: x86 → ARM
Assignee: ehung → gchen
Setting every app that needs keyboard permission as a keyboard app is going to break the current UITest application. In bug 892397 there is also an ongoing discussion about how to distinguish services/keyboards/apps from each other.
See Also: → bug 892397

Updated

4 years ago
See Also: → bug 898786
I've updated on IME wikipage about locale issues in each keyboard layout. (https://wiki.mozilla.org/WebAPI/KeboardIME#IME_App_Manifest_Example)

This conclusion is based on https://bugzilla.mozilla.org/show_bug.cgi?id=898786#c2.

If there is anything I missed, please do not hesitate to talk on this bug.
Thanks~

Updated

4 years ago
Whiteboard: [ucid:SystemPlatform1], [FT: System Platform], [Sprint: 2]

Updated

4 years ago
blocking-b2g: --- → koi+
Priority: -- → P1
When trying to submit a testing keyboard app to marketplace dev,

I found the following errors around "keyboard" permissions,
https://marketplace-dev.allizom.org/developers/upload/9c44c0d1f0084b03a7e074b49d696975


Dear Wil,

Do you know who should I talk to to get this addressed?
Thank you.
Flags: needinfo?(clouserw)
(In reply to Rudy Lu [:rudyl] from comment #11)
> When trying to submit a testing keyboard app to marketplace dev,
> 
> I found the following errors around "keyboard" permissions,
> https://marketplace-dev.allizom.org/developers/upload/
> 9c44c0d1f0084b03a7e074b49d696975
> 
> 
> Dear Wil,
> 
> Do you know who should I talk to to get this addressed?
> Thank you.

You want Matt in this case. This is probably occurring right now because we haven't updated the validator for keyboards yet, so we should sync up with him to address the immediate concerns to get testing unblocked here.

redirecting to Matt here as such
Flags: needinfo?(clouserw) → needinfo?(mattbasta)

Comment 13

4 years ago
https://github.com/mozilla/app-validator/commit/54ed3ccb0711b334711dfa4382db6c9ed1da1dff

done
Flags: needinfo?(mattbasta)
Jason,

Thanks for redirecting the issue to the right person.

Matt,

Thanks for your quick response!
May I know when we could get this deployed to the dev. marketplace so that we can start the testing?
Flags: needinfo?(mattbasta)
(In reply to Rudy Lu [:rudyl] from comment #14)
> Jason,
> 
> Thanks for redirecting the issue to the right person.
> 
> Matt,
> 
> Thanks for your quick response!
> May I know when we could get this deployed to the dev. marketplace so that
> we can start the testing?

Rudy - What's the name of the specific app here you are trying to deploy? I can push it if you can give me a name of the app.

Note - the app has to pass the validator to be allowed to be deployed.
Flags: needinfo?(mattbasta)

Comment 16

4 years ago
https://github.com/mozilla/zamboni/commit/73852a9f05f48378dfad1776f9f685f4556e82bd

done
Whiteboard: [ucid:SystemPlatform1], [FT: System Platform], [Sprint: 2] → [ucid:SystemPlatform1], [FT:System-Platform], [Sprint: 2]
As discussed with Marcos and Rudy in bug#898786, as following link is our new keyboard manifest structure.
https://wiki.mozilla.org/WebAPI/KeboardIME#Proposed_Manifest_of_a_3rd-Party_IME

3rd-party keyboard apps will expose their supported IME under "entry_points", and each IME has their own manifest under this entry which includes "name", "description", "locales" etc... 

The important thins is every IME need to declare which input types they want to support in "types" field. 
  for example:"types": ["url", "text"],
              "types": ["number"],
              "types": ["password"],  

I close this bug in this phase, if anyone have question we(Eveyln, Tim, Rudy and GaryChen) are looking forward to reopen this bug and discuss with you in bugzilla or IRC #b2g channel.
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED

Comment 18

4 years ago
in https://wiki.mozilla.org/WebAPI/KeboardIME#Proposed_Manifest_of_a_3rd-Party_IME
"type": "certified",
"permissions": {
   "settings": { "access": "readwrite" },
   "keyboard": {}
 },

they all for mozilla and device manufacturer, so does the KeyboardIME open to thrid party developers? I have port my IME to Firefox OS,  tested on b2g desktop, want to publish it to my users.

(In reply to dgod.osa from comment #18)
> in
> https://wiki.mozilla.org/WebAPI/KeboardIME#Proposed_Manifest_of_a_3rd-
> Party_IME
> "type": "certified",
> "permissions": {
>    "settings": { "access": "readwrite" },
>    "keyboard": {}
>  },
> 
Thanks your remind, I've updated in wiki.
Please check it again.
https://wiki.mozilla.org/WebAPI/KeboardIME#Proposed_Manifest_of_a_3rd-Party_IME

Keyboard api will expose to privilege when bug 901434 is fixed.

> they all for mozilla and device manufacturer, so does the KeyboardIME open
> to thrid party developers? I have port my IME to Firefox OS,  tested on b2g
> desktop, want to publish it to my users.

Really happy to heard this news, good job. \o/

Updated

4 years ago
Blocks: 912852
Uh didn't we rename keyboard -> inputmethod permission?
(In reply to Jan Jongboom [:janjongboom] from comment #20)
> Uh didn't we rename keyboard -> inputmethod permission?

Not until bug 901434 lands.
Alright, didn't realize that wasn't landed yet.

Updated

4 years ago
Keywords: dev-doc-needed
I am working on bug 944641 to ensure all Marketplace part of the Mozilla infra is ready for receiving input apps (and the first app for testing the installation process).
You need to log in before you can comment on or make changes to this bug.