1595177 - Implement Input.insertText

Status: RESOLVED WONTFIX

(Remote Protocol :: CDP, enhancement, P2)

Description

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

insertText emulates inserting text that doesn't come from a key press, for example an emoji keyboard or an IME

Usage in gutenberg for non-ASCII characters e.g. await page.keyboard.type( ARABIC_ZERO ).

2019-10-10T23:59:38.976Z puppeteer:protocol SEND ► {"sessionId":"0AB032265F672A69A4358A9DA9861D76","method":"Input.insertText","params":{"text":"’"},"id":612}
2019-10-11T00:03:28.670Z puppeteer:protocol SEND ► {"sessionId":"0AB032265F672A69A4358A9DA9861D76","method":"Input.insertText","params":{"text":" "},"id":2736}
2019-10-11T00:16:28.614Z puppeteer:protocol SEND ► {"sessionId":"37908ECD3339B45F5FC2B68DBEC2246D","method":"Input.insertText","params":{"text":"’"},"id":216}
2019-10-11T00:55:53.777Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"١"},"id":45}
2019-10-11T00:55:53.901Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"٢"},"id":46}
2019-10-11T00:55:54.390Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"٠"},"id":51}
2019-10-11T00:55:59.811Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"٠"},"id":85}
2019-10-11T00:55:59.931Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"١"},"id":86}
2019-10-11T00:56:05.696Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"٠"},"id":124}
2019-10-11T00:56:06.071Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"١"},"id":127}
2019-10-11T00:56:11.581Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"٠"},"id":165}
2019-10-11T00:56:11.956Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"١"},"id":168}
2019-10-11T00:56:17.612Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"٠"},"id":208}
2019-10-11T00:56:17.979Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"١"},"id":211}
2019-10-11T00:56:18.458Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"٢"},"id":216}
2019-10-11T00:56:19.710Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"١"},"id":229}
2019-10-11T00:56:20.026Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"٠"},"id":232}
2019-10-11T00:56:25.618Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"١"},"id":274}
2019-10-11T00:56:26.110Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"٢"},"id":279}
2019-10-11T00:56:26.411Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"٠"},"id":282}
2019-10-11T00:56:26.981Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"٠"},"id":288}
2019-10-11T00:56:27.556Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"٠"},"id":294}
2019-10-11T00:56:28.128Z puppeteer:protocol SEND ► {"sessionId":"DF637B365B7F6A45680F41431C04648B","method":"Input.insertText","params":{"text":"٠"},"id":300}

Used in Puppeteer's page.keyboard.type and page.keyboard.sendCharacter

Comment 1

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

This isn't used that often. We had talked about it being P2, but upon closer inspection I think it's more of a P3.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Note that I also started to experiment with that when I was working on bug 1418995. Due to my recent system crash I don't have that code anymore. :/ If someone wants to work on it, I'm happy to provide some details as far as I can remember.

Comment 3

[Andreas Tolfsen ❲:ato❳]

This is the Chrome implementation we have to reverse-engineer:
https://cs.chromium.org/chromium/src/content/browser/devtools/protocol/input_handler.cc?type=cs&q=devtools+insertText&sq=package:chromium&g=0&l=627

It looks like it calls the IME subsystem in Chrome, but not sure what practical web-observable side-effects that causes. Maybe none, and we can just set the element’s own property value?

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

No, this will not work. As far as I can remember we have to do text composition. Details about IME can be found here:

https://developer.mozilla.org/en-US/docs/Mozilla/IME_handling_guide

Comment 5

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

(In reply to Andreas Tolfsen 「:ato」 from comment #3)

This is the Chrome implementation we have to reverse-engineer:
https://cs.chromium.org/chromium/src/content/browser/devtools/protocol/input_handler.cc?type=cs&q=devtools+insertText&sq=package:chromium&g=0&l=627

It looks like it calls the IME subsystem in Chrome, but not sure what practical web-observable side-effects that causes. Maybe none, and we can just set the element’s own property value?

From my own experiments a couple of months ago, I believe it synthesizes or triggers an InputEvent (not sure if that's a direct or indirect effect, though)

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Juggler actually uses the following code tight to a frame:

textInputProcessor() {
  if (!this._textInputProcessor) {
    this._textInputProcessor = Cc["@mozilla.org/text-input-processor;1"].createInstance(Ci.nsITextInputProcessor);
    this._textInputProcessor.beginInputTransactionForTests(this._docShell.DOMWindow);
  }
  return this._textInputProcessor;
}

textInputProcessor().commitCompositionWith(text);

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I just tried the above code and adding the following to the content Input domain makes it work:

  async insertText({text}) {
    const textInputProcessor = Cc["@mozilla.org/text-input-processor;1"]
      .createInstance(Ci.nsITextInputProcessor);
    textInputProcessor.beginInputTransactionForTests(this.docShell.DOMWindow);
    textInputProcessor.commitCompositionWith(text);
  }

There is still the question in how to handle sub frames.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

We are not going to implement this API for CDP.