1649519 - Support AArch64 macOS for IME

Status: RESOLVED FIXED

(Core :: DOM: UI Events & Focus Handling, enhancement, P3)

Description

[Gian-Carlo Pascutto [:gcp]]

Comments from Masayuki:

My worry is just guessing, I've not investigated about the new macOS.

When previous Mac's architecture change happened (i.e., from PowerPC to Intel), IMEs were needed to rebuild. So, I guess that at this time, it may happen again. If it's true, this might be a chance to Apple's developers to sort out API around keyboard/IME because most of them were introduced at 10.5 (13 years ago), and they are too old design for now (e.g., it's hard to implement async input handling like browsers). Therefore, I worry about that we might need to work on something for new macOS to keep handling keyboard/IME.

Now, our text input handler, TextInputHandler.mm, works with 4 types of API.

* Cocoa framework (NSEvent and NSResponder's interpretKeyEvents: to send key events to IME)
* Text Input Services that are C style API whose name starts with "TIS" (to manage active keyboard layout/IME)
* Legacy API, UCKeyTranslate() (to compute `KeyboardEvent.key` value etc)
* Legacy API, LMGetKbdType() (to get physical keyboard layout type, required to call `UCKeyTranslate()`, IIRC, this is older than Mac OS X 10.0)

#2 and #4 are not published by Apple's developer document even though they were published. Therefore, they might be treated as private API in Apple (i.e., I meant that they are not removed from SDK's hearder files, but they might be unstable status).

And according to Makoto Kato-san who is also working around IME handling, WebKit uses non-published API, which were added to NSTextInputContext. His patch is here, but not yet considered to use the non-published API. (The API name is handleEventByInputMethod:.) And I'm afraid that Apple would block browsers to use such API since most applications don't need to use such low level API.

So, my points are:

• Whether Gecko will be able to handle keyboard/IME input without big changes or not.
• If rewriting is required, whether new API is published or not, and whether new API is enough for initializing WidgetKeyboardEvent for DOM KeyboardEvent and shortcut key handling.

Comment 1

[Gian-Carlo Pascutto [:gcp]]

Masayuki, if we can get you access to a DTK via macStadium, would this allow you to investigate this issue?

Comment 2

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Although I don't know about macStadium, if it's like a kind of remote desktop, yes, I could.

Comment 3

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Taking.

Comment 4

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

I checked all pre-installed CJKT IMEs except strokes (although, I'm not familiar with non-Japanese languages) and Google Japanese Input, and they work fine as what I see on Catalina.

Comment 5

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Here is the detail of my investigation. Now, it can be open:

Hiragana (Japanese IME, called Kotoeri formerly)

• As a native speaker, it looks like this works perfectly. Including reconversion of selected word, undo of the last commit.
• But I found an existing bug which sometimes suggests appearing at the bottom-left of the screen when I try to undo the last commit (bug 1677684).

Google Japanese Input

• Third party’s IME, surprisingly, this works even though not built for Apple Silicon!
• Works fine as far as I’ve tested.

ATOK

• Third party’s IME, this also works the binary for Intel.
• So, macOS’s IME framework must be not depending on CPU architecture different from the PowerPC -> Intel era.
• Works fine as far as I’ve tested.

Pinyin - Simplified, Shuangpin - Simplified, Wubi - Simplified, Zhuyin - Traditional, Pinyin - Traditional, Changie - Traditional, Sucheng - Traditional, Zhuyin Eten - Traditional

• I don’t speak Chinese languages. Therefore, I’ve tested only simple cases and it looks like they work fine.
• Showing candidate list at conversion.
• Selecting an item in candidate list with keyboard navigation.
• Committing composition with corresponding key (Space bar, Enter, numeric keys, etc.)
• Canceling composition (e.g., with Escape key).

2-Set Korean, 3-Set Korean, 390 Sebulshik, GongjinCheong Romaja, HNC Romaja

• I don’t speak Korean languages. Therefore, I’ve tested basic cases and the cases I’ve regressed.
• Typing Hangul Syllables.
• Typing space while composition.
• Typing Enter while composition.
• Typing Tab key while composition.
• Typing arrow keys while composition.
• Showing candidate list to convert Hangul to Hanja (Chinese characters).
• Selecting items in it.
• Committing an item in it.
• Typing the next Hangul character to commit current composition.

Simple Telex, Telex

• I don’t speak Vietnamese neither :-) So, I just tested dead key like simple input, and it works.

Emoji palette

• Works perfectly.

Comment 6

[Gian-Carlo Pascutto [:gcp]]

Do the third-party Intel IME's also work if Rosetta is not installed yet? Or does it perhaps trigger automated installation?

Comment 7

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

(In reply to Gian-Carlo Pascutto [:gcp] from comment #6)

Do the third-party Intel IME's also work if Rosetta is not installed yet? Or does it perhaps trigger automated installation?

I tested without any installation except Firefox Nightly and the third parties' IME. So, I believed that Rosetta is pre-installed. I don't remember whether installing IME causes additional installation dialog for Rosetta (and perhaps, Apple does not like such UX).

Comment 8

[Gian-Carlo Pascutto [:gcp]]

So, I believed that Rosetta is pre-installed.

It is not. I'm not clear how the magic works, but until we have evidence there's a problem we can probably just close this.

Comment 9

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

(In reply to Gian-Carlo Pascutto [:gcp] from comment #8)

So, I believed that Rosetta is pre-installed.

It is not. I'm not clear how the magic works, but until we have evidence there's a problem we can probably just close this.

Well, but Rosetta is (probably) required only for running third party IMEs which are built for Intel Mac. I.e., not required for Firefox. What's a problem?

Comment 10

[Gian-Carlo Pascutto [:gcp]]

Rosetta is (probably) required only for running third party IMEs which are built for Intel Mac. I.e., not required for Firefox. What's a problem?

You guessed earlier that Rosetta comes preinstalled, but we know it doesn't. So presumably the request to install it did in fact happen when you installed the third party IME - as it's unclear how it could have worked otherwise.

As said in comment 8, we have no evidence there's any problem.