1584542 - Binding support for desktop hotkeys

Status: NEW

(WebExtensions :: General, enhancement, P5)

Description

[Marc Streckfuß [:MeFisto94]]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0

Steps to reproduce:

This is related to 1583858,
I already have a basic implementation of the WinAPI Side working, which I will publish to phabricator shortly for review.

Comment 1

[Marc Streckfuß [:MeFisto94]]

Attachment: Bug 1584542 - First draft Implementation of possible methods to get keyboard input when the browser windows are not focused

Comment 2

[Jim Mathies [:jimm]]

For this, the legacy way was listening for WM_APPCOMANDs, but under Windows 10, my media keys did not trigger such a message.

According to MSDN this is supported under win10. I have a keyboard with such keys, maybe work up a test app we can throw around for people to test?

Also in Order to grab them when the window has no focus...

Why is this a requirement?

Comment 3

[Jim Mathies [:jimm]]

The "linux-compat" method uses RegisterHotKey() which has exactly been made to listen for Hotkeys.

Pros:
Feedback when hotkey could not be registered[1]
Only listening for Keys that are required
Cons:
Only one application can have a hotkey at a time, making it unavailable for other applications

This seems like a good approach with few tradeoff. You could register when media starts, and unregister when it stops.

Comment 4

[Marc Streckfuß [:MeFisto94]]

(In reply to Jim Mathies [:jimm] (OOTO until Oct 7th) from comment #2)

For this, the legacy way was listening for WM_APPCOMANDs, but under Windows 10, my media keys did not trigger such a message.

According to MSDN this is supported under win10. I have a keyboard with such keys, maybe work up a test app we can throw around for people to test?

Yes, I will cobble together a test app containing WM_APPCOMANDS as well as evaluating the use of Raw Input (https://stackoverflow.com/a/41154576 something like this)

Also in Order to grab them when the window has no focus...

Why is this a requirement?

Because that is what "global" Hotkeys are about. Actually this bug is a dependant of https://bugzilla.mozilla.org/show_bug.cgi?id=1411795 (https://developer.chrome.com/apps/commands#scope)

(In reply to Jim Mathies [:jimm] (OOTO until Oct 7th) from comment #3)

The "linux-compat" method uses RegisterHotKey() which has exactly been made to listen for Hotkeys.

Pros:
Feedback when hotkey could not be registered[1]
Only listening for Keys that are required
Cons:
Only one application can have a hotkey at a time, making it unavailable for other applications

This seems like a good approach with few tradeoff. You could register when media starts, and unregister when it stops.

Actually to clarify this: Now that I know better, Media Keys should not be any relevant for this, apart from support for Windows < 8.1 (W7, which is close to EOL), because there will be a better API which will handle Media Keys automatically and better.
So this is mainly for Web Extensions in general or maybe, depending on the API, anything within firefox in desire of global hotkeys.
I also like this method because it's easy. AKlotz has suggested evaluating RawInput as well, the discussion for this is on phabricator.

Comment 7

[Marc Streckfuß [:MeFisto94]]

I've now added yet another two possible methods to get hotkeys with RawInput vs. RegisterHotKeys being the two relevant implementations I think.
RegisterHotKeys supports only one application at a time, where RawInput essentially is a "keylogger" where we get a WM_INPUT for every keydown or keyup event.

I'd need some feedback on the revision itself as well as guidance onto what method to persue.
On Linux only the RegisterHotKeys equivalent works without root and on Mac OS X trying to "keylogg" prompts a security warning/dialog (see alwu's prior Mac Media Keys Implementation).

Thus probably only RegisterHotKeys is persuable, unless we don't care about equivalent behavior along the platforms.

Comment 8

[Jim Mathies [:jimm]]

Hey Masayuki, I'm looking for a good mentor / reviewer here. This work has to do with keyboard input and media keys, curious if you might be able to help or know of someone on staff who could?

Comment 9

[Marc Streckfuß [:MeFisto94]]

To expand on that, as I've done a bad job describing/filing this issue:
This is about global shortcut support in web extensions (and maybe everywhere else).
Media Keys themselves are handled differently starting on Windows 8.1, but there are web extensions wanting to bind them as well.
I'm actually going to outline this better in the parent issue and make this a meta issue then.

Comment 10

[Jim Mathies [:jimm]]

Ah, ok that helps. Lets get this over to the webext team. They need to decide if they want to support this. Then we would file bugs in various platform components (where your patch would come into use).

Comment 11

[Tomislav Jovanovic :zombie]

We are in favor of supporting (global) media keys in the WebExtension API. It's a bit unclear which bug will be used to implement which part of this, and what's the right product/component for each, since this patch doesn't seem to touch any of the extensions code.

Comment 12

[Marc Streckfuß [:MeFisto94]]

Hey! Thanks for your support.
Actually this patch is meant as a native implementation for Windows only, so it was at the right place,
the tracking (meta) issue is supposed to be webext-commands-global (https://bugzilla.mozilla.org/show_bug.cgi?id=1583858).

I wonder how generic we should implement though, like are there other use cases where global shortcuts (note: this is NOT about media keys only, they are just a detail which has to be supported as well) could be of use, specifically:

• Is there any JS API for Websites to hook global shortcuts?
• Could the Browser itself profit from global shortcuts?

This is probably a question for masayuki then. Specifically because existing (local) shortcuts are handled by composing pressed keys.
In theory we could also add these events with a specific "global shortcut" flag.
The extreme opposite could be a class "GlobalHotkeyManager" where the loaded webextension ultimatively calls in to register specific hotkeys and the manager just calls the passed method pointer.

Who could I ask for such general purpose questions and to maybe coordinate the concept?

Comment 13

[Tomislav Jovanovic :zombie]

(In reply to Marc Streckfuß [:MeFisto94] from comment #12)

Who could I ask for such general purpose questions and to maybe coordinate the concept?

I'm not sure who would be the right person to answer those questions, maybe as you mention Masayuki or Alastor Wu based on bug 1572869.

It certainly looks like wider support for global keys could be useful across the platform, but I'm very far from the person who could decide that.

Comment 14

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee didn't login in Bugzilla in the last 7 months.
:mixedpuppy, could you have a look please?
For more information, please visit auto_nag documentation.