Bugzilla

Comment 1

•

5 years ago

FYI, handling hardware media keys is tracking in bug1572869. Now we have implemented that in OSX first.

Reporter

Comment 2

•

5 years ago

Oh yeah, good catch, yet another bug in there. For Linux, we have 1353652 and on Mac there is "Now Playing Info and registering for Remote Command Center actions.", did you already implement that?

Actually I am now also thinking that Global Hotkeys aren't the right way to go for Media Keys, however some web-extensions seem to have registered as a listener to them. So maybe we have to connect the systems a bit later-on. But I am certain now that Use Case 2 won't be relevant to this issue / handled separately.

Comment 3

•

5 years ago

(In reply to Marc Streckfuß from comment #2)

Oh yeah, good catch, yet another bug in there. For Linux, we have 1353652 and on Mac there is "Now Playing Info and registering for Remote Command Center actions.", did you already implement that?

Thank for your information! It sounds like a good idea to support MPRIS API. But I'm not quite sure what the Remote Command Center actions you meant?

Now in bug1251795, the author would like to use MediaPlayer framework to tell system what media is playing now, is is the same thing?

Blocks: media-control

Emilio Cobos Álvarez (:emilio)

Reporter

Comment 4

•

5 years ago

I think this is the correct framework, yes. I took that quote from apples site, there is a sample project for that which I just downloaded. There seems to be NowPlayingInfo to expose the information to the system and Remote Command Center to "remote control" the player, that is listening to play and pause. Actually I just saw a glimpse of "RemoteCommand" in the phabricator link, so I guess yes.
That's perfect so far, now I just need to see if windows supports such a thing as well, then we can completely isolate it from real hotkeys, apart from maybe injecting them as hotkeys as well, when they don't actually control the Media Session API

Updated

•

5 years ago

Depends on: 1584542

Comment 5

•

5 years ago

I think we can mark this bug as a meta bug for supporting receiving media hardware keys, even if Firefox is not on foreground. And use separate bugs to track the implementation of each platform.

Depends on: 1584030, 1591230

Summary: Support for Global Hotkeys → [meta] Support for Global Media Hotkeys

BugBot [:suhaib / :marco/ :calixte]

Comment 6

•

5 years ago

•

Edited

Except for keyboard events, I wonder how would the hardware headset events are handling? Are they going to be converted to certain keyboard keycodes and then we can use same mechanism to capture them?

It seems MediaPlayer framework on OSX could handle headset events as well, but I don't know how they would be handled on Windows and Linux. Any ideas?

Flags: needinfo?(masayuki)

Flags: needinfo?(marc.streckfuss)

Karl Tomlinson (:karlt)

Comment 7

•

5 years ago

I have seen headset "mute" buttons converted to keyboard events on Linux.

Updated

•

5 years ago

Keywords: meta

http://developer.chrome.com/apps/commands#scope

Comment 8

•

5 years ago

I would like to add that Chrome has had this for several years now:

And Firefox had it too, up until Firefox 57

Comment 9

•

5 years ago

(In reply to Steven Penny from comment #8)

I would like to add that Chrome has had this for several years now:

http://developer.chrome.com/apps/commands#scope

And Firefox had it too, up until Firefox 57

That is a different thing from what we will do in this bug, that issue is being tracked by bug1251795.

Comment 10

•

5 years ago

(In reply to Alastor Wu [:alwu] from comment #6)

Except for keyboard events, I wonder how would the hardware headset events are handling? Are they going to be converted to certain keyboard keycodes and then we can use same mechanism to capture them?

I'm not sure. I guess it depends on the implementation of utility of the device. I guess that posing WM_APP_COMMAND message to focused application is standard manner.

Flags: needinfo?(masayuki)

Reporter

Comment 11

•

5 years ago

(In reply to Alastor Wu [:alwu] from comment #6)

Except for keyboard events, I wonder how would the hardware headset events are handling? Are they going to be converted to certain keyboard keycodes and then we can use same mechanism to capture them?

It seems MediaPlayer framework on OSX could handle headset events as well, but I don't know how they would be handled on Windows and Linux. Any ideas?

I think we make the mistake of thinking about Keys here at all. While Keys work when in foreground and there are ways to capture them in the background, this is only some "legacy" or "simple" way of solving things.

MPRIS for instance allows me to use the KDEConnect App on Linux, so I can control Media with my Smartphone, as long as I am in the same local network.
For Windows I'd suspect a headset maybe even having configurable buttons and then some software/driver decides what to do (thinking of gaming headsets, at least).

Both are things we don't need to worry about, as the OS should take care of it and translate it into the desired API.
For Windows: Since my keyboard triggers SMTC without any special knowledge, I guess the OS converts the keycode (as long as no one handles it?) to SMTC.
For Linux: I have no intel here what the Desktop Environment is doing, but I suspect they do it just the same. And if a certain headset doesn't react, then it's a thing between the headset and the DE.

So: Keys would be only relevant when the desired API isn't available, which means Windows < 8.1 or Desktop Environments without D-Bus or MPRIS Handling.

TBH I think we have a few bugs where we mix up keys and media api in general and even pulled in some bug about web extensions global shortcuts which is only related because the example web extension provides like a media session implementation.

Reporter

Updated

•

5 years ago

Flags: needinfo?(marc.streckfuss)

Comment 12

•

5 years ago

Thank you for your explanation.

I totally misunderstood what MPRIS and SMTC can provide to us, I first thought SMTC is just providing a virtual control interface and didn't realize platforms (Windows&Linux) have already intergated media keys handling to SMTC and MPRIS. So the bug1591288 I filed was totally nosense, for each platform we should only need one media keys event source, which would uses MediaPlayer on OSX, MPRIS on Linux, SMTC on Windows and MediaController on Android (I don't know if it would handle headset media keys event though).

Therefore, in the meanwhile when you're implementing those native implementations, I would probably start to extend the interface of MediaControllerin order to support adding metadata and create other platform media keys event interfaces, in which you could easily add your native implementation for bug1353652 and bug1584501.

Updated

•

5 years ago

Depends on: 1353652

Comment 13

•

5 years ago

As Marc mentioned before, I incorrectly mixed the global hotkeys with the media keys control, but actually they are two different things, because for media keys we have other framework which directly supported and be integrated in platform, we have no need to monitor media keys directly.

Therefore, remove all those media keys related bugs dependency.

No longer blocks: media-control

No longer depends on: 1353652, 1584030, 1584542, 1591230

Updated

•

5 years ago

Keywords: meta

Reporter

Updated

•

4 years ago

Alias: webext-commands-global

Blocks: 1584542, 1411795

URL: https://developer.chrome.com/extensio...

Summary: Support for Global Hotkeys → [meta] Support Global Hotkeys in Web Extensions (commands.global)

BugBot [:suhaib / :marco/ :calixte]

Reporter

Comment 14

•

4 years ago

So, now that we've splitted Media Keys, this will be the Meta/Tracking Bug for adding the commands.global flag for Web-Extensions.
One usecase is outlined in 1411795 (I used this bug as tracking bug, since the other one is more focused on getting media keys to work, which would've been one usecase).
Another usecase would be the Riot WebExtension (https://addons.mozilla.org/de/firefox/addon/riot/) which needs that to support Push-To-Talk in voice calls, e.g.

On all platforms besides Windows, we cannot register a hotkey without being rejected by the OS when someone already claimed that hotkey.
Moreover commands can be changed by the user, so one question will be to determine how to propagate a failure.
My guess/recommendation would be to make a hint in the UI where one can change keybindings and maybe a notification on addon load when it fails (remember what hotkeys are available can be very dynamic, depending on the other applications) or maybe pass an event to the addon so it can decide how to notify the user, it might not be critical for its functioning.

I'm currently waiting on some directions for the windows implementation as there are multiple ways to get hotkeys.

Updated

•

4 years ago

Keywords: meta

Reporter

Comment 15

•

4 years ago

I am looking for someone to help me coordinate global hotkeys, in this specific case for web extensions but there could very well be other use cases for global hotkeys, which is why this is rather vague.

See https://bugzilla.mozilla.org/show_bug.cgi?id=1584542 for most of the background, the conversation which should've belonged here has happened over there.

Do you know someone better fitting or would you be willing to mentor this/give me some guidance?

Flags: needinfo?(masayuki)

Comment 16

•

4 years ago

I don't understand the requirements of "global" attribute of command of WebExtensions. Should it be triggered even when all Firefox windows are inactivated? Or, should it be triggered when a window of Firefox has focus?

Flags: needinfo?(masayuki) → needinfo?(marc.streckfuss)

https://bugzilla.mozilla.org/show_bug.cgi?id=1411795

Comment 17

•

4 years ago

@masayuki it was laid out pretty clearly in the previous issue:

Reporter

Comment 18

•

4 years ago

Yes, the key here is to have hotkeys whenever the browser windows are not in focus.
Don't focus on webextensions specifically, though. They are just the one use case we currently have.
The reason I ni?d you is also because other parts of the browser could benefit from these hotkeys too (i.e. for push-to-talk voice communication, a button to bring a new browser window upfront (i.e. a CTRL+N but without the need of an active window) or many other things I just can't imagine now).

So it's specifically about how the events should be propagated etc.

@svnpenn:
MediaKeys are natively supported in current Nightly, with Metadata being about to land. It probably doesn't replace the extension, but maybe a few sites started using MediaSession in the meantime.

Flags: needinfo?(marc.streckfuss) → needinfo?(masayuki)

https://addons.mozilla.org/firefox/addon/media-keys

Comment 19

•

4 years ago

@MeFisto94 no I dont think so.

I just tried Firefox 75.0a1 (2020-03-09) with Media Keys 2.5.2 and global is
still broken. Again, since Firefox 57 it only works if browser has focus.

Comment 20

•

4 years ago

Sorry for the delay due to 2 urgent bugs...

So, in my understanding, WebExtensions wants to listen to key events even when Firefox is inactive application.

Currently, I know only global message hook on Windows for doing that, but my Windows native knowledge is not updated after Vista so that there might be better way to do that. Anyway, it seems that Chromium does not use it for the purpose.
https://source.chromium.org/search?q=SetWindowsHook&ss=chromium%2Fchromium%2Fsrc&originalUrl=https:%2F%2Fcs.chromium.org%2F

On macOS, global key listening causes that macOS warns the fact to the user when application launches or the listener is installed, but I'm not familiar with that.

Flags: needinfo?(masayuki)

Comment 21

•

4 years ago

Cc-ing Makoto-san and emk-san, they may know how to listen keyboard events even while we're inactive.

Reporter

Comment 22

•

4 years ago

Don't worry, I'm in no hurry, I am still at the planning phase of this feature :)

So, in my understanding, WebExtensions wants to listen to key events even when Firefox is inactive application.
Exactly and the primary questions are: How do we transfer those events and (to be able to answer that better:) who could benefit from those key events (i.e. the browser itself?)

As for the native implementations, in the other bug there was: https://phabricator.services.mozilla.com/D47440
Basically for windows I've found 5 possible implementations, but since on Mac OS X there is the pop-up and for Linux it's impossible without root, we have to resort to RegisterHotkey (WINAPI) and XGrabKey, I fear, because otherwise we'd have a platform dependant behavior.
But that is something for the other bug and probably for jimm/someone else to decide.

The "problem" I had is: Those native implementations need to be wired up to some HotkeysManager or something and there the problems start:

Due to native APIs we can't listen for arbitrary keys but everyone needs to register with the HotkeysManager
Registration with a callback procedure or should we emit dom::KeyboardEvents with a global flag?
Allow Registration for the same keybinding from multiple sources? I.e. forward the event to all web extensions?
Probably everything else I am overlooking around this (specifically testing etc), which are required for a complete metabug.

Thanks in advance for your time.

@svnpenn:
Unfortunately not the Addon, but MediaKeys itself should be able to control arbitrary HTML5 Videos and Audios (media.hardwaremediakeys.enabled is the pref, it should be enabled on nightly [only])

Flags: needinfo?(masayuki)

James Teh [:Jamie]

Comment 23

•

4 years ago

On Windows, there's also RegisterHotKey. This isn't quite as flexible as using a global key hook via SetWindowsHookEx. For example, I don't think it allows the Windows key as a modifier, even when there isn't a conflict and even though it has a constant for it:

MOD_WIN
0x0008
Either WINDOWS key was held down. These keys are labeled with the Windows logo. Keyboard shortcuts that involve the WINDOWS key are reserved for use by the operating system.

That said, Chrome severely limits what can be defined as shortcuts, even when the user configures them. You can't use the Windows key and you have to use either control or alt, not both. So, Windows+alt+p isn't possible, nor is control+alt+p. IMO, this severely limits its usefulness.

James Teh [:Jamie]

Comment 24

•

4 years ago

Oh. Comment 22 already pointed out RegisterHotKey. Sorry. :(

Comment 25

•

4 years ago

@MeFisto94 I just tested again, and I found 2 issues with your comments.

First, media.hardwaremediakeys.enabled is already TRUE on nightly. Second, even
with that set, it doesnt do anything. Even if the browser is active and even
if the active tab is playing a video, pressing PAUSE or PLAY/PAUSE does
nothing.

Comment 26

•

4 years ago

Well, be careful to use RegisterHotKey API because its matching algorithm is different from Gecko's shortcut key matching algorithm. Gecko's one was designed for providing consistent behavior on all desktop platforms with any keyboard layouts (as far as possible, of course) at shipping Firefox 3. The main differences between them is, Gecko refers character which is produced by the key. But Windows refers virtual keycode value which can specified by RegisterHotKey. It depends on the dll of keyboard layout that which key produces which virtual keycode. In other words, if the key is a function key (not a printable key), there must not be difference.

Flags: needinfo?(masayuki)