733620 - Allow extensions to access volume button events

Status: RESOLVED WONTFIX

(Firefox for Android Graveyard :: General, enhancement)

Description

[microrffr]

I want to listen for volume button presses in my extension.
The keyup events make it into gecko here
http://mxr.mozilla.org/mozilla-central/source/mobile/android/base/GeckoInputConnection.java#805
but get discarded here
http://mxr.mozilla.org/mozilla-central/source/widget/android/nsWindow.cpp#1926

Background: I'm making an extension to switch tabs with the volume buttons.

Comment 1

[Matt Brubeck (:mbrubeck)]

Another potential problem is that we eat the keydown events here:
https://mxr.mozilla.org/mozilla-central/source/mobile/android/base/GeckoInputConnection.java#720

It's necessary to return "false" here if we want the keys to have their default action of changing the ringer volume / media volume.  If we want to let extensions override these keys, we'll need to give them a chance to prevent the default action.  We'll need to wait for a response from Gecko, or maybe add a pref extensions could set to change the key handling.

Comment 2

[Mark Finkle (:mfinkle) (use needinfo?)]

I think this is a cool idea and we should try to equip add-on devs for this kind of innovation where ever possible.

Comment 3

[kelly joseph price]

Hey there, I was hoping I could take this on as my first bug.

Comment 4

[Matt Brubeck (:mbrubeck)]

(In reply to Matt Brubeck (:mbrubeck) from comment #1)
> Another potential problem is that we eat the keydown events here:

Sorry that link is out of date.  Here's the correct line:
http://hg.mozilla.org/mozilla-central/file/cf4face65451/mobile/android/base/GeckoInputConnection.java#l826

(In reply to kelly joseph price from comment #3)
> Hey there, I was hoping I could take this on as my first bug.

Great!  The tricky part of this bug is making sure that the default behavior (changing the system volume) still works *except* in the case where an add-on wants to override it.

Ideally we can send a message from the UI code (the parts written in Java) to the "Gecko" platform code (written in C++/JavaScript), then wait for a reply to come back and tell us whether to perform the default action or not.  Either we need to do this all synchronously (so we can get the reply before deciding what to return from GeckoInputConnection.processKeyDown) or if it's asynchronous then we need a way to consume the event when we first receive it, then re-create it (or fake its effects) when we get the response).

I wasn't sure how feasible that is, so my back-up plan from comment 1 was to create a hidden preference (say, "widget.ime.android.override_volume_key_action") that is off by default, but could be enabled by add-ons.  GeckoInputConnection.processKeyDown would check the value of the preference before deciding whether to return true or false.  But the downside is that the default action would be disabled all the time, not just when the add-on chooses to do so.

You can send messages from the Java UI to Gecko asynchronously using GeckoAppShell's "sendEventToGecko" method, and you receive those messages in JavaScript using the observer service.  For example, the Session:Back message is send and received here:
http://hg.mozilla.org/mozilla-central/file/cf4face65451/mobile/android/base/Tab.java#l421
http://hg.mozilla.org/mozilla-central/file/cf4face65451/mobile/android/chrome/content/browser.js#l145
http://hg.mozilla.org/mozilla-central/file/cf4face65451/mobile/android/chrome/content/browser.js#l891

We don't have a ready-made way to send synchronous messages from Java to Gecko, so as a first step I recommend seeing if an asynchronous message-passing solution is possible.  That means finding out whether we can "re-play" a volume key event in the Android Java code after we initially discarded it.

Comment 5

[Kevin Brosnan [Ex-Mozilla]]

Resetting to unassigned. If you are still planning on working on this comment in the bug and it can be set back to assigned.

Comment 6

[Andrew]

Hi, a classmate and I would like to take this on as a class assignment if possible.

Comment 7

[Matt Brubeck (:mbrubeck)]

Certainly.  Let me know if you have any questions.  Comment 4 has some pointers to the code involved.  For general developer documentation, see https://wiki.mozilla.org/Mobile/Fennec/Android

Comment 8

[Lala]

 Hi Matt this is Lala. i am Andrew's partner and we are working on it together. We'll get back to you with any questions after detailed reviewing of the code. Thanks.

Comment 9

[immerfox]

It seems that nobody is working on this one and it does not really seem complicated, but interesting on the other side (I have experience in Java/JS and I know a bit of C++). Could I take it?

Comment 10

[Kartikaya Gupta (email:[e-mail])]

immerfox, you're more than welcome to work on this bug. If you haven't done so already, the first thing to do is to set up a build environment as described at https://wiki.mozilla.org/Mobile/Fennec/Android. Don't hesitate to ask questions either by posting in this bug, emailing us, or in the #mobile channel on IRC.

Comment 11

[immerfox]

I have installed everything (also the jimdb), but what IDE do you use for both C++ & Java? And how do you debug the code? I mean, fennec runs on the device (emulator in my case), and how do I attach/ see the running code?

Comment 12

[Aaron Train [:aaronmt]]

(In reply to immerfox from comment #11)
> I have installed everything (also the jimdb), but what IDE do you use for
> both C++ & Java? And how do you debug the code? I mean, fennec runs on the
> device (emulator in my case), and how do I attach/ see the running code?

Best to ask these questions in #mobile on irc.mozilla.org and to keep comments on this bug related to the bug at hand.

Comment 13

[immerfox]

Matt,

probably the code changed and I suppose the line with the behaviour is now http://mxr.mozilla.org/mozilla-central/source/mobile/android/base/GeckoInputConnection.java#688 

After debugging a bit for the asynchronous way, I thought: why not to have ALWAYS the default behaviour and additionally create ALWAYS and send to Gecko a new Event generated with GeckoEvent.createKeyEvent(), no matter whether some addon want to process it? IF that would also work, we should probably do this only for the volume buttons?

Comment 14

[Matt Brubeck (:mbrubeck)]

(In reply to immerfox from comment #13)
> After debugging a bit for the asynchronous way, I thought: why not to have
> ALWAYS the default behaviour and additionally create ALWAYS and send to
> Gecko a new Event generated with GeckoEvent.createKeyEvent(), no matter
> whether some addon want to process it?

I think this might make the volume buttons too awkward for most purposes.  For example, see the use case in comment 0.  Or a similar add-on might turn the volume buttons into scroll buttons.  But the user definitely doesn't want to turn the ringer volume up or down when they are trying to switch tabs or scroll a web page!

[By the way, since this is a niche use case (it would be used by a small number of add-ons, not by most of our users), we should make sure it doesn't complicate the code too much.  If it's not possible to fix this in a reasonably simple way, we might need to resolve it as WONTFIX.]

Comment 15

[Mike Hoye [:mhoye]]

Hi, Immerfox - Are you still working on this?

Comment 16

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

For the complexity reasons outlined in comment 14 and the lack of activity on this bug (meaning it's not too important and we can spend our time elsewhere), let's close this.