855364 - Implement gamepad remapping

Status: RESOLVED DUPLICATE of bug 1542893

(Core :: DOM: Device Interfaces, defect, P3)

Description

[(not currently active) Ted Mielczarek]

The Gamepad spec defines a way to map buttons/axes to a known layout that matches most modern game controllers. We should implement this. This will require us to have a built-in table of known controllers for everything but the (not-yet-implemented) XInput backend.

I started to prototype such a database here:
https://github.com/luser/gamepad-data

with the intent that it could be useful both for browser implementations as well as content libraries that want to backfill.

Comment 1

[John Drinkwater (:beta)]

It might be useful to get pcgamingwiki.com involved, as they have quite detailed collection of controller mapping info for games.

Comment 2

[(not currently active) Ted Mielczarek]

I couldn't find that info on their wiki, can you point me at it?

Comment 3

[(not currently active) Ted Mielczarek]

So I added a Gamepad.mapping field to the spec:
https://dvcs.w3.org/hg/gamepad/raw-file/default/gamepad.html#widl-Gamepad-mapping

So that we can set .mapping = "standard" to indicate that we're using the standard mapping as described in the spec. bug 860413 covers adding support for that. I'm going to use this bug to add support for having a table of known controller mappings. I have a Python script and a little C++ that takes the data from the github repo in comment 0 and turns it into structs usable from C++.

Comment 4

[John Drinkwater (:beta)]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #2)
> I couldn't find that info on their wiki, can you point me at it?

Some game entries have listings, if its worth putting in unverified data I can do a pull request for gamepad-data.

There’s also this, wonder if it would be good to coordinate with SDL guys, as they’re trying to bake some default mappings into SDL2
http://hg.libsdl.org/SDL/file/45df4552bdfe/src/joystick/SDL_gamecontroller.c#l93

Comment 5

[(not currently active) Ted Mielczarek]

That is interesting, thanks for the link!

Comment 6

[(not currently active) Ted Mielczarek]

Attachment: Implement gamepad mapping on Linux

Here's a first cut at implementing this on Linux. gamepad_mapping.{cpp,h} are something I intend to commit to the gamepad-data repo as the "C++ library" bit. gamepad_mapping_data.h is the output of running writemappings.py from that repo on the mappings.json.

This seems to work well, I added mappings for all the controllers I own (I have an as-yet-unpushed program to spit out a mapping for a controller) and they all show up as expected.

Comment 7

[John Drinkwater (:beta)]

(In reply to John Drinkwater (:beta) from comment #4)
> There’s also this, wonder if it would be good to coordinate with SDL guys,
> as they’re trying to bake some default mappings into SDL2
> http://hg.libsdl.org/SDL/file/45df4552bdfe/src/joystick/SDL_gamecontroller.
> c#l93

Since my last comment, SDL2 has expanded and moved these mappings to a header file which, IANAL, is compatible with Gecko and could easily be updated periodically.
http://hg.libsdl.org/SDL/file/5ce21e1ca0c3/src/joystick/SDL_gamecontrollerdb.h

Might be worth sharing the same gamecontroller code as well as its making up a part of SteamOS.

Comment 8

[Edward Rudd]

What is the status on implementing this mapping system into Firefox?  w/o it, gamepad support in firefox is rather useless for most controllers.  (and apparently Google Chrome does have a mapping layer in-place now, as my controller is mapped as 'Standard' in Google Chrome).

Ideally making the system compatible with the SDL2 mapping layer would be ideal. (although the SDL2 layer uses axies for L2/R2 vs the Web Gamepad API spec puts those as buttons).  The main reason why it's worth using that is the Database is crowd sourced ( https://github.com/gabomdq/SDL_GameControllerDB ) and built-in to steam as well. For gamedevelopement this makes supporting controllers REALLY easy.

And please let me know how I can assist in developing this.  I am porting a large number of games to asm.js through Humble Bundle and we really want nice gamepad support in the browsers.

Comment 9

[(not currently active) Ted Mielczarek]

It has stalled because the more I think about it the more I think it was a terrible mistake to include in the spec. Yes, Chrome is shippping with some built-in mappings. Yes, Firefox could do the same. However, there will always be controllers that the browser doesn't know about that will not present a standard mapping, and content using the API will either have to handle that or be broken with those controllers. Yes, we could say "we support all the major console controllers and that's good enough" but when the next generation of consoles comes out you get a new set of controllers that you don't support right away.

Right now I'm worried that developers will develop against Chrome with a supported gamepad and not test the non-standard case at all, making their games useless for people without the same type of gamepad. If we implement this in Firefox then that case is likely to wind up completely untested as people will test in Chrome and Firefox, see that it works for them, and not bother dealing with the non-standard case.

Comment 10

[(not currently active) Ted Mielczarek]

Related: we've talked a lot (and people have written multiple times) content-based polyfills for controller remapping, and I still think that might be the best way to go. I wrote one that uses the repo I mentioned in comment 0:
https://github.com/luser/gamepadmapping.js

If it was possible to use the SDL2 mapping layer for this that'd be great. Last I looked at it I wasn't sure it was possible because SDL relied on some hardware IDs that weren't available on any browser via the Gamepad API (and might not be attainable, since SDL was using DirectInput last I looked and I can't figure out any way to get the device ID DirectInput uses via Raw Input).

Comment 11

[Edward Rudd]

The issue right now, is that the gamepad API is completely useless without these mappings, as the DPad is not mapped through what-so-ever.  So I can't even use the ID that is mapped through to get these controllers working. 

And the IDs can map to the SDL2 mappings.. for example in the referenced bug #952773 the user had a "46d-c218-Logitech RumblePad 2 USB"  which in the SDL2 mapping is this line

(Win ID)
6d0418c2000000000000504944564944,Logitech RumblePad 2 USB
(OSX ID)
6d0400000000000018c2000000000000,Logitech F510 Gamepad (DInput)

Both of those are for the same device.
(note the endian difference of the first part of the string).

Also, a mapping system is a good thing to have in the spec, having it in SDL2 has made game development/porting significantly easier because of the mapping system.. and updates could be handle either via an about:config setting, or some simple UI (about:gamepad), or a feed that can be pushed separately from browser releases.  

Requiring each "game" developed against the Gamepad API to have their own mapping is rather ridiculous and is what has gotten us to this point today, where developers only support one controller(Xinput/Xbox 460) and ignore the rest..  Having a mapping layer in SDL2 and the Gamepad API makes it so that database is maintained in a single place and means the developer doesn't have to DO anything for it to "just work".

The SDL2 GameController API is a proven API mechanism that is used in MANY MANY games over the past 2 years (I know, as I've used that API in at least a dozen games I've ported). A polyfill is simply the wrong way to go IMHO.  Especially when the underlying Gamepad drivers are simply broken in Firefox.. (e.g. why in the WORLD are the Axes off by one on the Logitech F510? and why do I not have my DPad at all)

Comment 12

[Edward Rudd]

And Note, that I am willing to write and contribute the code to the Firefox/Gecko codebase for this effort as we are wanting this functionality @ Humble Bundle.

Comment 13

[Jukka Jylänki]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #9)
> However, there will always be controllers that the browser doesn't know
> about

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #9)
> Right now I'm worried that developers will develop against Chrome with a
> supported gamepad and not test the non-standard case at all, ...

What is the non-standard case? How does one test for the non-standard case, i.e. what does the testing plan look like for the non-standard case if one does want to diligently carry that out?

You argue that it is impossible for the browser to know all the controllers, i.e. even Mozilla or Google as the browser developers don't have the ability to map out all controllers in the world. I can't imagine that we could then expect or worry that the developer should have done that.

I think it is a very fine idea to provide precreated mapping layers for all the controllers that are effectively xbox 360/ps 3 controllers or have buttons that match those. However this mapping layer should not lose information provided by the controller and I think we should also provide a mechanism to get the unmapped raw data.

The spec currently does have the mapping feature written into it. I imagine it is not being proposed to be removed?

(In reply to Edward Rudd from comment #11)
> Especially when the underlying Gamepad drivers are simply broken in
> Firefox.. (e.g. why in the WORLD are the Axes off by one on the Logitech
> F510? and why do I not have my DPad at all)

Can you post STRs and information about the devices to reproduce these bugs, in separate bug items?

Comment 14

[(not currently active) Ted Mielczarek]

(In reply to Jukka Jylänki from comment #13)
> (In reply to Ted Mielczarek [:ted.mielczarek] from comment #9)
> > However, there will always be controllers that the browser doesn't know
> > about
> 
> (In reply to Ted Mielczarek [:ted.mielczarek] from comment #9)
> > Right now I'm worried that developers will develop against Chrome with a
> > supported gamepad and not test the non-standard case at all, ...
> 
> What is the non-standard case? How does one test for the non-standard case,
> i.e. what does the testing plan look like for the non-standard case if one
> does want to diligently carry that out?

The non-standard case is simply when Gamepad.mapping == "". In this case the buttons and axes are in no defined order, they're simply the raw data from the driver.

> You argue that it is impossible for the browser to know all the controllers,
> i.e. even Mozilla or Google as the browser developers don't have the ability
> to map out all controllers in the world. I can't imagine that we could then
> expect or worry that the developer should have done that.

I'm not, but historically games have provided controller remapping UI for this situation. I don't want to ship that in a browser, and I certainly don't want to try to spec it. I'm just worried that if most browsers ship support for remapping the most popular controllers that developers won't even consider this case. Furthermore, if someone is going to maintain a database of controller mappings, it seems a lot more sensible to do that in content rather than in browsers.

> I think it is a very fine idea to provide precreated mapping layers for all
> the controllers that are effectively xbox 360/ps 3 controllers or have
> buttons that match those. However this mapping layer should not lose
> information provided by the controller and I think we should also provide a
> mechanism to get the unmapped raw data.

Unfortunately the spec doesn't currently have any way to "unmap" a controller that's been mapped to the standard mapping. That would be a useful thing.

> The spec currently does have the mapping feature written into it. I imagine
> it is not being proposed to be removed?

It's not, I just have reservations about it now. I've already seen too many people suggest that Firefox is "broken" because it doesn't support remapping, but while the spec allows for remapping it doesn't (and couldn't) *require* it, so developers who don't have a fallback plan for this situation are already setting themselves up for failure. :-(

Comment 15

[Edward Rudd]

(In reply to Jukka Jylänki from comment #13)
> (In reply to Edward Rudd from comment #11)
> > Especially when the underlying Gamepad drivers are simply broken in
> > Firefox.. (e.g. why in the WORLD are the Axes off by one on the Logitech
> > F510? and why do I not have my DPad at all)
> 
> Can you post STRs and information about the devices to reproduce these bugs,
> in separate bug items?
This specific set of issues is documented in #952773

Comment 16

[Edward Rudd]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #14)
> (In reply to Jukka Jylänki from comment #13)
> > You argue that it is impossible for the browser to know all the controllers,
> > i.e. even Mozilla or Google as the browser developers don't have the ability
> > to map out all controllers in the world. I can't imagine that we could then
> > expect or worry that the developer should have done that.
> 
> I'm not, but historically games have provided controller remapping UI for
> this situation. I don't want to ship that in a browser, and I certainly
> don't want to try to spec it. I'm just worried that if most browsers ship
> support for remapping the most popular controllers that developers won't
> even consider this case. Furthermore, if someone is going to maintain a
> database of controller mappings, it seems a lot more sensible to do that in
> content rather than in browsers.

Modern games, however don't provide in-game mapping systems.  The general case is they support XInput and that's it.  And for us all of the games we are bringing to the browser are build using SDL2 and the GameController API, thus the game continues to believe it's using an XInput like controller.  The reality is that for games, developers do not want to build a mapping system, they just want to support one controller style and be done with it as that is an extra complexity (that is easy to screw up). Also users simply expect the controller to "just work" in games.. and if they have to remap for every game, they will be unhappy.  Hence the reason why the SDL2 GameController API was created.. to remove that need and push it to a central place. 

> > I think it is a very fine idea to provide precreated mapping layers for all
> > the controllers that are effectively xbox 360/ps 3 controllers or have
> > buttons that match those. However this mapping layer should not lose
> > information provided by the controller and I think we should also provide a
> > mechanism to get the unmapped raw data.
> 
> Unfortunately the spec doesn't currently have any way to "unmap" a
> controller that's been mapped to the standard mapping. That would be a
> useful thing.

Perhaps we should build/propose a spec change to allow a game to request unmapped data?, however we need to ensure that the HAT is properly exposed in those scenarios.
 
> > The spec currently does have the mapping feature written into it. I imagine
> > it is not being proposed to be removed?
> 
> It's not, I just have reservations about it now. I've already seen too many
> people suggest that Firefox is "broken" because it doesn't support
> remapping, but while the spec allows for remapping it doesn't (and couldn't)
> *require* it, so developers who don't have a fallback plan for this
> situation are already setting themselves up for failure. :-(

Plane and simple, requiring games to implement a mapping system isn't going to work.. And what will happen is those games will simply only support the "Standard" layout and ignore everything else.  Frankly this is what we do with all our game ports.. We use the SDL2 GameController API and ignore anything that doesn't map to that.   Now if the user is playing the game in steam, they can easily "configure" an unknown controller inside Big Picture Mode and Steam will export that configuration out to the game.. Thus again, centrally controlled and any game using SDL2 will gain knowledge of that controller.

Comment 17

[Katelyn Gadd (:kael)]

Control reassignment/binding is important for accessibility scenarios, in addition to nonstandard controllers. Sony recently added system-wide binding/reassignment to the PS4 firmware and the response to it (from accessibility advocates, etc) was strong. Many games do in fact ship with built in button mapping and binding support so you can reassign keyboard hotkeys and switch gamepad buttons around - though in some cases they only let you remap the keyboard.

It's fair to say that games shouldn't have to implement this, but someone has to implement it. It seems unlikely that all browsers could ship a high quality binding implementation at this point. IMO this should be solved in a content library so that standard bindings can be maintained on a central server and updated to accommodate new hardware. That central server's domain can be used as a localStorage/indexedDB scope to allow multiple games to share the user's mapping information so that they don't have to set it per-game.

If SDL2 already makes attempts to handle controller mapping/binding through the API, I would suggest that the browser version of SDL2 is the right place to implement this stuff. The partial implementation of remapping in some current browsers discourages the use of a content library for this but I think a content library is the right choice.

Comment 19

[Daosheng Mu[:daoshengmu]]

Fixed by Bug 1542893.