[webvr] Support HTC Vive controller trackpads through the Gamepad API

RESOLVED FIXED in Firefox 52

Status

()

Core
Graphics
RESOLVED FIXED
a year ago
a year ago

People

(Reporter: kip, Assigned: daoshengmu)

Tracking

unspecified
mozilla52
Points:
---

Firefox Tracking Flags

(firefox52 fixed)

Details

(Whiteboard: [gfx-noted])

MozReview Requests

()

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(1 attachment, 2 obsolete attachments)

(Reporter)

Description

a year ago
Each HTC Vive Controllers include an independent, clickable trackpad.

Updated

a year ago
Whiteboard: [gfx-noted]
(Assignee)

Updated

a year ago
Assignee: nobody → dmu
(Assignee)

Comment 1

a year ago
I already have a prototype for supporting trackpads via Gamepad API's buttons and axes attributes.

Kip, do you think we should wait for the conclusion of https://github.com/w3c/gamepad/issues/34, or we can support it in my current way first?
Flags: needinfo?(kgilbert)
(Reporter)

Comment 2

a year ago
(In reply to Daosheng Mu[:daoshengmu] from comment #1)
> I already have a prototype for supporting trackpads via Gamepad API's
> buttons and axes attributes.
> 
> Kip, do you think we should wait for the conclusion of
> https://github.com/w3c/gamepad/issues/34, or we can support it in my current
> way first?

I will bring this up with the rest of the WebVR team and in the implementers meeting.  (We meet every tuesday).

Ideally, we can make it easy for developers to find the trackpad axis on a wide range of controllers and support features such as multitouch through the same interface.

This may be either a new webidl interface in the Gamepad API or it could be some attributes to provide metadata for mapping the axis and buttons.

I'll keep the NI open until I can better answer your question.
(Assignee)

Comment 3

a year ago
In the VR controller sample from WebVR 1.1 in Chromium, https://github.com/toji/webvr-samples/blob/1cc580b81ad6afb268febbf1698624be6d7373b1/XX-vr-controllers.html#L277. They use buttons and axes attributes for trackpads as well. Probably, we can follow this way.
(Assignee)

Comment 4

a year ago
Created attachment 8802416 [details] [diff] [review]
Bug 1299930 Support OpenVR controller trackpads

This is close to review until we confirm some details about if we should define more attributes for trackpads.
(Assignee)

Comment 5

a year ago
Comment on attachment 8802416 [details] [diff] [review]
Bug 1299930 Support OpenVR controller trackpads

>From 56d4824d175f2211c2457b3509782d332891a1ab Mon Sep 17 00:00:00 2001
>From: Daosheng Mu <daoshengmu@gmail.com>
>Date: Wed, 19 Oct 2016 15:46:44 +0800
>Subject: [PATCH] Bug 1299930 - Support OpenVR controller trackpads via Gamepad
> API;
>
>MozReview-Commit-ID: 41C0rc7fZTi
>---
> gfx/vr/gfxVR.cpp       | 12 ++++++++++
> gfx/vr/gfxVR.h         |  3 +++
> gfx/vr/gfxVROpenVR.cpp | 60 +++++++++++++++++++++++++++++++++++++++++---------
> gfx/vr/gfxVROpenVR.h   |  2 ++
> 4 files changed, 66 insertions(+), 11 deletions(-)
>
>diff --git a/gfx/vr/gfxVR.cpp b/gfx/vr/gfxVR.cpp
>index 23eefbc..3a53bb9 100644
>--- a/gfx/vr/gfxVR.cpp
>+++ b/gfx/vr/gfxVR.cpp
>@@ -84,8 +84,20 @@ VRControllerManager::NewButtonEvent(uint32_t aIndex, uint32_t aButton,
> {
>   dom::GamepadButtonInformation a(aIndex, dom::GamepadServiceType::VR,
>                                   aButton, aPressed, aPressed ? 1.0L : 0.0L);
> 
>   VRManager* vm = VRManager::Get();
>   MOZ_ASSERT(vm);
>   vm->NotifyGamepadChange<dom::GamepadButtonInformation>(a);
> }
>+
>+void
>+VRControllerManager::NewAxisMove(uint32_t aIndex, uint32_t aAxis,
>+                                 double aValue)
>+{
>+  dom::GamepadAxisInformation a(aIndex, dom::GamepadServiceType::VR,
>+                                aAxis, aValue);
>+
>+  VRManager* vm = VRManager::Get();
>+  MOZ_ASSERT(vm);
>+  vm->NotifyGamepadChange<dom::GamepadAxisInformation>(a);
>+}
>\ No newline at end of file
>diff --git a/gfx/vr/gfxVR.h b/gfx/vr/gfxVR.h
>index 9f4d099..50c55d7 100644
>--- a/gfx/vr/gfxVR.h
>+++ b/gfx/vr/gfxVR.h
>@@ -247,28 +247,31 @@ public:
> 
>   static uint32_t AllocateControllerID();
>   virtual bool Init() = 0;
>   virtual void Destroy() = 0;
>   virtual void HandleInput() = 0;
>   virtual void GetControllers(nsTArray<RefPtr<VRControllerHost>>& aControllerResult) = 0;
>   virtual void ScanForDevices() = 0;
>   void NewButtonEvent(uint32_t aIndex, uint32_t aButton, bool aPressed);
>+  void NewAxisMove(uint32_t aIndex, uint32_t aAxis, double aValue);
>   void AddGamepad(const char* aID, dom::GamepadMappingType aMapping,
>                   uint32_t aNumButtons, uint32_t aNumAxes);
> 
> protected:
>   VRControllerManager() : mInstalled(false), mControllerCount(0) {}
>   virtual ~VRControllerManager() {}
> 
>   bool mInstalled;
>   uint32_t mControllerCount;
>   static Atomic<uint32_t> sControllerBase;
> 
> private:
>   virtual void HandleButtonPress(uint32_t aControllerIdx,
>                                  uint64_t aButtonPressed) = 0;
>+  virtual void HandleAxisMove(uint32_t aControllerIdx, uint32_t aAxis,
>+                              float aValue, int32_t aInvert) = 0;
> };
> 
> } // namespace gfx
> } // namespace mozilla
> 
> #endif /* GFX_VR_H */
>diff --git a/gfx/vr/gfxVROpenVR.cpp b/gfx/vr/gfxVROpenVR.cpp
>index 7bc80e1..fbfda79 100644
>--- a/gfx/vr/gfxVROpenVR.cpp
>+++ b/gfx/vr/gfxVROpenVR.cpp
>@@ -67,26 +67,40 @@ const uint64_t gOpenVRButtonMask[] = {
>   // vr::ButtonMaskFromId(vr::EVRButtonId::k_EButton_A),
>   vr::ButtonMaskFromId(vr::EVRButtonId::k_EButton_SteamVR_Touchpad),
>   vr::ButtonMaskFromId(vr::EVRButtonId::k_EButton_SteamVR_Trigger)
> };
> 
> const uint32_t gNumOpenVRButtonMask = sizeof(gOpenVRButtonMask) /
>                                       sizeof(uint64_t);
> 
>-const uint64_t gOpenVRAxisMask[] = {
>-  vr::ButtonMaskFromId(vr::EVRButtonId::k_EButton_Axis0),
>-  vr::ButtonMaskFromId(vr::EVRButtonId::k_EButton_Axis1),
>-  vr::ButtonMaskFromId(vr::EVRButtonId::k_EButton_Axis2),
>-  vr::ButtonMaskFromId(vr::EVRButtonId::k_EButton_Axis3),
>-  vr::ButtonMaskFromId(vr::EVRButtonId::k_EButton_Axis4)
>+enum class VRControllerAxisType : uint16_t {
>+  TrackpadXAxis,
>+  TrackpadYAxis,
>+  Trigger,
>+  NumVRControllerAxisType
> };
> 
>-const uint32_t gNumOpenVRAxisMask = sizeof(gOpenVRAxisMask) /
>-                                    sizeof(uint64_t);
>+struct VRControllerAxisInfo
>+{
>+  uint32_t mIdx;
>+  int32_t mInvert;
>+
>+  VRControllerAxisInfo(vr::EVRButtonId aId, uint32_t aInvert)
>+    : mIdx(aId - vr::EVRButtonId::k_EButton_Axis0), mInvert(aInvert) {}
>+};
>+
>+const VRControllerAxisInfo gOpenVRAxes[] = {
>+  VRControllerAxisInfo(vr::EVRButtonId::k_EButton_Axis0, 1),
>+  VRControllerAxisInfo(vr::EVRButtonId::k_EButton_Axis0, -1),

This is strange for trackpad axis, but we have a spec at GamepadAPI (https://w3c.github.io/gamepad/#remapping). It mentions negative is up, positive is down.


>+  VRControllerAxisInfo(vr::EVRButtonId::k_EButton_Axis1, 1)
>+};
>+
>+const uint32_t gNumOpenVRAxis = sizeof(gOpenVRAxes) /
>+                                sizeof(VRControllerAxisInfo);
> 
> bool
> LoadOpenVRRuntime()
> {
>   static PRLibrary *openvrLib = nullptr;
> 
>   nsAdoptingCString openvrPath = Preferences::GetCString("gfx.vr.openvr-runtime");
>   if (!openvrPath)
>@@ -474,17 +488,17 @@ VRDisplayManagerOpenVR::GetHMDs(nsTArray<RefPtr<VRDisplayHost>>& aHMDResult)
> 
> VRControllerOpenVR::VRControllerOpenVR()
>   : VRControllerHost(VRDeviceType::OpenVR)
> {
>   MOZ_COUNT_CTOR_INHERITED(VRControllerOpenVR, VRControllerHost);
>   mControllerInfo.mControllerName.AssignLiteral("OpenVR HMD");
>   mControllerInfo.mMappingType = dom::GamepadMappingType::_empty;
>   mControllerInfo.mNumButtons = gNumOpenVRButtonMask;
>-  mControllerInfo.mNumAxes = gNumOpenVRAxisMask;
>+  mControllerInfo.mNumAxes = gNumOpenVRAxis;
> }
> 
> VRControllerOpenVR::~VRControllerOpenVR()
> {
>   MOZ_COUNT_DTOR_INHERITED(VRControllerOpenVR, VRControllerHost);
> }
> 
> void
>@@ -554,32 +568,46 @@ VRControllerManagerOpenVR::Destroy()
>   mOpenVRInstalled = false;
> }
> 
> void
> VRControllerManagerOpenVR::HandleInput()
> {
>   RefPtr<impl::VRControllerOpenVR> controller;
>   vr::VRControllerState_t state;
>+  uint32_t axis = 0;
> 
>   MOZ_ASSERT(mVRSystem);
> 
>   // Process OpenVR controller state
>   for (uint32_t i = 0; i < mOpenVRController.Length(); ++i) {
>     controller = mOpenVRController[i];
> 
>     MOZ_ASSERT(mVRSystem->GetTrackedDeviceClass(controller->GetTrackedIndex())
>                == vr::TrackedDeviceClass_Controller);
> 
>     if (mVRSystem->GetControllerState(controller->GetTrackedIndex(), &state)) {
>       if (state.ulButtonPressed) {
>         HandleButtonPress(controller->GetIndex(), state.ulButtonPressed);
>       }
> 
>-      // Handle Axis support in Bug 1299930
>+      axis = static_cast<uint32_t>(VRControllerAxisType::TrackpadXAxis);
>+      VRControllerAxisInfo axisInfo(gOpenVRAxes[axis]);
>+      HandleAxisMove(controller->GetIndex(), axis,
>+                     state.rAxis[axisInfo.mIdx].x, axisInfo.mInvert);
>+
>+      axis = static_cast<uint32_t>(VRControllerAxisType::TrackpadYAxis);
>+      axisInfo = gOpenVRAxes[axis];
>+      HandleAxisMove(controller->GetIndex(), axis,
>+                     state.rAxis[axisInfo.mIdx].y, axisInfo.mInvert);
>+
>+      axis = static_cast<uint32_t>(VRControllerAxisType::Trigger);
>+      axisInfo = gOpenVRAxes[axis];
>+      HandleAxisMove(controller->GetIndex(), axis,
>+                     state.rAxis[axisInfo.mIdx].x, axisInfo.mInvert);
>     }
>   }
> }
> 
> void
> VRControllerManagerOpenVR::HandleButtonPress(uint32_t aControllerIdx,
>                                              uint64_t aButtonPressed)
> {
>@@ -587,16 +615,26 @@ VRControllerManagerOpenVR::HandleButtonPress(uint32_t aControllerIdx,
> 
>   for (uint32_t i = 0; i < gNumOpenVRButtonMask; ++i) {
>     buttonMask = gOpenVRButtonMask[i];
>     NewButtonEvent(aControllerIdx, i, aButtonPressed & buttonMask);
>   }
> }
> 
> void
>+VRControllerManagerOpenVR::HandleAxisMove(uint32_t aControllerIdx, uint32_t aAxis,
>+                                          float aValue, int32_t aInvert)
>+{
>+  if (aValue != 0.0f) {
>+    printf_stderr("controller %d axis %d value %f \n", aControllerIdx, aAxis, aValue * aInvert);
>+    NewAxisMove(aControllerIdx, aAxis, aValue * aInvert);
>+  }
>+}
>+
>+void
> VRControllerManagerOpenVR::GetControllers(nsTArray<RefPtr<VRControllerHost>>& aControllerResult)
> {
>   if (!mOpenVRInstalled) {
>     return;
>   }
> 
>   aControllerResult.Clear();
>   for (uint32_t i = 0; i < mOpenVRController.Length(); ++i) {
>@@ -626,12 +664,12 @@ VRControllerManagerOpenVR::ScanForDevices()
> 
>     RefPtr<VRControllerOpenVR> openVRController = new VRControllerOpenVR();
>     openVRController->SetIndex(mControllerCount);
>     openVRController->SetTrackedIndex(trackedDevice);
>     mOpenVRController.AppendElement(openVRController);
> 
>     // Not already present, add it.
>     AddGamepad("OpenVR Gamepad", GamepadMappingType::_empty,
>-               gNumOpenVRButtonMask, gNumOpenVRAxisMask);
>+               gNumOpenVRButtonMask, gNumOpenVRAxis);
>     ++mControllerCount;
>   }
> }
>\ No newline at end of file
>diff --git a/gfx/vr/gfxVROpenVR.h b/gfx/vr/gfxVROpenVR.h
>index 636b7e7..3cbf4a3 100644
>--- a/gfx/vr/gfxVROpenVR.h
>+++ b/gfx/vr/gfxVROpenVR.h
>@@ -117,16 +117,18 @@ public:
>   virtual void ScanForDevices() override;
> 
> private:
>   VRControllerManagerOpenVR();
>   ~VRControllerManagerOpenVR();
> 
>   virtual void HandleButtonPress(uint32_t aControllerIdx,
>                                  uint64_t aButtonPressed) override;
>+  virtual void HandleAxisMove(uint32_t aControllerIdx, uint32_t aAxis,
>+                              float aValue, int32_t aInvert) override;
> 
>   bool mOpenVRInstalled;
>   nsTArray<RefPtr<impl::VRControllerOpenVR>> mOpenVRController;
>   vr::IVRSystem *mVRSystem;
> };
> 
> } // namespace gfx
> } // namespace mozilla
>-- 
>2.8.1.windows.1
>
(Reporter)

Comment 6

a year ago
I think it would be okay to implement the same as the Chromium WebVR builds, exposing as regular axis and button until https://github.com/w3c/gamepad/issues/34 is concluded.
Flags: needinfo?(kgilbert)
(Assignee)

Comment 7

a year ago
Created attachment 8803294 [details] [diff] [review]
Bug 1299930 - Support OpenVR controller trackpads via Gamepad API

In this update, I remove the invert function for VRControllerAxis because I think axis move in [top, bottom] to be [1, -1] is more make sense.
Attachment #8802416 - Attachment is obsolete: true
Comment hidden (mozreview-request)
(Assignee)

Updated

a year ago
Attachment #8803294 - Attachment is obsolete: true
(Reporter)

Comment 9

a year ago
mozreview-review
Comment on attachment 8803428 [details]
Bug 1299930 - Support OpenVR controller trackpads via Gamepad API;

https://reviewboard.mozilla.org/r/87706/#review86700

LGTM
Attachment #8803428 - Flags: review?(kgilbert) → review+
(Reporter)

Updated

a year ago
Duplicate of this bug: 1302859
Comment hidden (mozreview-request)
(Assignee)

Comment 12

a year ago
Rebase and try looks good, https://treeherder.mozilla.org/#/jobs?repo=try&revision=0a35397627fa
(Assignee)

Updated

a year ago
Keywords: checkin-needed

Comment 13

a year ago
Pushed by cbook@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6a52e0212b16
Support OpenVR controller trackpads via Gamepad API; r=kip
Keywords: checkin-needed

Comment 14

a year ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/6a52e0212b16
Status: NEW → RESOLVED
Last Resolved: a year ago
status-firefox52: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla52
You need to log in before you can comment on or make changes to this bug.