Closed Bug 1299930 Opened 7 years ago Closed 7 years ago

[webvr] Support HTC Vive controller trackpads through the Gamepad API

Categories

(Core :: Graphics, defect)

Product:
Core
Component:
Graphics
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla52
Tracking Flags:
Tracking Status
firefox52 --- fixed

People

(Reporter: kip, Assigned: daoshengmu)

References

Details

(Whiteboard: [gfx-noted])

Attachments

(1 file, 2 obsolete files)

Bug 1299930 - Support OpenVR controller trackpads via Gamepad API;
58 bytes, text/x-review-board-request
kip
: review+
Details 
Each HTC Vive Controllers include an independent, clickable trackpad.
Whiteboard: [gfx-noted]
Assignee: nobody → dmu
I already have a prototype for supporting trackpads via Gamepad API's buttons and axes attributes.

Kip, do you think we should wait for the conclusion of https://github.com/w3c/gamepad/issues/34, or we can support it in my current way first?
Flags: needinfo?(kgilbert)
(In reply to Daosheng Mu[:daoshengmu] from comment #1)
> I already have a prototype for supporting trackpads via Gamepad API's
> buttons and axes attributes.
> 
> Kip, do you think we should wait for the conclusion of
> https://github.com/w3c/gamepad/issues/34, or we can support it in my current
> way first?

I will bring this up with the rest of the WebVR team and in the implementers meeting.  (We meet every tuesday).

Ideally, we can make it easy for developers to find the trackpad axis on a wide range of controllers and support features such as multitouch through the same interface.

This may be either a new webidl interface in the Gamepad API or it could be some attributes to provide metadata for mapping the axis and buttons.

I'll keep the NI open until I can better answer your question.
In the VR controller sample from WebVR 1.1 in Chromium, https://github.com/toji/webvr-samples/blob/1cc580b81ad6afb268febbf1698624be6d7373b1/XX-vr-controllers.html#L277. They use buttons and axes attributes for trackpads as well. Probably, we can follow this way.
This is close to review until we confirm some details about if we should define more attributes for trackpads.
Comment on attachment 8802416 [details] [diff] [review]
Bug 1299930 Support OpenVR controller trackpads

>From 56d4824d175f2211c2457b3509782d332891a1ab Mon Sep 17 00:00:00 2001
>From: Daosheng Mu <daoshengmu@gmail.com>
>Date: Wed, 19 Oct 2016 15:46:44 +0800
>Subject: [PATCH] Bug 1299930 - Support OpenVR controller trackpads via Gamepad
> API;
>
>MozReview-Commit-ID: 41C0rc7fZTi
>---
> gfx/vr/gfxVR.cpp       | 12 ++++++++++
> gfx/vr/gfxVR.h         |  3 +++
> gfx/vr/gfxVROpenVR.cpp | 60 +++++++++++++++++++++++++++++++++++++++++---------
> gfx/vr/gfxVROpenVR.h   |  2 ++
> 4 files changed, 66 insertions(+), 11 deletions(-)
>
>diff --git a/gfx/vr/gfxVR.cpp b/gfx/vr/gfxVR.cpp
>index 23eefbc..3a53bb9 100644
>--- a/gfx/vr/gfxVR.cpp
>+++ b/gfx/vr/gfxVR.cpp
>@@ -84,8 +84,20 @@ VRControllerManager::NewButtonEvent(uint32_t aIndex, uint32_t aButton,
> {
>   dom::GamepadButtonInformation a(aIndex, dom::GamepadServiceType::VR,
>                                   aButton, aPressed, aPressed ? 1.0L : 0.0L);
> 
>   VRManager* vm = VRManager::Get();
>   MOZ_ASSERT(vm);
>   vm->NotifyGamepadChange<dom::GamepadButtonInformation>(a);
> }
>+
>+void
>+VRControllerManager::NewAxisMove(uint32_t aIndex, uint32_t aAxis,
>+                                 double aValue)
>+{
>+  dom::GamepadAxisInformation a(aIndex, dom::GamepadServiceType::VR,
>+                                aAxis, aValue);
>+
>+  VRManager* vm = VRManager::Get();
>+  MOZ_ASSERT(vm);
>+  vm->NotifyGamepadChange<dom::GamepadAxisInformation>(a);
>+}
>\ No newline at end of file
>diff --git a/gfx/vr/gfxVR.h b/gfx/vr/gfxVR.h
>index 9f4d099..50c55d7 100644
>--- a/gfx/vr/gfxVR.h
>+++ b/gfx/vr/gfxVR.h
>@@ -247,28 +247,31 @@ public:
> 
>   static uint32_t AllocateControllerID();
>   virtual bool Init() = 0;
>   virtual void Destroy() = 0;
>   virtual void HandleInput() = 0;
>   virtual void GetControllers(nsTArray<RefPtr<VRControllerHost>>& aControllerResult) = 0;
>   virtual void ScanForDevices() = 0;
>   void NewButtonEvent(uint32_t aIndex, uint32_t aButton, bool aPressed);
>+  void NewAxisMove(uint32_t aIndex, uint32_t aAxis, double aValue);
>   void AddGamepad(const char* aID, dom::GamepadMappingType aMapping,
>                   uint32_t aNumButtons, uint32_t aNumAxes);
> 
> protected:
>   VRControllerManager() : mInstalled(false), mControllerCount(0) {}
>   virtual ~VRControllerManager() {}
> 
>   bool mInstalled;
>   uint32_t mControllerCount;
>   static Atomic<uint32_t> sControllerBase;
> 
> private:
>   virtual void HandleButtonPress(uint32_t aControllerIdx,
>                                  uint64_t aButtonPressed) = 0;
>+  virtual void HandleAxisMove(uint32_t aControllerIdx, uint32_t aAxis,
>+                              float aValue, int32_t aInvert) = 0;
> };
> 
> } // namespace gfx
> } // namespace mozilla
> 
> #endif /* GFX_VR_H */
>diff --git a/gfx/vr/gfxVROpenVR.cpp b/gfx/vr/gfxVROpenVR.cpp
>index 7bc80e1..fbfda79 100644
>--- a/gfx/vr/gfxVROpenVR.cpp
>+++ b/gfx/vr/gfxVROpenVR.cpp
>@@ -67,26 +67,40 @@ const uint64_t gOpenVRButtonMask[] = {
>   // vr::ButtonMaskFromId(vr::EVRButtonId::k_EButton_A),
>   vr::ButtonMaskFromId(vr::EVRButtonId::k_EButton_SteamVR_Touchpad),
>   vr::ButtonMaskFromId(vr::EVRButtonId::k_EButton_SteamVR_Trigger)
> };
> 
> const uint32_t gNumOpenVRButtonMask = sizeof(gOpenVRButtonMask) /
>                                       sizeof(uint64_t);
> 
>-const uint64_t gOpenVRAxisMask[] = {
>-  vr::ButtonMaskFromId(vr::EVRButtonId::k_EButton_Axis0),
>-  vr::ButtonMaskFromId(vr::EVRButtonId::k_EButton_Axis1),
>-  vr::ButtonMaskFromId(vr::EVRButtonId::k_EButton_Axis2),
>-  vr::ButtonMaskFromId(vr::EVRButtonId::k_EButton_Axis3),
>-  vr::ButtonMaskFromId(vr::EVRButtonId::k_EButton_Axis4)
>+enum class VRControllerAxisType : uint16_t {
>+  TrackpadXAxis,
>+  TrackpadYAxis,
>+  Trigger,
>+  NumVRControllerAxisType
> };
> 
>-const uint32_t gNumOpenVRAxisMask = sizeof(gOpenVRAxisMask) /
>-                                    sizeof(uint64_t);
>+struct VRControllerAxisInfo
>+{
>+  uint32_t mIdx;
>+  int32_t mInvert;
>+
>+  VRControllerAxisInfo(vr::EVRButtonId aId, uint32_t aInvert)
>+    : mIdx(aId - vr::EVRButtonId::k_EButton_Axis0), mInvert(aInvert) {}
>+};
>+
>+const VRControllerAxisInfo gOpenVRAxes[] = {
>+  VRControllerAxisInfo(vr::EVRButtonId::k_EButton_Axis0, 1),
>+  VRControllerAxisInfo(vr::EVRButtonId::k_EButton_Axis0, -1),

This is strange for trackpad axis, but we have a spec at GamepadAPI (https://w3c.github.io/gamepad/#remapping). It mentions negative is up, positive is down.


>+  VRControllerAxisInfo(vr::EVRButtonId::k_EButton_Axis1, 1)
>+};
>+
>+const uint32_t gNumOpenVRAxis = sizeof(gOpenVRAxes) /
>+                                sizeof(VRControllerAxisInfo);
> 
> bool
> LoadOpenVRRuntime()
> {
>   static PRLibrary *openvrLib = nullptr;
> 
>   nsAdoptingCString openvrPath = Preferences::GetCString("gfx.vr.openvr-runtime");
>   if (!openvrPath)
>@@ -474,17 +488,17 @@ VRDisplayManagerOpenVR::GetHMDs(nsTArray<RefPtr<VRDisplayHost>>& aHMDResult)
> 
> VRControllerOpenVR::VRControllerOpenVR()
>   : VRControllerHost(VRDeviceType::OpenVR)
> {
>   MOZ_COUNT_CTOR_INHERITED(VRControllerOpenVR, VRControllerHost);
>   mControllerInfo.mControllerName.AssignLiteral("OpenVR HMD");
>   mControllerInfo.mMappingType = dom::GamepadMappingType::_empty;
>   mControllerInfo.mNumButtons = gNumOpenVRButtonMask;
>-  mControllerInfo.mNumAxes = gNumOpenVRAxisMask;
>+  mControllerInfo.mNumAxes = gNumOpenVRAxis;
> }
> 
> VRControllerOpenVR::~VRControllerOpenVR()
> {
>   MOZ_COUNT_DTOR_INHERITED(VRControllerOpenVR, VRControllerHost);
> }
> 
> void
>@@ -554,32 +568,46 @@ VRControllerManagerOpenVR::Destroy()
>   mOpenVRInstalled = false;
> }
> 
> void
> VRControllerManagerOpenVR::HandleInput()
> {
>   RefPtr<impl::VRControllerOpenVR> controller;
>   vr::VRControllerState_t state;
>+  uint32_t axis = 0;
> 
>   MOZ_ASSERT(mVRSystem);
> 
>   // Process OpenVR controller state
>   for (uint32_t i = 0; i < mOpenVRController.Length(); ++i) {
>     controller = mOpenVRController[i];
> 
>     MOZ_ASSERT(mVRSystem->GetTrackedDeviceClass(controller->GetTrackedIndex())
>                == vr::TrackedDeviceClass_Controller);
> 
>     if (mVRSystem->GetControllerState(controller->GetTrackedIndex(), &state)) {
>       if (state.ulButtonPressed) {
>         HandleButtonPress(controller->GetIndex(), state.ulButtonPressed);
>       }
> 
>-      // Handle Axis support in Bug 1299930
>+      axis = static_cast<uint32_t>(VRControllerAxisType::TrackpadXAxis);
>+      VRControllerAxisInfo axisInfo(gOpenVRAxes[axis]);
>+      HandleAxisMove(controller->GetIndex(), axis,
>+                     state.rAxis[axisInfo.mIdx].x, axisInfo.mInvert);
>+
>+      axis = static_cast<uint32_t>(VRControllerAxisType::TrackpadYAxis);
>+      axisInfo = gOpenVRAxes[axis];
>+      HandleAxisMove(controller->GetIndex(), axis,
>+                     state.rAxis[axisInfo.mIdx].y, axisInfo.mInvert);
>+
>+      axis = static_cast<uint32_t>(VRControllerAxisType::Trigger);
>+      axisInfo = gOpenVRAxes[axis];
>+      HandleAxisMove(controller->GetIndex(), axis,
>+                     state.rAxis[axisInfo.mIdx].x, axisInfo.mInvert);
>     }
>   }
> }
> 
> void
> VRControllerManagerOpenVR::HandleButtonPress(uint32_t aControllerIdx,
>                                              uint64_t aButtonPressed)
> {
>@@ -587,16 +615,26 @@ VRControllerManagerOpenVR::HandleButtonPress(uint32_t aControllerIdx,
> 
>   for (uint32_t i = 0; i < gNumOpenVRButtonMask; ++i) {
>     buttonMask = gOpenVRButtonMask[i];
>     NewButtonEvent(aControllerIdx, i, aButtonPressed & buttonMask);
>   }
> }
> 
> void
>+VRControllerManagerOpenVR::HandleAxisMove(uint32_t aControllerIdx, uint32_t aAxis,
>+                                          float aValue, int32_t aInvert)
>+{
>+  if (aValue != 0.0f) {
>+    printf_stderr("controller %d axis %d value %f \n", aControllerIdx, aAxis, aValue * aInvert);
>+    NewAxisMove(aControllerIdx, aAxis, aValue * aInvert);
>+  }
>+}
>+
>+void
> VRControllerManagerOpenVR::GetControllers(nsTArray<RefPtr<VRControllerHost>>& aControllerResult)
> {
>   if (!mOpenVRInstalled) {
>     return;
>   }
> 
>   aControllerResult.Clear();
>   for (uint32_t i = 0; i < mOpenVRController.Length(); ++i) {
>@@ -626,12 +664,12 @@ VRControllerManagerOpenVR::ScanForDevices()
> 
>     RefPtr<VRControllerOpenVR> openVRController = new VRControllerOpenVR();
>     openVRController->SetIndex(mControllerCount);
>     openVRController->SetTrackedIndex(trackedDevice);
>     mOpenVRController.AppendElement(openVRController);
> 
>     // Not already present, add it.
>     AddGamepad("OpenVR Gamepad", GamepadMappingType::_empty,
>-               gNumOpenVRButtonMask, gNumOpenVRAxisMask);
>+               gNumOpenVRButtonMask, gNumOpenVRAxis);
>     ++mControllerCount;
>   }
> }
>\ No newline at end of file
>diff --git a/gfx/vr/gfxVROpenVR.h b/gfx/vr/gfxVROpenVR.h
>index 636b7e7..3cbf4a3 100644
>--- a/gfx/vr/gfxVROpenVR.h
>+++ b/gfx/vr/gfxVROpenVR.h
>@@ -117,16 +117,18 @@ public:
>   virtual void ScanForDevices() override;
> 
> private:
>   VRControllerManagerOpenVR();
>   ~VRControllerManagerOpenVR();
> 
>   virtual void HandleButtonPress(uint32_t aControllerIdx,
>                                  uint64_t aButtonPressed) override;
>+  virtual void HandleAxisMove(uint32_t aControllerIdx, uint32_t aAxis,
>+                              float aValue, int32_t aInvert) override;
> 
>   bool mOpenVRInstalled;
>   nsTArray<RefPtr<impl::VRControllerOpenVR>> mOpenVRController;
>   vr::IVRSystem *mVRSystem;
> };
> 
> } // namespace gfx
> } // namespace mozilla
>-- 
>2.8.1.windows.1
>
I think it would be okay to implement the same as the Chromium WebVR builds, exposing as regular axis and button until https://github.com/w3c/gamepad/issues/34 is concluded.
Flags: needinfo?(kgilbert)
In this update, I remove the invert function for VRControllerAxis because I think axis move in [top, bottom] to be [1, -1] is more make sense.
Attachment #8802416 - Attachment is obsolete: true
Attachment #8803294 - Attachment is obsolete: true
Comment on attachment 8803428 [details]
Bug 1299930 - Support OpenVR controller trackpads via Gamepad API;

https://reviewboard.mozilla.org/r/87706/#review86700

LGTM
Attachment #8803428 - Flags: review?(kgilbert) → review+
Keywords: checkin-needed
Pushed by cbook@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6a52e0212b16
Support OpenVR controller trackpads via Gamepad API; r=kip
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/6a52e0212b16
Status: NEW → RESOLVED
Closed: 7 years ago
status-firefox52: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla52
You need to log in before you can comment on or make changes to this bug.