1436791 - Implement VRDisplayExternal

Status: RESOLVED FIXED

(Core :: WebVR, enhancement)

Description

[:kip (Kearwood Gilbert)]

Rather than including support for every device-specific XR API as a subclass of VRDisplayHost, we can allow Android apps using GeckoView to implement the API interface in the host application.

A VRDisplayExternal subclass of VRDisplayHost will allow the host application to connect to shmem with Android's Binder.  This shmem will contain a struct with a VRDisplayInfo member (including VRHMDSensorData), a struct to describe controller state, additional fields needed for threadsafe synchronization, and fields added as needed for lifecycle management.

VRDisplayExternal will only enumerate a single HMD, but can emit multiple controllers as requested by the host.

Gecko will submit textures and layer information back to the host application by populating a struct describing the requested VR layers for presentation.  The texture used by submitted frames will support all handles included in SurfaceDescriptor.

Although initially used only with GeckoView, the shmem-based interface is intended to be used for embedding on other platforms, including those on desktop.

Comment 1

[:kip (Kearwood Gilbert)]

Attachment: Bug 1436791 - Implement gfxVRExternal

- gfxVRExternal enables other processes to present
  real or simulated VR hardware to Firefox.
- This functionality is disabled by default, under
  dom.vr.external.enabled.
- VRDisplayInfo, VRControllerInfo, and associated
  structs have been restructured to ensure internal
  state is not exposed via shmem interface.
- Some refactoring to convert structs to
  POD types, enabling them to be located
  in shmem and be memcpy'd.
- Work needed before unpreffing marked
  with "TODO" comments.

Review commit: https://reviewboard.mozilla.org/r/227260/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/227260/

Comment 2

[:kip (Kearwood Gilbert)]

Comment on attachment 8958323 [details]
Bug 1436791 - Implement gfxVRExternal

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/227260/diff/1-2/

Comment 3

[:kip (Kearwood Gilbert)]

Attachment: Removed TTY spam in gfxVRExternal

Review commit: https://reviewboard.mozilla.org/r/227478/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/227478/

Comment 4

[Randall Barker [:rbarker]]

Comment on attachment 8958323 [details]
Bug 1436791 - Implement gfxVRExternal

https://reviewboard.mozilla.org/r/227260/#review233340

::: gfx/vr/gfxVR.h:62
(Diff revision 2)
>    uint32_t mPresentingGroups;
>    uint32_t mGroupMask;
> -  Size mStageSize;
> -  Matrix4x4 mSittingToStandingTransform;
>    uint64_t mFrameId;
>    uint32_t mPresentingGeneration;

I think mPresentingGeneration should be in the Display State as it is how the external client can stop presenting such as when a phone is taken out of a head set. Since there will need to be follow up to add Android support, this can probably wait for that.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/04ed9a706cfc
Implement gfxVRExternal,r=rbarker

Comment 6

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/04ed9a706cfc