[webvr] Implement Linux support for OpenVR

NEW
Assigned to

Status

()

2 years ago
25 days ago

People

(Reporter: kip, Assigned: kip)

Tracking

(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [webvr])

Attachments

(1 attachment, 1 obsolete attachment)

Comment hidden (empty)
(Assignee)

Updated

2 years ago
Blocks: 1310655
Whiteboard: [webvr]
(Assignee)

Updated

2 years ago
Component: Graphics → WebVR

Comment 1

2 years ago
Hi :kip, is Valve actively working on the OpenVR linux version then?
(Assignee)

Updated

2 years ago
Depends on: 1374393
QA Contact: cristian.comorasu
Comment hidden (mozreview-request)
(Assignee)

Comment 3

a year ago
I need to determine if it will be safe to flip the gfx.use-glx-texture-from-pixmap pref back on (it was turned off several years back and since untested).  Perhaps we will need to whitelist this for specific GPUs/drivers that are known not to cause instability.  I have pinged @milan for feedback on this approach.

Comment 4

a year ago
hi, tested this head, and it doesn't build here (linux mint 18.2) with errors in gfxVROpenVR.cpp arround line 445 related to importing GLXpixmap (can provide log report in case of need)
Comment hidden (mozreview-request)
(Assignee)

Comment 6

a year ago
(In reply to ol from comment #4)
> hi, tested this head, and it doesn't build here (linux mint 18.2) with
> errors in gfxVROpenVR.cpp arround line 445 related to importing GLXpixmap
> (can provide log report in case of need)

Thanks for your help!

I've just updated the patch with my latest work-in-progress.  This one builds for me on Ubuntu 17.04 / 64-bit.

Any logs would be greatly appreciated, thanks!

If we can get these pixmaps working to shuttle the submitted WebVR textures from the content process to the GPU process, it will be the last piece to getting WebVR to work.

Once this is working, we will need to find any instability or visual rendering problems caused by flipping the pref that enables use of the pixmaps.  It may be necessary to whitelist specific GPU drivers and test those that are likely to be seen in a WebVR user's system.

Comment 7

a year ago
Not sure if this is the best place to report this but I've been following this bug and decided to test that pixmap variable mentioned earlier.

Creating a bool 'gfx.use-glx-texture-from-pixmap' and setting true causes crashes for me on Google Maps on the latest Nvidia drivers w/ Arch Linux. stderr shows:

(firefox:18666): Gdk-ERROR **: The program 'firefox' received an X Window System error.
This probably reflects a bug in the program.
The error was 'BadAlloc (insufficient resources for operation)'.
  (Details: serial 2140 error_code 11 request_code 154 (GLX) minor_code 22)
  (Note to programmers: normally, X errors are reported asynchronously;
   that is, you will receive the error a while after causing it.
   To debug your program, run it with the GDK_SYNCHRONIZE environment
   variable to change this behavior. You can then get a meaningful
   backtrace from your debugger if you break on the gdk_x_error() function.)
ExceptionHandler::GenerateDump cloned child ExceptionHandler::WaitForContinueSignal waiting for continue signal...
19858
ExceptionHandler::SendContinueSignalToChild sent continue signal to child

Comment 8

a year ago
Sorry to double comment, forgot to mention that the above is on the latest Nightly build straight from Mozilla's FTP: 60.0a1 2018-01-24 64b with xorg-server 1.19.6 & Nvidia 390.12.

Comment 9

a year ago
Hi, I pulled the revisions from mozilla-unified with your tag branch, 

it breaks at same place (I hope to have correctly updated, I'm not used to mercurial...)

 ./mach build
 0:00.15 Clobber not needed.
 0:00.15 Adding make options from None
    MOZ_OBJDIR=/home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu
    OBJDIR=/home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu
 0:00.15 /usr/bin/make -f client.mk -s
 0:00.37 Elapsed: 0.00s; From dist/private: Kept 0 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:00.37 Elapsed: 0.00s; From dist/xpi-stage: Kept 14 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:00.38 Elapsed: 0.00s; From dist/public: Kept 0 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:00.56 Elapsed: 0.19s; From dist/idl: Kept 968 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:00.59 Elapsed: 0.22s; From _tests: Kept 1002 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:00.75 Elapsed: 0.40s; From dist/bin: Kept 2156 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:01.03 Elapsed: 0.67s; From dist/include: Kept 5401 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:01.05 buildid.h
 0:01.05 source-repo.h
 0:01.28 application.ini
 0:01.34 application.ini.h
 0:01.41 Elapsed: 0.04s; From ../../dist/idl: Kept 968 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:01.67 force-cargo-library-build
 0:01.80 warning: unused manifest key: patch.crates-io.serde_derive.feature
 0:02.18 SkJumper_generated.o
 0:02.88     Finished release [optimized] target(s) in 0.1 secs
 0:07.47 /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp: In member function ‘virtual bool mozilla::gfx::impl::VRDisplayOpenVR::SubmitFrame(const mozilla::layers::SurfaceDescriptorX11*, const Rect&, const Rect&)’:
 0:07.48 /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:434:48: error: request for member ‘OpenForeign’ in ‘aDescriptor’, which is of pointer type ‘const mozilla::layers::SurfaceDescriptorX11*’ (maybe you meant to use ‘->’ ?)
 0:07.48    RefPtr<gfxXlibSurface> surface = aDescriptor.OpenForeign();
 0:07.48                                                 ^
 0:07.48 /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:442:5: error: expected primary-expression before ‘(’ token
 0:07.48    gl()->fGenTextures(1, &tex);
 0:07.48      ^
 0:07.48 /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:444:5: error: expected primary-expression before ‘(’ token
 0:07.48    gl()->fBindTexture(LOCAL_GL_TEXTURE_2D, tex);
 0:07.48      ^
 0:07.48 /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:444:22: error: ‘LOCAL_GL_TEXTURE_2D’ was not declared in this scope
 0:07.48    gl()->fBindTexture(LOCAL_GL_TEXTURE_2D, tex);
 0:07.48                       ^
 0:07.48 /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:445:3: error: ‘sGLXLibrary’ is not a member of ‘mozilla::gl’
 0:07.48    gl::sGLXLibrary.BindTexImage(surface->XDisplay(), surface->GetGLXPixmap());
 0:07.48    ^
 0:07.48 /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:445:39: error: invalid use of incomplete type ‘class gfxXlibSurface’
 0:07.48    gl::sGLXLibrary.BindTexImage(surface->XDisplay(), surface->GetGLXPixmap());
 0:07.48                                        ^
 0:07.48 In file included from /home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/include/gfxipc/ShadowLayerUtils.h:20:0,
 0:07.48                  from /home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu/ipc/ipdl/_ipdlheaders/mozilla/layers/LayersSurfaces.h:21,
 0:07.48                  from /home/user/firefox-build/src/mozilla-unified/gfx/vr/VRDisplayHost.h:21,
 0:07.48                  from /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.h:20,
 0:07.48                  from /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:23:
 0:07.48 /home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/include/mozilla/layers/ShadowLayerUtilsX11.h:20:7: note: forward declaration of ‘class gfxXlibSurface’
 0:07.48  class gfxXlibSurface;
 0:07.48        ^
 0:07.48 /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:445:60: error: invalid use of incomplete type ‘class gfxXlibSurface’
 0:07.48    gl::sGLXLibrary.BindTexImage(surface->XDisplay(), surface->GetGLXPixmap());
 0:07.48                                                             ^
 0:07.48 In file included from /home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/include/gfxipc/ShadowLayerUtils.h:20:0,
 0:07.48                  from /home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu/ipc/ipdl/_ipdlheaders/mozilla/layers/LayersSurfaces.h:21,
 0:07.48                  from /home/user/firefox-build/src/mozilla-unified/gfx/vr/VRDisplayHost.h:21,
 0:07.48                  from /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.h:20,
 0:07.48                  from /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:23:
 0:07.48 /home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/include/mozilla/layers/ShadowLayerUtilsX11.h:20:7: note: forward declaration of ‘class gfxXlibSurface’
 0:07.48  class gfxXlibSurface;
 0:07.48        ^
 0:07.48 /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:449:34: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
 0:07.48      result = SubmitFrame((void *)tex,
 0:07.48                                   ^
 0:07.49 /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:451:26: error: ‘aSize’ was not declared in this scope
 0:07.49                           aSize, aLeftEyeRect, aRightEyeRect);
 0:07.49                           ^
 0:07.49 /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:453:11: error: expected primary-expression before ‘(’ token
 0:07.49      if (gl() && gl()->MakeCurrent()) {
 0:07.49            ^
 0:07.49 /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:453:19: error: expected primary-expression before ‘(’ token
 0:07.49      if (gl() && gl()->MakeCurrent()) {
 0:07.49                    ^
 0:07.49 /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:454:7: error: ‘sGLXLibrary’ is not a member of ‘mozilla::gl’
 0:07.49        gl::sGLXLibrary.ReleaseTexImage(mSurface->XDisplay(), mSurface->GetGLXPixmap());
 0:07.49        ^
 0:07.49 /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:454:39: error: ‘mSurface’ was not declared in this scope
 0:07.49        gl::sGLXLibrary.ReleaseTexImage(mSurface->XDisplay(), mSurface->GetGLXPixmap());
 0:07.49                                        ^
 0:07.49 /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:455:9: error: expected primary-expression before ‘(’ token
 0:07.49        gl()->fDeleteTextures(1, &tex);
 0:07.49          ^
 0:07.97 In file included from /home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/include/mozilla/LinkedList.h:71:0,
 0:07.97                  from /home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/include/jspubtd.h:16,
 0:07.97                  from /home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/include/nsCycleCollectionTraversalCallback.h:10,
 0:07.97                  from /home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/include/nsCycleCollectionNoteChild.h:13,
 0:07.97                  from /home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/include/nsTArray.h:27,
 0:07.97                  from /home/user/firefox-build/src/mozilla-unified/gfx/thebes/gfxPrefs.h:15,
 0:07.97                  from /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:11:
 0:07.97 /home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/include/mozilla/RefPtr.h: In instantiation of ‘static void mozilla::RefPtrTraits<U>::Release(U*) [with U = gfxXlibSurface]’:
 0:07.97 /home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/include/mozilla/RefPtr.h:398:40:   required from ‘static void RefPtr<T>::ConstRemovingRefPtrTraits<U>::Release(U*) [with U = gfxXlibSurface; T = gfxXlibSurface]’
 0:07.97 /home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/include/mozilla/RefPtr.h:79:44:   required from ‘RefPtr<T>::~RefPtr() [with T = gfxXlibSurface]’
 0:07.97 /home/user/firefox-build/src/mozilla-unified/gfx/vr/gfxVROpenVR.cpp:434:60:   required from here
 0:07.97 /home/user/firefox-build/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/include/mozilla/RefPtr.h:41:5: error: invalid use of incomplete type ‘class gfxXlibSurface’
 0:07.97      aPtr->Release();
[...]

Comment 10

a year ago
correction : sorry, it was obviously still on the previous version, I'm now rebuilding with updated code, that seems to pass :D , and will report on testing with vive if it compile to the end (waiting for it to finish... )

Comment 11

a year ago
it launch and I can set gfx.use-glx-texture-from-pixmap to true without crash (up to now), but can't get any change in the headset (with aframe.io exemples), even with dom.vr* options set to true ... 

(probably I'm missing something... )

here is the console while trying these exemples

[...]

JavaScript error: https://cdn.jsdelivr.net/docsearch.js/2/docsearch.min.js, line 2: Error: Error: No input element in the page matches .docs-search-sidebar .search-field
[DEBUG SHUTDOWN] Register: decoder=0x7fe20d507c00, count=2
[DEBUG SHUTDOWN] ShutdownDecoder: decoder: 'vorbis audio decoder' (0x7fe21f6994a0) flush:1
[DEBUG SHUTDOWN] operator(): pool=0x7fe21f1f7df0 shutdown=false count=0
Crash Annotation GraphicsCriticalError: |[C0][GFX1-]: Failed to do glGetActiveAttrib: 1282 (t=21.3422) |[C1][GFX1-]: Failed to do glGetActiveAttrib: 1282 (t=81.7696) |[C2][GFX1-]: Failed to do glGetActiveAttrib: 1282 (t=137.128) [GFX1-]: Failed to do glGetActiveAttrib: 1282
[DEBUG SHUTDOWN] ShutdownDecoder: decoder: 'vorbis audio decoder' (0x7fe21fabe620) flush:1
[DEBUG SHUTDOWN] Shutdown: reader=0x7fe21fbc0000 shutdown demuxer=0x7fe21fb7e910
[DEBUG SHUTDOWN] Shutdown: pool=0x7fe2219c9e80 count=2
[DEBUG SHUTDOWN] operator(): pool=0x7fe2219c9e80 shutdown=true count=1
[DEBUG SHUTDOWN] operator(): pool=0x7fe2219c9e80 shutdown=true count=0
[DEBUG SHUTDOWN] TearDownDecoders: reader=0x7fe21fbc0000 shut down audio task queue
[DEBUG SHUTDOWN] TearDownDecoders: reader=0x7fe21fbc0000 shut down video task queue
[DEBUG SHUTDOWN] ShutdownDecoder: decoder: 'vorbis audio decoder' (0x7fe21fabe840) flush:1
[DEBUG SHUTDOWN] Shutdown: reader=0x7fe2207a3000 shutdown demuxer=0x7fe21fb7eae0
[DEBUG SHUTDOWN] Shutdown: pool=0x7fe2219e60d0 count=2
[DEBUG SHUTDOWN] ShutdownDecoder: decoder: 'vorbis audio decoder' (0x7fe21f6f6ae0) flush:1
[DEBUG SHUTDOWN] Shutdown: reader=0x7fe20e147000 shutdown demuxer=0x7fe21fb7e720
[DEBUG SHUTDOWN] Shutdown: pool=0x7fe2219c9910 count=2
[DEBUG SHUTDOWN] operator(): pool=0x7fe2219e60d0 shutdown=true count=1
[DEBUG SHUTDOWN] operator(): pool=0x7fe2219e60d0 shutdown=true count=0
[DEBUG SHUTDOWN] TearDownDecoders: reader=0x7fe2207a3000 shut down audio task queue
[DEBUG SHUTDOWN] TearDownDecoders: reader=0x7fe2207a3000 shut down video task queue
[DEBUG SHUTDOWN] operator(): pool=0x7fe2219c9910 shutdown=true count=1
[DEBUG SHUTDOWN] operator(): pool=0x7fe2219c9910 shutdown=true count=0
[DEBUG SHUTDOWN] TearDownDecoders: reader=0x7fe20e147000 shut down audio task queue
[DEBUG SHUTDOWN] TearDownDecoders: reader=0x7fe20e147000 shut down video task queue
[DEBUG SHUTDOWN] Shutdown: decoder=0x7fe21f686200 state machine=0x7fe21f666800
[DEBUG SHUTDOWN] Enter: state machine=0x7fe20e0f9070 reader=0x7fe21fb56ee0
[DEBUG SHUTDOWN] Shutdown: reader=0x7fe21e8e0000 shutdown demuxer=0x7fe21f6a5780
[DEBUG SHUTDOWN] Shutdown: pool=0x7fe21f6a02e0 count=1
[DEBUG SHUTDOWN] operator(): pool=0x7fe21f6a02e0 shutdown=true count=0
[DEBUG SHUTDOWN] TearDownDecoders: reader=0x7fe21e8e0000 shut down audio task queue
[DEBUG SHUTDOWN] TearDownDecoders: reader=0x7fe21e8e0000 shut down video task queue
[DEBUG SHUTDOWN] FinishShutdown: state machine=0x7fe21f666800
[DEBUG SHUTDOWN] Unregister: decoder=0x7fe21f686200, count=1

[...]

Comment 12

a year ago
so is it only me or did anyone had success using a headset in WebVR with this version ?

Comment 13

7 months ago
Hi, wondering if there was any update on this?  I'm trying to test WebVR tools using Nightly 63.0a1 in Ubuntu 16.04, and when I enter VR in any WebVR app it tracks headset motion and shows the split display on the screen, but no display is sent to the HMD.  SteamVR's status shows as "(unresponsive) firefox".  Firefox is using my GPU, I have the VR flags enabled in about:config and Steam-based apps project to the HMD.  Are there any logs that would be useful for you to see to help track down this issue?

Comment 14

7 months ago
I believe in the current state it can still not work, but an actual update is that since recently someone is officially working for Mozilla on it: https://twitter.com/beVR_nz/status/1015714068496199681
(Assignee)

Comment 15

7 months ago
(In reply to haagch+ff from comment #14)
> I believe in the current state it can still not work, but an actual update
> is that since recently someone is officially working for Mozilla on it:
> https://twitter.com/beVR_nz/status/1015714068496199681

Indeed!  I am excited to share that we have some great people dedicated to this, starting immediately.  Stay tuned for updates.

Comment 16

5 months ago
any updates ?
(Assignee)

Comment 17

5 months ago
(In reply to ol from comment #16)
> any updates ?
We are still fully committed to standing this up on Linux -- this is in active development.  There has been a breakthrough recently, enabling Linux VR and compositing to be accelerated without pulling textures back through the CPU in SharedSurfaceBasic.  This is fundamental to enabling the VR textures to be submitted to OpenVR.

Please stay tuned for updates!
(Assignee)

Comment 18

2 months ago
Created attachment 9032560 [details] [diff] [review]
WIP work from Pete, unbitrotted, rebased and reformatted for new code style
Attachment #8943422 - Attachment is obsolete: true

Comment 19

a month ago

(In reply to :kip (Kearwood Gilbert) from comment #18)

Created attachment 9032560 [details] [diff] [review]
WIP work from Pete, unbitrotted, rebased and reformatted for new code style

This is awesome, but being a noob i was wondering if I can use this patch in my computer or if I need to wait for the release on Firefox Nightly?

Flags: needinfo?(cristian.comorasu)
Flags: needinfo?(cristian.comorasu)
(Assignee)

Comment 20

25 days ago

(In reply to ben g from comment #19)

(In reply to :kip (Kearwood Gilbert) from comment #18)

Created attachment 9032560 [details] [diff] [review]
WIP work from Pete, unbitrotted, rebased and reformatted for new code style

This is awesome, but being a noob i was wondering if I can use this patch in my computer or if I need to wait for the release on Firefox Nightly?

You can certainly experiment -- I would love to hear about successes and/or failures along with your particular configuration. (especially GPU driver version)

The browser is expected to crash after a VR session is complete with the current patch, so not quite yet production ready.

The latest graphics drivers, with Vulkan support, need to be installed.

Some prefs will need to be set as well:
dom.vr.process.enabled=true
dom.vr.external.enabled=true
dom.vr.openvr.enabled=true
gfx.use-vulkan-allocator=true

Expect to see more updates to this patch soon.

You need to log in before you can comment on or make changes to this bug.