1384279 - WebVR content does not present when using Oculus Home before launching Firefox

Status: VERIFIED FIXED

(Core :: WebVR, defect, P1)

Description

[Diego Marcos [:dmarcos]]

[Affected versions]:
 Fx 55.0b12

[Affected platforms]:
 Windows 10 x64 + Oculus Rift

[Steps to reproduce]:
 1. Close Oculus runtime if it's open
 2. Put the headset on (the Oculus Home will start automatically) or open the Oculus application manually and then put the headset on. Wait for the Oculus Home environment to start presenting.
 3. Open Firefox and load any WebVR content, for instance: https://webvr.info/samples/03-vr-presentation.html
 4. Click on Enter VR
 
[Expected result]:
Content presents in the headset.

[Actual result]:
The Oculus Home remains in the headset while in the browser I can see the WebVR content rendered with proper head tracking.

Comment 1

[Diego Marcos [:dmarcos]]

It also affects Firefox Nightly 56.0a1 (2017-07-25) (64-bit)

Comment 2

[Diego Marcos [:dmarcos]]

I changed dom.vr.oculus.quit.timeout to 0 but it does not seem to make any difference.

Comment 3

[Daosheng Mu[:daoshengmu]]

My dom.vr.oculus.quit.timeout is 30000. It happens to me as well. I click the enter vr button, but I would see a black image or still be at Oculus home at 56.0a1 (2017-07-25)

Comment 4

[:kip (Kearwood Gilbert)]

With an older NVidia driver (382.05), I was able to reproduce, but with a slightly different STR:

- Make sure no Firefox.exe or Nightly.exe processes are running
- Start Oculus Home
- Start Firefox
- Start a WebVR site (I used https://webvr.info/samples/04-simple-mirroring.html)
- Click the Enter VR button
- Close Oculus Home (Do not exit VR first)
- Start Oculus Home again (must be within 30 seconds)
- Refresh the page
- Page reports that no HMD's are present (We purposefully don't allow pages to start Oculus runtimes back up until 30 seconds after Oculus us shut down in the middle of WebVR, so we can allow the processes to close during Oculus automated runtime updates)
- Wait 30 seconds
- Refresh the page
- The headset is detected again
- Click "Enter VR"
- Expected behavior: Content is displayed in the headset
- Actual behavior: Content is not displayed in the headset, but the mirror image shows tracking.  Framerate is low at ~15 fps.  Connecting a debugger to the process indicates that the ovr_submitframe call is returning 1000, indicating that the content is hidden.

I haven't been able to repro this after updating to the latest NVidia driver (384.94 at this time)

Please note that the 15fps is due to the throttling mechanism in Firefox's WebVR implementation that kicks in when content is presenting but blurred / hidden by other things in the headset.  This is normal behavior if the content was actually hidden behind an interstitial such as the Oculus Health and Safety Warning or system menu.

Comment 5

[Daosheng Mu[:daoshengmu]]

[Affected versions]:
 FF 55.0b12 and 56.0a1 (2017-07-25)

[Affected platforms]:
 Windows 10 x64 + Oculus Rift. NVidia driver (384.94)

[Steps to reproduce]:
 1. Open Oculus runtime if it's closed.
 2. Put the headset on. Wait for the Oculus Home environment to start presenting.
 3. Open Firefox and load any WebVR content, for instance: https://aframe.io/a-painter/
 4. Click on Enter VR
 
[Expected result]:
Content presents in the headset.

[Actual result]:
The Oculus Home remains in the headset while in the browser I can see the WebVR content rendered with proper head tracking.


However, if I close the Oculus runtime, then opening FF and asking it go to a WebVR page. It works properly.

Comment 6

[Fernando Serrano [:fserrano]]

I was able to reproduce on 56.0a1 (2017-07-26), Windows 10 x64.
Both with Nvidia drivers 384.94 and older 382.66

[STR]
1. Put my headset, it will open Oculus home
2. Open Firefox and go to any WebVR content
3. Click Enter VR

[Expected result]:
Content presents in the headset

[Actual result]:
Oculus home still in the headset. While the browser seems to freeze and goes everything to white. I can't see anything and as I move the mouse around the browser's window the menu bar and bookmarks goes also white, but I can click stil click on the "invisible" items.

Comment 7

[Fernando Serrano [:fserrano]]

After changing "layers.mlgpu.dev-enabled" to false, I don't get the freeze to white, but the result is the same the webvr content doesn't show up on the headset and it's still presenting oculus home correctly.

Comment 8

[Fernando Serrano [:fserrano]]

I've been using the following steps to enter VR correctly from webvr content, but if the Oculus home is closed after that, it will block the page and the headset will go to black

[STR]
- Launch oculus home
- Launch firefox
- Close oculus home
- Enter VR
- (Oculus app will launch in the background but it won't present oculus home)
- (If you exit VR the headset will goes to black, if you enter VR again, it will show the webvr content as expected).
- Close oculus home

[Expected result]:
Content will keep presenting on the headset

[Actual result]:
Headset goes to black, and page freezes

Comment 9

[Daosheng Mu[:daoshengmu]]

It still happens to me intermittently.

Comment 10

[Daosheng Mu[:daoshengmu]]

I have checked ovr_SubmitFrame() always returns ovrSuccess even though we can't see the content in Oculus Rift. I am curious if after Bug 1381085 is landed, it could solve this problem.

Comment 11

[Daosheng Mu[:daoshengmu]]

It can be resolved by replacing the init flag, (ovrInit_RequestVersion | ovrInit_MixedRendering) with ovrInit_RequestVersion.

But, the drawback is after we exit from the present mode, our HMD will be black until closing Firefox. We still can use Touch back to Oculus Home but Firefox will be force to deactivate from Oculus.

Kip, do you think we should give a fix like this?

Comment 12

[:kip (Kearwood Gilbert)]

(In reply to Daosheng Mu[:daoshengmu] from comment #11)
> It can be resolved by replacing the init flag, (ovrInit_RequestVersion |
> ovrInit_MixedRendering) with ovrInit_RequestVersion.
> 
> But, the drawback is after we exit from the present mode, our HMD will be
> black until closing Firefox. We still can use Touch back to Oculus Home but
> Firefox will be force to deactivate from Oculus.
> 
> Kip, do you think we should give a fix like this?

We need to keep the ovrInit_MixedRendering flag to ensure that the Oculus runtime behaves as needed for the browser use case.  In particular, this prevents Oculus home and the Oculus compositor from spinning up when users hit a site that probes for webvr hardware.  We are working with Oculus on the issue and they are helping to investigate the issue with the missing/occluded presentations.  It's possible that an update to the Oculus runtime could correct the problem.

On our end, we should double-check that we are tearing down and re-creating the Oculus session every time the VR presentation is started and stopped.  Also, we need to double-check that the ovrInit_Invisible flag is set correctly each time the session is re-created.

I'll be sure to follow up with our contacts at Oculus now that the FF57 merge date activity is settling down.

Comment 13

[:kip (Kearwood Gilbert)]

I have sent details about our usage of the Oculus API to the Oculus SDK team for review and will continue exploring on our end.

Comment 14

[Daosheng Mu[:daoshengmu]]

It seems to be resolved after Bug 1410493 land, I just pull the latest m-c and can't reproduce it again...

Comment 15

[:kip (Kearwood Gilbert)]

(In reply to Daosheng Mu[:daoshengmu] from comment #14)
> It seems to be resolved after Bug 1410493 land, I just pull the latest m-c
> and can't reproduce it again...
It was harder to reproduce, but I was able to confirm it still happens after a few tries.  Once Bug 1411055 and Bug 1407423 have landed, I'll follow up again with Oculus.

Comment 16

[Liz Henry (:lizzard) (relman/hg->git project)]

Seems likely that 58 is also affected. Too late to fix in 57 though.

Comment 17

[:kip (Kearwood Gilbert)]

Oculus Dash has just been released to Beta so we can fine-tune the prefs and adjust the session handling code to work best in the latest runtime.

Comment 18

[:kip (Kearwood Gilbert)]

Attachment: Bug 1384279 - Oculus Rift Core 2.0 Adjustments

- Add pref to enable the ovrInit_Invisible flag for Oculus sessions, enabled by default.
- Ensure that the Oculus library is unloaded every time it is uninitialized,
  improving reliability of exiting and returning to WebVR.

Review commit: https://reviewboard.mozilla.org/r/206470/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/206470/

Comment 19

[Daosheng Mu[:daoshengmu]]

Comment on attachment 8935586 [details]
Bug 1384279 - Oculus Rift Core 2.0 Adjustments

https://reviewboard.mozilla.org/r/206470/#review212158

r=me. I am curious if it would fix the problem of Oculus Rift can't enter the present mode when Oculus home has already launched after these changes.

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/50a8780085a0
Oculus Rift Core 2.0 Adjustments r=daoshengmu

Comment 21

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/50a8780085a0

Comment 22

[Gerry Chang [:gchang]]

Too late for Beta58. Mark 58 won't fix.

Comment 23

[Cristian Comorasu, QA [:ccomorasu], Releae Desktop QA - Please contact [e-mail]]

Sorry for the long pause.
I verified this issue using Oculus rift on Fx 59.0b9 and Fx 60.0a1(build ID: 20180213100127). 
I can confirm this issue is no longer reproducible.

Comment 24

[CharlesVanNoland]

This issue seems to have come back from the dead. Content tracks in the mirror image in the browser window but Oculus Home is all that is shown in the headset.

Comment 25

[Cristian Comorasu, QA [:ccomorasu], Releae Desktop QA - Please contact [e-mail]]

I will verify this as soon as possible, I'll needinfo me to make sure I will not forget.

Comment 26

[Alain Bryden]

I'm also having this issue on the latest Firefox. No matter how I launch Oculus Home (in advance or automatically) any VR content presents in Firefox only (but with head tracking), and is not visible on the Oculus. Instead, Oculus home is displayed.

Comment 27

[Alain Bryden]

If it helps debug, here's what I see when I visit https://webvr.info/samples/00-hello-webvr.html


1 - Oculus VR HMD

    isConnected: true
    isPresenting: false
    capabilities:
        hasPosition: true
        hasOrientation: true
        hasExternalDisplay: true
        canPresent: true
        maxLayers: 1
    stageParameters:
        sittingToStandingTransform: [ 1.000, 0.000, 0.000, 0.000, 0.000, 1.000, 0.000, 0.000, 0.000, 0.000, 1.000, 0.000, 0.000, 1.675, 0.000, 1.000 ]
        sizeX: 2.275
        sizeZ: 1.575
    displayId: 1
    displayName: Oculus VR HMD
    depthNear: 0.010
    depthFar: 10000
    getFrameData():
        timestamp: 5867718.565
        leftProjectionMatrix: [ 1.200, 0.000, 0.000, 0.000, 0.000, 1.008, 0.000, 0.000,-0.147,-0.113,-1.000,-1.000, 0.000, 0.000,-0.010, 0.000 ]
        leftViewMatrix: [ 0.994, 0.098, 0.046, 0.000,-0.067, 0.892,-0.447, 0.000,-0.085, 0.442, 0.893, 0.000,-0.054, 0.682, 0.668, 1.000 ]
        rightProjectionMatrix: [ 1.200, 0.000, 0.000, 0.000, 0.000, 1.008, 0.000, 0.000, 0.147,-0.113,-1.000,-1.000, 0.000, 0.000,-0.010, 0.000 ]
        rightViewMatrix: [ 0.994, 0.098, 0.046, 0.000,-0.067, 0.892,-0.447, 0.000,-0.085, 0.442, 0.893, 0.000,-0.113, 0.682, 0.668, 1.000 ]
        pose:
            position: [-0.014,-0.315,-0.904 ]
            linearVelocity: [ 0.006, 0.005, 0.016 ]
            linearAcceleration: [-0.070, 0.093,-0.211 ]
            orientation: [-0.229,-0.034, 0.043,-0.972 ]
            angularVelocity: [ 0.035,-0.036, 0.013 ]
            angularAcceleration: [-4.490,-3.305,-3.567 ]
    getEyeParameters('left'):
        offset: [-0.030, 0.000, 0.000 ]
        fieldOfView:
            upDegrees: 41.359
            rightDegrees: 35.405
            downDegrees: 47.820
            leftDegrees: 43.704
        renderWidth: 1328
        renderHeight: 1584
    getEyeParameters('right'):
        offset: [ 0.030, 0.000, 0.000 ]
        fieldOfView:
            upDegrees: 41.359
            rightDegrees: 43.704
            downDegrees: 47.820
            leftDegrees: 35.405
        renderWidth: 1328
        renderHeight: 1584

Comment 28

[me]

(In reply to Alain Bryden from comment #26)
> I'm also having this issue on the latest Firefox. No matter how I launch
> Oculus Home (in advance or automatically) any VR content presents in Firefox
> only (but with head tracking), and is not visible on the Oculus. Instead,
> Oculus home is displayed.

Alain, this is definitely not a general problem, WebVR on Oculus Rift with Firefox 61.0.2 works fine for me. The two biggest gotchas are:

- Make sure you are running the 64 bit version of Firefox
- If you are on a laptop with a discrete graphics card, make absolutely sure Firefox is configured to use it. For NVIDIA cards, this in the NVIDIA Control Panel > Manage 3D Settings > Program Settings > CUDA - GPUs

Comment 29

[sadanoh]

I am also having this issue 
- open oculus home
- open browser (firefox version 62) with the sample page https://webvr.info/samples/04b-simple-mirroring-2.html
- in the rift all I see is the home page 

In the side of the rift homepage I see the icon of firefox after pressing it the screen become black for few seconds
and then I see the home page again.

I have the latest oculus version and nvidia gtx-1060 with version 416 (latest)

Comment 30

[Cristian Comorasu, QA [:ccomorasu], Releae Desktop QA - Please contact [e-mail]]

I tried to reproduce this issue with the following Firefox versions: 65.0b1, 64.0b3, 63.0 and 62.0.3, using nvidia gtx 1050 with version 416.36, but to no avail. Everything works as intended with both https://webvr.info/samples/04b-simple-mirroring-2.html and https://aframe.io .
Did you update the Oculus software?

Comment 31

[:kip (Kearwood Gilbert)]

In testing this, please note that the Oculus runtime won't allow apps (including Firefox) to interrupt your VR session if you are wearing the headset when it is started from the desktop.  If the proximity sensor is triggered (eg, you are wearing the headset) while pressing the "Enter VR" button on a WebVR site, the browser likely won't be able to present in the headset.

Please try taking off the headset while pressing the "Enter VR" button on a WebVR site, if this happens to be your case.

Comment 32

[jgaardsted]

I solved this by running firefox 'as administrator',
for me this was the difference between having this problem,
and not having it.

Comment 33

[Daosheng Mu[:daoshengmu]]

(In reply to jgaardsted from comment #32)

I solved this by running firefox 'as administrator',
for me this was the difference between having this problem,
and not having it.

Thanks :jgaardsted. I'm glad to have your update.