1771854 - GPU process broke WebXR support

Status: RESOLVED WONTFIX

(Core :: WebVR, defect)

Description

[Sergio Villar]

Steps to reproduce:

Wolvic browser was using Gecko from January 2022. I proceed to rebase it against the current top of the tree.

Actual results:

WebXR content stopped working. In particular Gecko is not able to list the current runtimes so the isSessionSupported promise always return false

Expected results:

WebXR content should work as expected

Comment 1

[Sergio Villar]

I bisected the issue to this

    Bug 1331109 - Enable GPU process on Android. r=gfx-reviewers,aosmond
    
    Differential Revision: https://phabricator.services.mozilla.com/D138144

enabling the GPU process in Android by default broke the WebXR content. I've verified that setting layers.gpu-process.enabled=false in Wolvic when creating the GeckoView does indeed fix the issue.

Comment 2

[Jamie Nicol [:jnicol]]

I'm afraid I'm not at all familiar with webXR. Can you expand on what you mean by "Gecko is not able to list the current runtimes"?

Is there anything useful in the logcat at all?

Comment 3

[Sergio Villar]

(In reply to Jamie Nicol [:jnicol] from comment #2)

I'm afraid I'm not at all familiar with webXR. Can you expand on what you mean by "Gecko is not able to list the current runtimes"?

Is there anything useful in the logcat at all?

Sure. So WebXR requires a platform component or runtime that talks to the XR devices. There are several SDKs in the market, almost each vendor has its own: Oculus, Pico, HTC... and there is also a "universal" and standard one called OpenXR which is under the umbrella of Khronos.

The first thing a WebXR experience should do is to request a XR session. Before doing that developers can check whether or not a given session is supported. That is done with isSessionSupported call https://searchfox.org/mozilla-central/source/dom/vr/XRSystem.cpp#100. As you can see the first thing it needs to do is to check which xr runtimes are available in the system by calling VRManagerChild::DetectRuntimes() https://searchfox.org/mozilla-central/source/dom/vr/XRSystem.cpp#115. We should get notified about that call later in XRSystem::ResolveIsSessionSupportedRequests https://searchfox.org/mozilla-central/source/dom/vr/XRSystem.cpp#386. I didn't have time to double check whether the problem is that that method is not called or that vm->RuntimeSupportsVR() returns always false but it should be pretty easy to verify.

I insist this works fine when GPU process is disabled, so perhaps the issue here is that there is some missing IPC that should be implemented.

Comment 4

[Jamie Nicol [:jnicol]]

Thanks. Yeah some missing IPC certainly seems plausible, or perhaps something just isn't being initialized in the GPU process that needs to be.

Is it possible to debug this in geckoview_example or does it need wolvic? And without any XR hardware?

Comment 5

[Sergio Villar]

(In reply to Jamie Nicol [:jnicol] from comment #4)

Thanks. Yeah some missing IPC certainly seems plausible, or perhaps something just isn't being initialized in the GPU process that needs to be.

Is it possible to debug this in geckoview_example or does it need wolvic? And without any XR hardware?

Really good questions, I guess you don't need Wolvic for this. Just visit any of the examples in https://immersive-web.github.io/webxr-samples/ for example (perhaps you'd have to enable dom.vr.external.enable: true and dom.vr.webxr.enabled: true first). If there is any issue then you wouldn't get the "Enter VR" button.

WRT XR hardware if the problem is a missing IPC I guess you won't need any either, after implementing the required IPC you'd just get an empty list of devices and the code will report that there is no WebXR support. But if the problem is in some other place then perhaps you'd need some, not sure.

Comment 6

[Jamie Nicol [:jnicol]]

It looks like the IPC around VRManager and friends is working as intended. When running in geckoview_example with those prefs enabled (I couldn't see a dom.vr.external.enable, did you mean dom.vr.enabled?) VRManager::DetectRuntimes gets called in the GPU process, but the VrShMem::CreateShMemForAndroid call here fails because GeckoVRManager::GetExternalContext() returns null.

This also returns null for me when the GPU process is disabled, presumably because I'm running in geckoview_example as opposed to wolvic, and nothing in geckoview_example calls setExternalContext() . But I'm guessing that your problem is caused because Wolvic is currently loading this "external VR context" thing and calling setExternalContext() in the parent process. So perhaps we need to make GeckoVRManager use shared memory so that it can be set in the parent process and used from the GPU process?

Does that sound plausible, or am I barking up the wrong tree? Could you expand a bit upon what the external context does?

Comment 7

[Sergio Villar]

(In reply to Jamie Nicol [:jnicol] from comment #6)

It looks like the IPC around VRManager and friends is working as intended. When running in geckoview_example with those prefs enabled (I couldn't see a dom.vr.external.enable, did you mean dom.vr.enabled?) VRManager::DetectRuntimes gets called in the GPU process, but the VrShMem::CreateShMemForAndroid call here fails because GeckoVRManager::GetExternalContext() returns null.

Yeah, the dom.vr.external.enable was perhaps deprecated at some point and it isn't needed (dom.vr.enabled is a different thing, that's for enabling the deprecated WebVR). We should indeed remove it.

This also returns null for me when the GPU process is disabled, presumably because I'm running in geckoview_example as opposed to wolvic, and nothing in geckoview_example calls setExternalContext() . But I'm guessing that your problem is caused because Wolvic is currently loading this "external VR context" thing and calling setExternalContext() in the parent process. So perhaps we need to make GeckoVRManager use shared memory so that it can be set in the parent process and used from the GPU process?

Does that sound plausible, or am I barking up the wrong tree? Could you expand a bit upon what the external context does?

Yeah we do set a external context in Wolvic. Basically Wolvic talks to gecko via the GeckoView interface. In particular the API that is used is https://searchfox.org/mozilla-central/source/gfx/vr/external_api/moz_external_vr.h.

Comment 8

[Jamie Nicol [:jnicol]]

Can you link me to where wolvic uses this API? (In the wolvic source code)

Comment 9

[Sergio Villar]

(In reply to Jamie Nicol [:jnicol] from comment #8)

Can you link me to where wolvic uses this API? (In the wolvic source code)

Sure we call it from here https://github.com/Igalia/wolvic/blob/d56fb5f3b30e562b02ab386954ba7a2971513343/app/src/common/gecko/com/igalia/wolvic/browser/api/impl/RuntimeImpl.java#L114

We reach that point after a sequence of calls which start here https://github.com/Igalia/wolvic/blob/d56fb5f3b30e562b02ab386954ba7a2971513343/app/src/main/cpp/BrowserWorld.cpp#L860

Comment 10

[Sergio Villar]

Jamie any further thoughts on the matter? Anything else I could help with?

Comment 11

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:jimm, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 12

[Sergio Villar]

May I ask why it was marked as Wontfix? I can provide any additional information that is required to fix it.

Comment 13

[Daniel Adams[:msub2]]

(In reply to Sergio Villar from comment #12)

May I ask why it was marked as Wontfix? I can provide any additional information that is required to fix it.

Many existing bug reports (and even recent ones such as https://bugzilla.mozilla.org/show_bug.cgi?id=1785869) in the WebVR module have been closed as WONTFIX, I would imagine because there is no one to help work on or maintain the module internally anymore and there haven't been plans for WebXR since unshipping WebVR https://groups.google.com/a/mozilla.org/g/dev-platform/c/S9ltYIfjCpg?pli=1.

Comment 14

[Jim Mathies [:jimm]]

(In reply to Daniel Adams from comment #13)

(In reply to Sergio Villar from comment #12)

May I ask why it was marked as Wontfix? I can provide any additional information that is required to fix it.

Many existing bug reports (and even recent ones such as https://bugzilla.mozilla.org/show_bug.cgi?id=1785869) in the WebVR module have been closed as WONTFIX, I would imagine because there is no one to help work on or maintain the module internally anymore and there haven't been plans for WebXR since unshipping WebVR https://groups.google.com/a/mozilla.org/g/dev-platform/c/S9ltYIfjCpg?pli=1.

This is correct. We may develop WebXR at some point but currently it is not on a roadmap. If and when we do we'll likely resurrect some of these wontfix bugs.

Also, contributors are welcome to work on this area of the code if they like, I'm sure we can find the bandwidth to do reviews.