1481327 - Handle VRService shmem between GPU and VR process

Status: RESOLVED FIXED

(Core :: WebVR, enhancement)

Description

[Daosheng Mu[:daoshengmu]]

In Bug 1430038, I was trying to open memory in GPU process, and duplicate its memory handle by VR process handle. But, I always get `ERROR_INVALID_HANDLE` in Windows.

VR Service process:
OpenFileMapping() -> Get a HANDLE for the virtual file ... Perhaps, we can try tto use CreateFileMapping()?
MapViewOfFile() -> Get a pointer to the memory backed by the file
DuplicateHandle() -> Get a HANDLE for the other process to use

GPU process:
MapViewOfFile() -> Get a pointer to the memory shared by the other process by the HANDLE from the other process.

When each process is done, they call CloseHandle() on their own HANDLE.

------

We need to figure out the reason, and give a cross-platform solution.

Comment 1

[Daosheng Mu[:daoshengmu]]

Attachment: Bug 1481327 - Handle VRService Shmem between GPU and VR process.

MozReview-Commit-ID: CdyIFim7zE1

Comment 2

[Daosheng Mu[:daoshengmu]]

We shouldn't let VRManager::ManagerInit() be called in the parent process, https://dxr.mozilla.org/mozilla-central/rev/4e56a2f51ad739ca52046723448f3129a58f1666/gfx/thebes/gfxPlatform.cpp#511, is it because we still wanna support non-e10s mode? At least, we should check if it is XRE_MultiProcess().

Comment 3

[Daosheng Mu[:daoshengmu]]

I follow the Comment 0 to make CreateFileMapping(), then DuplicateHandle from GPU process to VR process.

But, it always gets ERROR_INVALID_HANDLE from GetLastError();

Comment 4

[Daosheng Mu[:daoshengmu]]

Attachment: Bug 1481327 - Make sure only named shared memory works, duplicate handle does not;

MozReview-Commit-ID: 9xkrl271uNe

Comment 5

[Daosheng Mu[:daoshengmu]]

We find only the named shared memory mechanism works to get the file mapping. Duplicate handle approach always gets failed in multiple processes. I would like to use named shared memory mechanism to confirm the VR functionalities, then will come back to investigate it again.

Comment 6

[Daosheng Mu[:daoshengmu]]

Doing debug about WebVR reftest get stuck in try servers. I was trying to do this in local, but SteamVR runtime always pop-up and the tests can't be execute properly. After uninstalling the SteamVR, the tests work for the m-c build. Continue to investigating the WebVR refest issue after applying my patches.

Comment 7

[Daosheng Mu[:daoshengmu]]

Attachment: Bug 1481327 - Part 1: Launch VR service in VR process.

MozReview-Commit-ID: DA300lYAuC1

Comment 8

[Daosheng Mu[:daoshengmu]]

Attachment: Bug 1481327 - Part 2: Handle VR process shared memory.

MozReview-Commit-ID: Hs4sluLoYBn

Comment 9

[Daosheng Mu[:daoshengmu]]

Attachment: Bug 1481327 - Part 1: Launch VR service in VR process.

MozReview-Commit-ID: DA300lYAuC1

Comment 10

[Daosheng Mu[:daoshengmu]]

Try looks good, https://treeherder.mozilla.org/#/jobs?repo=try&revision=ac3dada2e2d3164d57aa9084600de668d94ab790, although reftest in Windows try server are stuck, but the latest m-c build also gets this stuck. Anyway, I have verified WebVR reftest in local, they are good.

Comment 11

[:kip (Kearwood Gilbert)]

Comment on attachment 9001526 [details]
Bug 1481327 - Part 1: Launch VR service in VR process.

:kip (Kearwood Gilbert) has approved the revision.

Comment 12

[:kip (Kearwood Gilbert)]

Comment on attachment 9001521 [details]
Bug 1481327 - Part 2: Handle VR process shared memory.

:kip (Kearwood Gilbert) has approved the revision.

Comment 13

[Andrew McCreight [:mccr8]]

Comment on attachment 9001526 [details]
Bug 1481327 - Part 1: Launch VR service in VR process.

Andrew McCreight [:mccr8] has approved the revision.

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/17e156ebb40a
Part 1: Launch VR service in VR process., r=kip,r=mccr8
https://hg.mozilla.org/integration/mozilla-inbound/rev/58a3875a8cb1
Part 2: Handle VR process shared memory.,r=kip

Comment 15

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/17e156ebb40a
https://hg.mozilla.org/mozilla-central/rev/58a3875a8cb1

Comment 16

[Daosheng Mu[:daoshengmu]]

I think I have figured out how to use the duplicate handle approach in Windows. We need to call CreateFileMappingA() with NULL string id at the parent process, then must call DuplicateHandle() at the parent process instead of at the child process. Following, forward the duplicate handle to the child process to ask the child process call MapViewOfFile() with the duplicate handle.