Closed Bug 1320616 Opened 3 years ago Closed 2 years ago

[webvr, HTC Vive] Scene moves as headset rotates

Categories

(Core :: WebVR, defect, P3)

53 Branch
defect

Tracking

()

VERIFIED FIXED
mozilla63
Tracking Status
firefox-esr52 --- unaffected
firefox-esr60 --- wontfix
firefox61 --- wontfix
firefox62 --- fixed
firefox63 --- fixed

People

(Reporter: agentme49, Assigned: kip)

References

Details

(Keywords: regression, steps-wanted, Whiteboard: [gfx-noted][webvr])

Attachments

(2 files)

User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0
Build ID: 20161104212021

Steps to reproduce:

When using any webvr site with the HTC Vive, as I rotate the headset, the entire scene shifts in position slightly. The effect is a bit disorienting, though hard to pin down if it weren't for the chaperone bounds. The chaperone bounds are rendered by SteamVR and stay still, so the movement of the scene is obvious relative to the chaperone bounds.

The scene only moves relative to the chaperone bounds while the headset is rotated. Moving the headset while keeping the rotation static doesn't cause the scene to move relative to the chaperone bounds. The position of the scene seems to be tied to the specific angle the headset is pointed at: rotating the headset and returning to the same angle puts the scene back into the same place.

Chrome appears to have the same bug, which I've reported here with screenshots: https://github.com/toji/chrome-webvr-issues/issues/73#issuecomment-263172110 . I couldn't get the chaperone bounds to appear in screenshots while using webvr in Firefox, but the issue appears identical in appearance and behavior to me.

Using Firefox Nightly 53.0a1 (2016-11-27) (64-bit)
Component: General → Graphics
Hi Daosheng,
Could you please have a look to see what's going on? Thanks
Flags: needinfo?(dmu)
Priority: -- → P3
Whiteboard: [gfx-noted]
A possibly useful comment was made by klausw on the Chrome issue:

>Is WebVR using the GetEyeToHeadTransform matrices from OpenVR? If those include 3D offsets from the HMD origin, it would be wrong to just apply a 1D +/- IPD adjustment. The symptoms seem to match a wrong head rotation origin.
I think it is more like the janky framerate makes the chaperone bounds mismatch the origin. We have GetEyeToHeadTransform(), that is for helping us compute the left/right eyes translation from the middle of the viewport.
Flags: needinfo?(dmu)
FWIW

This sounds very much similar to the issue that Chromium experiences with tracking:
https://github.com/toji/chrome-webvr-issues/issues/73

I haven't myself experienced the issue in Firefox however.
Whiteboard: [gfx-noted] → [gfx-noted][webvr]
Component: Graphics → WebVR
I am currently experiencing this issue with Nightly Version 56.0a1 using a Vive and Windows 10.

Models I have uploaded to sketchfab.com which were displaying correctly 4 weeks ago and are now jittery and follow head movement before finding their correct position after a delay.

Let me know if you require anymore details.
Assignee: nobody → kgilbert
(In reply to Steve B from comment #5)
> I am currently experiencing this issue with Nightly Version 56.0a1 using a
> Vive and Windows 10.
> 
> Models I have uploaded to sketchfab.com which were displaying correctly 4
> weeks ago and are now jittery and follow head movement before finding their
> correct position after a delay.
> 
> Let me know if you require anymore details.

Thanks for reporting this.  We have recently noticed the same regression and are investigating under Bug 1394561.  Once we have fixed it there, I'll comment here to see if you can still repro it.  Hopefully this is the same issue.
See Also: → 1394561
Could you please re-test with the latest nightly, now that Bug 1394561 has landed?  Hopefully the fix in that bug also helps for you.
Flags: needinfo?(agentme49)
Hiya just updated Nightly and it appears that this issue is still present for me. I'm on Windows 10, HTC Vive, GTX1080 w/latest firmware). I tried SteamVR and SteamVR beta.

To verify, I opened some of the webvr.info & a-frame.io examples. The chaperone bounds are rock solid, but the WebVR content lags behind / swims around. It is subtle but definitely contributes to nausea and discomfort in the space, breaking presence.

The easiest way to spot this is to hit the home button, so that the Steam Menu and controller models appear (with the WebVR content still-rendered, just greyed out). From there, make a slight head waggle.. the Steam Menu + controllers stay firmly planted / rock solid in the space, whereas the WebVR content swims around. I can't quite tell if it is lag or alignment.

Perhaps there is something off with my configuration as I too have experienced this with Chromium's WebVR build (I don't think I can try Edge as it doesn't seem to recognize Steam w/Vive, maybe it needs a Windows Mixed Reality unit).  

I do believe the other issue is fixed.. a few days ago, if I opened the a-painter example on a-frame.io with its Fox drawing, I was seeing low frame rate and as a result major major lag (I'd have to wait 5-10 seconds for the controllers to catch up to my hand). Now it is just frame drop & rate instability, but I don't have a slow motion queue of frames to wait to catch up. The a-painter Fox drawing has never run well for me — maybe this is useful to know as well? (If I clear that drawing I get stable 90 FPS in a-painter until I add more stroked).

Thank you for your work on all this, I hope this issue can get tracked down. It feels like one of the few remaining hurdles for me to spend a lot of time making WebVR content!
(In reply to chrisnovello from comment #8)
> Hiya just updated Nightly and it appears that this issue is still present
> for me. I'm on Windows 10, HTC Vive, GTX1080 w/latest firmware). I tried
> SteamVR and SteamVR beta.
> 
> To verify, I opened some of the webvr.info & a-frame.io examples. The
> chaperone bounds are rock solid, but the WebVR content lags behind / swims
> around. It is subtle but definitely contributes to nausea and discomfort in
> the space, breaking presence.
> 
> The easiest way to spot this is to hit the home button, so that the Steam
> Menu and controller models appear (with the WebVR content still-rendered,
> just greyed out). From there, make a slight head waggle.. the Steam Menu +
> controllers stay firmly planted / rock solid in the space, whereas the WebVR
> content swims around. I can't quite tell if it is lag or alignment.
> 
> Perhaps there is something off with my configuration as I too have
> experienced this with Chromium's WebVR build (I don't think I can try Edge
> as it doesn't seem to recognize Steam w/Vive, maybe it needs a Windows Mixed
> Reality unit).  
> 
> I do believe the other issue is fixed.. a few days ago, if I opened the
> a-painter example on a-frame.io with its Fox drawing, I was seeing low frame
> rate and as a result major major lag (I'd have to wait 5-10 seconds for the
> controllers to catch up to my hand). Now it is just frame drop & rate
> instability, but I don't have a slow motion queue of frames to wait to catch
> up. The a-painter Fox drawing has never run well for me — maybe this is
> useful to know as well? (If I clear that drawing I get stable 90 FPS in
> a-painter until I add more stroked).
> 
> Thank you for your work on all this, I hope this issue can get tracked down.
> It feels like one of the few remaining hurdles for me to spend a lot of time
> making WebVR content!

Thanks for the detailed feedback.  I'll keep this open as a separate issue and continue investigation for FF58.
We're on GTX1080, Win 10/latest steam. This seems to have started in FF56, and is still around in Nightly as of 10/31/17. FF55.0.3 was fine.

To see this clearly you need chaperone and compare the scene to SteamVR's chaperone shader to see they don't align when you move. This also manifests loosely as user reports of disorientation or "buggy VR". It's like there's like a several frame lag between pose and presentation, as if firefox at some point added an extra buffer or delay to the rendering pipeline.

It's worth noting this also happens _sometimes_ on FF55.0.3, but much less rarely. It feels like there's a heuristic mode or desync that FF switches to sometimes, and after that the VR experience switches to lag mode.

It looks like it's _not_ a CPU or GPU performance issue; it's the same regardless of how light or heavy the render is, just Gecko is issuing VRDisplay RaFs at too low a rate (60 lock) or something. Most of the profile is in Gecko and even heavy scenes are only using like 30% of the budget and the JS itself is not missing frames, it's something in the core.

I couldn't find any settings to work around it, either in Steam or FF aboutconfig, other than using an older version of FF.
See Also: → 1419588
Status: UNCONFIRMED → NEW
Ever confirmed: true
The WebVR api was returning a headset pose predicted one additional frame in the
future, but the SteamVR async reprojection was reprojecting it using the
prior (correct) frame's pose.

This resulted in a sickness inducing swimming effect as well as deregistration
from the Vive chaperone bounds.
Comment on attachment 8997233 [details]
Bug 1320616 - Use the render pose rather than the gameplay pose for OpenVR headsets

Daosheng Mu[:daoshengmu] has approved the revision.

https://phabricator.services.mozilla.com/D2693
Attachment #8997233 - Flags: review+
Pushed by kgilbert@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/3e5c6876ab05
Use the render pose rather than the gameplay pose for OpenVR headsets r=daoshengmu
https://hg.mozilla.org/mozilla-central/rev/3e5c6876ab05
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla63
Kip, is this worth a Beta uplift request?
Flags: needinfo?(agentme49) → needinfo?(kgilbert)
(In reply to Ryan VanderMeulen [:RyanVM] from comment #15)
> Kip, is this worth a Beta uplift request?
Indeed - this is a simple, self contained fix that makes a large difference for end users.  I'll prepare an uplift request then cancel this NI.
Approval Request Comment
[Feature/Bug causing the regression]:
Bug 1218482 - Enable WebVR by default on all platforms
[User impact if declined]:
Most OpenVR devices (eg, HTC Vive), will continue to feel laggy / have an uncomfortable swimming effect when viewing WebVR content.
[Is this code covered by automated tests?]:
No - This specific line of code is used only when interfacing with physical hardware.
[Has the fix been verified in Nightly?]:
Yes
[Needs manual test from QE? If yes, steps to reproduce]: 
Optional; STR:
- Use an HTC Vive connected to a Windows PC
- Visit https://webvr.info/samples/05-room-scale.html
- Click the "Enter VR" button
- Walk towards the corner of the stage area until the Chaperone bounds appears in the headset.
- Turn your head from side-to-side
- Notice that the Chaperone bounds stay registered with the rendered cubes with the patch applied.  Without the patch applied, they appear to "float" or "swim" separately.
[List of other uplifts needed for the feature/fix]:
None
[Is the change risky?]:
No
[Why is the change risky/not risky?]:
This is a single line change, which is used only when using OpenVR based devices on a WebVR site.
[String changes made/needed]:
None
Flags: needinfo?(kgilbert)
Attachment #8999696 - Flags: approval-mozilla-beta?
Comment on attachment 8999696 [details] [diff] [review]
Bug 1320616 - (Beta Uplift) Use the render pose rather than the gameplay pose for OpenVR headsets

WebVR only issue; sounds like the impact for affected users could be high. 
Let's uplift for beta 18.
Attachment #8999696 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
I have tested this fix today and from the looks of things it seems to be fixed. I'm not very sure about it, though, because the only visual representation I have to go on are the 2 screenshots from 2 years ago on github. 
https://github.com/toji/chrome-webvr-issues/issues/73
The edges of the chaperone bounds seem to be fixed now, although sometimes there are some differences when viewing them from different angles, very small if at all. I need a second opinion on this one. Is there anyone that could help test this? 
Also I was unable to make some screenshots of my own to compare them. The "ctrl + f12" combo doesn't work and neither does the "system button + trigger" combo on the controller. 
I'm inclined to mark this as verified fixed but, as I've said, I'd need another pair of eyes just to be sure. 
Chris, could you take a look?
Flags: needinfo?(agentme49)
CC'ing Casey who also has a good idea for visually discerning this bug. See comment 20 above. Thanks!
Flags: needinfo?(caseyyee.ca)
Flags: qe-verify+
This seems fixed now. Love it when we have a long-standing issue that has finally been resolved :D
Flags: needinfo?(caseyyee.ca)
changing status based on comment #22 and my personal assessment in comment #20.
Status: RESOLVED → VERIFIED
Flags: needinfo?(agentme49)
You need to log in before you can comment on or make changes to this bug.