1239055 - crash in libovrrt64_0_8.dll@0x25b24

Status: RESOLVED FIXED

(Core :: Graphics, defect)

Description

[Tracy Walker [:tracy]]

[Tracking Requested - why for this release]:

This bug was filed from the Socorro interface and is 
report bp-5f39f238-8cf5-454e-a66e-0dbb02160112.
=============================================================

I am not sure where this belongs. I think it has something to do with Virtual Reality.

@mozillawebvrenabler

Anyway, it has quickly jumped to #1 on Nightly topcrash list.

Comment 1

[Liz Henry (:lizzard) (relman/hg->git project)]

The sky above crash-stats was the color of television, tuned to a dead channel. "Black ice!" Socorro screamed. "Black ice!"

Tracking for 46 since this is a top crash.

Comment 2

[Liz Henry (:lizzard) (relman/hg->git project)]

Kyle adds from crash-debug list that this is crashing in the Oculus Rift driver.

Comment 3

[Nathan Froyd [:froydnj]]

I think Kip did some of the work here, perhaps he has ideas.

Comment 4

[Andrew McCreight [:mccr8]]

Here are some crash reports with less junky looking stacks than the one in comment 0:
b4c2ae4c-d156-4336-8140-acc272160113
97e0ecaf-e44e-4a97-a07c-23e7c2160112

Comment 5

[Andrew McCreight [:mccr8]]

Hmm let's use a full URL instead, I guess.
https://crash-stats.mozilla.com/report/index/b4c2ae4c-d156-4336-8140-acc272160113
https://crash-stats.mozilla.com/report/index/97e0ecaf-e44e-4a97-a07c-23e7c2160112

Comment 6

[Liz Henry (:lizzard) (relman/hg->git project)]

Kyle suggested Josh Carpenter may be working in this area. Josh, want to take a look?

Comment 7

[:kip (Kearwood Gilbert)]

libovr is the "Oculus Runtime" -- a library that we don't distribute with Firefox, but use when enumerating and using Oculus VR devices.  This crash would only be possible if someone is using WebVR.  I'll take this bug and investigate reasons that we might be causing the libovr to crash.  This may possibly be crashes occurring during our own testing.

Comment 8

[Liz Henry (:lizzard) (relman/hg->git project)]

Kip, what do you think here? Does this block the feature from releasing with 46? If it's something we're running into while testing, I don't think I need to track it. Also noticing it's showing up more on Nightly (maybe confirmation of your "developers testing this out" theory) and hardly at all for 46.

Comment 9

[Liz Henry (:lizzard) (relman/hg->git project)]

Maybe make it block one of the webvr meta bugs. bug 1218482 ?

Comment 10

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Kip, it's been a while and this crash still shows up in the top 5 crashes by volume on Nightly47. Do you have any insights into a possible reason and a fix that we could try out? Thanks!

Comment 11

[:kip (Kearwood Gilbert)]

I am looking into this issue further today.  I suspect that the crash may be related to threading with the Oculus runtime.  Something may have regressed, causing inconsistency in which threads were used for Oculus API access, in particular the shutdown.

Comment 12

[Cornel Ionce [:noni] [Hubs QA]]

I'm able to reproduce this crash constantly on Windows 10x 64 using my nVIDIA GeForce 620 machine everytime I try to enter VR with any of the mozvr.com examples.
Reproduced with latest 46.0a2 Aurora and with 47.0a1 Nightly.
Here are some  crash reports:
bp-29dd8bb2-e74b-402c-a898-b4dca2160304
bp-df94ffd3-432a-408c-99ae-260392160304
bp-259f21b7-1b78-4662-812c-971852160304
bp-35896f17-1512-488a-9c04-28fdc2160304
bp-633aa467-56b2-455e-bd14-0fa5b2160304
bp-503cbdcf-1509-4775-93b9-4f64b2160304
bp-6927c269-140f-4b6d-85a4-564962160304
bp-34beb71f-a9de-4185-80ea-b9dfd2160304
bp-ceb748ae-5d1f-4e8d-bf4b-0451b2160304
bp-d035bd45-2449-4c8b-9806-750382160304
bp-5eb943c6-1688-480d-804c-e5cfe2160304
bp-ca95cc29-116c-4f63-858e-817382160304
bp-78b8b870-4847-4daf-88a4-6f9272160304

Using a Windows 10 x64 computer with a AMD r7 360 GPU also rans into this crash, but I'm able to enter VR. The crash occurs when exiting/closing the VR.
bp-5dc72ba1-58b0-4b28-a334-3c04f2160304
bp-cf5442d4-7ad4-457d-bcfe-e01432160304

I'm reproducing it with both e10s enabled/disabled.

Testing was performed with a Oculus DK2 and 0.8.0.0 runtime.

This is a major impact issue and it severely affects the WebVR feature functionality.
Please let me know if I can provide further information here in order to get this fixed ASAP.

Comment 13

[:kip (Kearwood Gilbert)]

Thanks Cornel,

I am landing to other bugs that are likely to affect this issue, 1239055 and 1251886.  Would you be able to re-test this one once those land?

Comment 14

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

(In reply to :kip (Kearwood Gilbert) from comment #13)
> Thanks Cornel,
> 
> I am landing to other bugs that are likely to affect this issue, 1239055 and
> 1251886.  Would you be able to re-test this one once those land?

I looked at the crash signature https://crash-stats.mozilla.com/signature/?product=Firefox&version=47.0a1&date=%3C2016-03-08T22%3A18%3A48&date=%3E%3D2016-03-01T22%3A18%3A48&signature=libovrrt64_0_8.dll%400x25b24&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&page=1#reports and noticed that the # of hits has dramatically reduced and there is only one instance of it build 03-06 build. This is good and bad. Good because it's not seen on recent builds, bad because it still showed up once on 03-06 build (which has the fix in bug 1251886).

Comment 15

[Cornel Ionce [:noni] [Hubs QA]]

Still running into this crash every single time when trying to enter VR.
Tested with latest 48.0a1 Nightly and 47.0a2 Aurora.

Here are a few crash reports:
bp-d307829f-9682-4f3d-bc78-68a072160314
bp-04679196-9515-4dbe-a7ab-9376e2160314
bp-30136ba6-d02e-4565-ab4a-3ffcc2160314
bp-da3c2c68-9841-4bea-b087-4c7f42160314
bp-fa82dd2a-a19d-4621-939e-9a71e2160314
bp-f67bfe48-674e-44c3-8110-12ab32160314

Comment 16

[Milan Sreckovic [:milan] (needinfo for best results)]

[Tracking Requested - why for this release]:

Everything I'm hearing suggests this only happens when people use VR.  Don't think we need to track this.

Comment 17

[Liz Henry (:lizzard) (relman/hg->git project)]

Thanks Milan, I don't think we are shipping WebVR in 46 and I don't need to track it. Looks like kip has this already on his radar.

Comment 18

[:kip (Kearwood Gilbert)]

The release version of the Oculus runtime is now public, so these stack traces will not be seen again once Bug 1260530 lands (Implement Oculus 1.3 runtime support).

If there are still similar issues with the release version of the Oculus runtime, "libovrrt64_0_8.dll" will be replaced with "libovrrt64_1.dll".

Also, we may need to update the test plans for SoftVision, as once Bug 1260530 lands they will need to update their Oculus runtime to the release version that involves different UX.

Comment 19

[Tracy Walker [:tracy]]

I think you want Cornel for the SV contact

Comment 20

[Cornel Ionce [:noni] [Hubs QA]]

Thanks for the heads-up Tracy! I've put bug 1260530 on my radar.

Comment 21

[David Anderson [:dvander] - inactive, e-mail if emergency]

(In reply to Cornel Ionce [QA] from comment #20)
> Thanks for the heads-up Tracy! I've put bug 1260530 on my radar.

This looks pretty low-volume, and it looks like there are no new crashes in 48 after bug 1260530. Safe to mark this as fixed?

Comment 22

[Cornel Ionce [:noni] [Hubs QA]]

I assume it's now safe to mark this fix. I've tried several times to obtain this signature these days and it didn't happen. Tested on the same machines where the crash occured.

Comment 23

[:kip (Kearwood Gilbert)]

It will no longer be possible to hit this stack trace, as the version of the Oculus runtime API is included in the .dll name and we no longer attempt to load the 0.8 runtime version which was involved with these crashes.

Marking as fixed...